Competency D: Utilize Research and Assessment for Developing Programs4 responses
It is important to look at research and assessment when developing programs, establishing learning outcomes, and improving the services we provide (Schuh, Jones, & Harper, 2011). With no research or assessment, we are stumbling blindly in our work, without the needed information to do it well. Throughout the program, I have invested time and energy into research and assessment, so my work on the programs and services that I have had my hand in designing and making stronger is informed by fact and not feeling.
Since the first term in the CSSA program, we have examined surveys available to higher education professionals that gather data from all over the country, including the National Survey of Student Engagement (NSSE) and the Beginning College Survey of Student Engagement (BCSSE). The NSSE and the BCSSE collect information about college students and conduct surveys to assess whether or not students are fully engaged, spending quality time and energy on their college experience. Administered and evaluated by the Indiana University Center for Postsecondary Research, the surveys provide essential understanding of the level to which students report that they are engaged in and out of the classroom as seniors (NSSE), as well as the level that they expect to be engaged before beginning college as first year students (BCSSE). In order to get an understanding of the way that students currently experience our campus, I looked to the Oregon State University’s NSSE data for demographic information, high impact pedagogy usage, academic challenge and rigor, and student-faculty interaction, and compared this to peer institution data. The BCSSE taught me a lot about student expectations before coming to college, and that when those expectations are met, students will tend to be more successful. There are other national surveys that institutions use to get a broad picture of how students are interacting with campus, including the Cooperative Institutional Research Program (CIRP). Wherever I take a position, I will need to determine which surveys the institution I am working for uses so that I can get a sense of the big picture of the student body and the campus.
In addition to large scale surveys, smaller quantitative and qualitative research projects dealing with a specific type of program, service, or student population, can be immensely helpful to aid in making decisions about programming. When developing the Career Trail, a new program for Career Services, I first studied a program that my Assistant Director loved, which she heard about from Willamette University. I studied the website of their program, called “Career Roadmap”, and spoke to the designer of the program for several hours on his insights, including the purpose, audience, and data collection he was conducting. Although I modeled OSU’s basic program on the Willamette program, I knew that there were some aspects that would be different for a school of 23,000 students than for Willamette’s 3,500 students. I also knew that the focus on research at OSU would be a factor in our program differences. I researched other programs that were using career development activities to connect career to curriculum, and I looked at various self-assessment tools that could offer a quick list of careers to a student to begin researching. I researched the concept of the name, which has changed from Career Trek to Career Trail, because of the connotations and possibilities inherent in the imagery of each, as well as how those related to the culture of OSU. What I developed was a program informed by all of this research, as well as the input and desires of my colleagues for the program, that I believe will be used successfully in Career Services for many years to come.
Looking at the assessment data that we produce in Career Services after sitting in on several Student Affairs Assessment Council meetings, it seemed to me that most departments on campus were assessing the learning of their student workers, whereas we were only assessing the students we served. Carolyn Killefer, who heads up assessment for Career Services, and I spoke about the possibilities for assessing the learning of our student workers, called Career Assistants. To begin this process, we needed some formal learning outcomes that were measurable, we needed a tool that would measure those learning outcomes over time, and we wanted to show and be able to report on some growth in their learning. I took this project on and began by looking at the OSU’s Learning Goals for Graduates and relating some of our goals for learning to those overall goals. I then broke one of those goals down even further to two measurable learning outcomes, associated specifically with students using best practices in performing a drop-in resume or cover letter critique. I then designed a rubric to be used in an observation of each CA while they were giving a drop in critique that touched on the various best practices. I will be performing these observations early in spring term with the hope that this will provide a baseline from which we can measure growth and learning.
In our Research in Higher Education course, AHE 513, we were given the task of developing a research proposal that used a particular quantitative or qualitative research method in order to investigate a compelling question related to higher education. Since I had experience at Colorado State University as a sexual assault advocate and I found the training experience quite intense, I wanted to examine students who do this work with the goal of creating more successful and comprehensive training. This research project exposed me to research methods outside of my experience in order to explore how to research people and their lived experiences. I hope to continue to pursue this type of research in the future, whether formally or informally, to help me develop programs and improve the services that my department provides.
I know I will continue to look to research and assessment data in order to make strong choices for my department and my institution. But it is also important to evaluate what is best for the particular students and campus culture, and these may not always overlap entirely. There is so much research out there; it is incredible! I doubt I will ever encounter all that I should, and the impetus to want to move on a project, a service, a program, rather than spend more time investigating it, is strong. I may get pressure from colleagues to move more quickly on a project when I believe more research needs to be done. Having a foundation in theoretical models and continuing to stay up on the latest research and assessment data will help me act accordingly more quickly while taking research and assessment into account.