James Madison University

Behind the Scenes on Assessment Day

By: Jordan Pye
Posted February 25, 2011

PHOTO:JMU StudentsEach year on Assessment Day, the psychology department tests how much graduating seniors learned about their major.  During Spring 2010, a group of undergraduate psychology students had the unique opportunity to participate in research that helped to evaluate their peers in ways that might lead to revisions of the psychology major.

In Dr. Kenneth Barron’s inaugural capstone course on program evaluation, a student research team took charge of the department’s assessment process. This work earned the group an invitation to present their research at the national Best Practices in Program Assessment in Psychology conference Atlanta, GA. during October, 2011.

Representatives of university psychology programs nationwide attended this conference, sponsored annually by the Society for the Teaching of Psychology. On Oct. 8, department head Dr. Michael Stoloff joined Barron and students Marjorie Levinstein, Alexandra Marston, Donna Melchione, Haley Mertins, Oksana Naumenko and Andrew Swinson to present their project, “Creating a New Model to Involve Students in all Phases of Assessment.”

Junior Haley Mertins said that that their presentation at the Best Practices Conference was mostly a description of their class. “This was the first time that  students anywhere in the nation were involved in the assessment process, besides being the ones assessed.” Mertins reviewed literature on assessment and procedures during the research phase of the project, and said “It [felt] cool to put the names on professional journals to faces” at the conference, where she met some of the authors of articles that she had read.

Levinstein, a senior on the research team, agreed it was a great experience.  “Having professional psychologists praise the work you have done at a national conference was an amazing feeling,” she said. “I plan to get my Ph.D. and I will have to attend and present at more conferences, so having this experience will be very valuable in the future.”

The presentation received positive feedback according to Stoloff.  After a similar conference in 2004, Stoloff and a team of faculty wrote a chapter for a book on educational assessment challenges and practices in psychology, titled “Seven Goals for Effective Program Assessment.”

“Through program assessment at JMU we are trying to improve courses, instructional strategies and the overall curriculum of our program,” Stoloff said.

This class is an experiment in multitasking that could serve as a template for other departments or universities, Barron said. Faculty members don’t always have the time or resources to assess their program in addition to their teaching, research and service responsibilities. By turning the process into a class, students supplied the manpower and carried out the primary tasks of designing the tools for Assessment Day, and then collecting and analyzing results. Afterward, the students participated in an audit of the department’s current assessment procedures.

JMU’s Psychology Department bases its program assessment on the American Psychologist Association’s guidelines, “Undergraduate Psychology Major Learning Goals and Outcomes.” This document suggests ten goals that summarize the knowledge, skills and values that a psychology curriculum should incorporate within the context of a liberal arts education.

One example of a project conducted by this research team is an examination of the effectiveness of the Goal 8, to increase their socio-cultural and international awareness. This research team looked at how well this goal is being addressed since a curriculum change made in 2006.  Compared to surveys from 2006, the data from 2010 showed a 4 percent increase in the number of students who agreed or strongly agreed that they have achieved this goal, and a 23 percent increase in students who said JMU’s psychology program provided opportunities to achieve the goal.

From analyzing the Goal 8 data, Levinstein said she learned how the department uses assessment results to make changes, not just to collect statistics.

As a result of the findings of these students, the psychology department has recently started testing students enrolled in socio-cultural awareness classes both before and after their classes Levinstein said.

Marston, who took Barron’s program evaluation course as a sophomore, said the research experience made her confident about running different forms of data analysis, and changed her understanding of how assessments affect students in the long term.
“I had a fairly basic knowledge about what the purpose of assessment was, but I had no idea that our psychology department took into account the feedback from the assessment as it did,” Marston said. “It made me feel good about being involved in a project that was really going to help change the major for the better.”