A NEW APPROACH TO WRITING ASSESSMENT: WRITING ASSESSMENT SURVEY, FEBRUARY 2003 by Curtis Gruenler and John Shaughnessy

Hope's graduating seniors write pretty well, but they could write better. This was the conclusion the Assessment Committee came to last year after a three-year attempt to measure the writing abilities of seniors directly. The committee is now looking to try a different approach to how assessment could help the college work toward the "could write better" part of that conclusion. We have begun with a survey this February of how much writing of various kinds students currently do and under what circumstances, and this report of the results is also meant to suggest why we've taken this approach and where we might go from here.

But first, how did the committee come to its preliminary, if not too surprising, conclusion? In 1999, it chose to try direct measurement of two "skills of learning" identified in the new general education program, writing and mathematical thinking. While seniors who respond to the exit survey have consistently marked mathematical thinking as one of their weakest skills, they have rated writing as one of their strongest. Yet the committee chose writing for this kind of assessment because of its importance and also, perhaps, because we were, shall we say, less at a loss how to go about measuring it. For three years, senior seminar instructors were asked to assign a somewhat uniform short paper that was then anonymously rated according to various elements by a team of faculty readers. The results of the first two years of this study were reported in the assessment newsletters for fall 2000 and spring 2001 [available on the assessment web site - www.hope.edu/resources/assessment/]. Other than the conclusion given above, any intriguing correlations in the data between writing ability and anything else about the students were no more than that-intriguing.

Taking some cues from Richard J. Light, leader of the Harvard Assessment Seminars, as discussed in his book Making the Most of College: Students Speak Their Minds, the committee designed the current survey as just the start of a longer process of questioning and conversation that will take shape in response to the results of each stage. It shifts from assessing outcomes to asking some more basic questions about what currently goes on in our students' process of practicing their writing skills. Indeed, you'll see that it asks questions about what students currently do rather than how well they think they're taught. Beyond questions of how much writing they do of various kinds, though, the questions focus on practices that have been found to be especially helpful in the teaching of writing.

The Registrar's Office randomly selected the names of 2,000 students and the Frost Center sent an email to each of these students asking them to participate in the writing survey. Students used a pin number provided in the email to access the web-based questionnaire, and 512 students responded (25.6% response rate). This sample contained a higher percentage of females (72%) than is found in the total student body (61%).

The distribution of graduation classes was: 21% first-year students, 27% sophomores, 28% juniors, and 23% seniors. [The complete questionnaire for the writing survey is available at www.frostcenter.hope.edu/Writing2003.html.] The survey contained 18 questions. The Assessment Committee will report its findings more fully to the Academic Affairs Board, but here we present students' responses to only some of the questions.

Students reported writing approximately 20 papers per semester (Table 1). A large majority of those papers were three or fewer pages (Table 2) and were classified by the students as being personal essay, journal or reflection (Table 3). On each of these measures, there appeared to be little change from first to fourth year of college. The one exception may be the increase in papers per semester that were 10+ pages for first-year (0.4) as compared to fourth-year (1.9) students.

Most of the students report that they are required to submit only one draft of their papers, that they usually do not conference with their professors about them, and that they do not have an opportunity to revise their papers after they have been graded (Tables 4-6). Careful revision is widely acknowledged as important to learning to write, especially in new disciplinary contexts. These survey results imply that faculty could do more to encourage these practices among students, but of course there would be costs. The challenge for us as an institution will be to find a way to enrich the writing experience for students on paper assignments (through multiple drafts, rewriting, or conferencing), without creating a huge burden on both students and faculty. Perhaps creating a peer-review process in our courses is one way to improve writing instruction. Whether we would choose to use peer review more and how we would use it most effectively would depend on many factors specific to particular courses.

The committee's hope is to contribute something helpful to the ongoing conversation among faculty, administrators, and students about how we could do the work we do better. Here are some questions for further dialogue and study:

• How could the college provide more support for the things we already know we could do to improve the teaching of writing, such as multiple drafts of papers? (In other words, how well can we afford to teach?)

• How could the college best apply its limited resources to improve the teaching of writing? What do we need to do in order to figure that out?

• What kind of study of student writing and the teaching of writing will have actual implications for policy?

• How can the college encourage innovation in the teaching of writing and evaluation of those innovations?

• How can the college benefit more widely from the assessment of writing that individual teachers and departments have already integrated into their classes and programs?

• How can integrating assessment more thoroughly and organically into the work that teachers and students do together help enhance that work?

  These are questions for the Assessment Committee itself, first of all, but also of course for the whole college community. The committee will be looking for ways to facilitate conversation, and invites responses.