by Dr. Donald Cronkite and Dr. Kathy Winnett-Murray

In May of 1995 the biologists held an all-day departmental retreat, following on the heels of an external departmental review and the subsequent exchange of multiple, multi-colored intra-departmental memos on the subject of curriculum reform. Three major initiatives emerged from that retreat:

• Our new biology core curriculum (4 required courses: Biological Unity and Diversity (Bio 150), Cell Biology and Genetics (Bio 240), Organismal Biology (Bio 260), and Ecology and Evolution (Bio 280)
• Our first beginnings of an assessment plan
• A comprehensive plan for how the new curriculum and assessment would be linked (In other words, we made the decision that our departmental goals would drive the curriculum rather than the other way around.)

We devoted a chunk of that retreat time to the generation of two lists: conceptual objectives and skills-based objectives. These were the things that we all agreed that every biology major should know and be able to do whether that individual was a potential forester, physician, toxicologist, or genetic therapist.

As a first step in doing assessment, our focus has been on assessing skills-based objectives, the focus of this article. We also created a new departmental niche, "The Assessment Person" (currently Dr. Virginia McDonough), who is responsible for coordinating assessment in the four introductory courses.

RESPONSIBILITY FOR SKILLS OBJECTIVES

We intend for students to gain frequent practice with all skills as they undertake a biology major, however, we assigned each of the four introductory courses certain objectives that will be specifically included as part of the departmental assessment of that course.

For example, lab skills such as microscopy and pipetting are targeted in Biology 150, spectrometry in Biology 240, dissection & surgery in Biology 260,

field studies and mathematics in Biology 280. Some lab skills such as microscopy are intentionally highlighted in more than one core course.

Some of our other objectives include writing, technological tools, ethics, statistics, experimental design and use of library/literature resources.

The written description for each lab/field exercise contains a lead section explaining objectives for that lab in two categories: concepts and skills. These, in turn, overlap with both the stated departmental objectives and the lab practical check-off categories (see below).

EVALUATION

We evaluate students through several means in the core biology courses, but we realized that laboratory practicals, already being used in the introductory sequence, were one of the best ways we had of gauging a student's ability to perform a particular task, whether it be focusing a microscope properly, identifying an insect species, constructing a graphical representation of data on a computer, or completing a statistical test.

Since many of our skills-based objectives were already being evaluated this way, we added a section to the cover page of each lab practical that lists the departmental-level skills that will be evaluated on that test, and a space to record information about whether or not the student completed that task.

Thus far, we have used a simple all-or-nothing gauge on skills-based performance (a student receives a 1 for successful completion, a 0 for unsuccessful completion), although such "demonstration of proficiency" may be gauged differently for different skills. For example, using a spectrophotometer properly may be worth 5 points on the lab practical and "proficiency" might be achieved with a score of 3 or better. A question evaluating whether or not a student can use a dichotomous key to identify an insect may require full credit to be counted "proficient".

Students receive feedback when their lab practicals are returned because these "check-offs" appear on the cover page with the overall score. This information helps us communicate how we value proficiency in these skills and how we are keeping track of our ability to teach them. Normally, there are multiple lab instructors for each introductory course. At lab prep meetings, the instructors discuss performance on lab objectives and, using information about the proportion of students who did or did not complete a particular objective successfully, certain ones are chosen for re-evaluation on the next lab practical. Thus, students often get "another chance" to demonstrate proficiency at a particular skill in the same class, and faculty communicate about which skills need more attention (e.g. computer graphics in Biology 150, Fall 1997).

SPREADSHEET

As an on-going database, we created a master spreadsheet which organizes every lab skill by every student. The spreadsheet also contains information about advanced placement, first semester in a biology course at Hope, and who the student had for lab instructor in which term. "Proficiency" designations of 1s and 0s are added by the lab instructors and collated by our department secretary, Bev Kindig.

Although unwieldy in some ways, the spreadsheet allows us to do many things:

• Theoretically, we can provide feedback to an individual student who needs to know "how they are doing" with respect to departmental goals for majors. For instance, a student who wanted to go into Physical Therapy, but had not yet demonstrated proficiency in dissection might want to take an elective in Comparative or Human Anatomy. (Because our Skills Spreadsheet is new, we have not yet actively implemented this into our academic advising.)
• We can assess our own performance in meeting stated goals for each introductory course, which we are in the process of doing. We have the ability to identify particular semesters, or even particular instructors for which proficiency scores are low or high on certain skills. For example, if we noted that all of the students who had Dr. Cronkite for lab were proficient in pipetting, and none of those who had Dr. Winnett-Murray were proficient in pipetting, we would probably ask Dr. Cronkite to give Dr. Winnett-Murray a pipet tutorial before she teaches this lab again!
• We have the ability to track student progress in skills that are repeated (with different applications) in more than one course (e.g. electrophoresis in Biology 150 and Biology 280) and to track longer-term changes as the faculty composition varies from year to year. We can compare how objectives were met (or not) for different student populations (men vs. women, students who go on to be biology majors or not, etc.)
• Ultimately, it has given us all a much greater sense of responsibility in achieving particular outcomes (in this case, skills-based), and a better understanding of the problems associated with measuring outcomes in other areas (e.g. content and concepts).

PROBLEMS AND SUCCESSES

We have discovered that maintaining a spreadsheet of this sort for a population of 200+ new students/year, for four introductory courses, taught by 14 different faculty members can be a record-keeping nightmare. Proficiency scores from the check-off section of each lab practical must be recorded by the instructors before exams are returned to students, and then transferred (at a later date) to either an interim spreadsheet or directly to the master, departmental spreadsheet. After a year, we know some things now that we didn't at first:

• record-keeping rules - we need a strict set of rules for entering data so that a) the secretary does not have a problem interpreting data from different people and b) data gets entered in a timely fashion
• we are taking at least one semester later to get "caught up" from the previous semester, so longer-term feedback is a little slow
• we will have to learn new skills ourselves to know how to extract the kinds of information we want for particular comparisons
• what we have on that spreadsheet is an absolute "goldmine" of information, much of which we had not anticipated

WHAT'S NEXT?

This assessment is about pracitical lab skills. We next have to devise a plan for assessing our success at teaching concepts -- a more difficult task. We are contemplating a departmental exam as one possibility, but we look forward to future pages of this newsletter for other ideas.