Initial Publication Date: February 13, 2009

Winona State University Geoscience Program Assessment: Our Evolution

Cathy Summa, Department of Geoscience, Winona State University


The Winona State University
(WSU) Geoscience program has 
been 
wrestling with assessing our curriculum and
 program
 effectiveness for the better part of the past decade. 
We began our 
work in earnest in the spring of 2003,
motivated by the
 arrival
of two new colleagues (representing 50%
of the faculty). As the department has grown and the curriculum 
has 
evolved, our assessment strategies 
have likewise evolved.


WSU
 enjoys
 an institutional culture that embraces assessment.
 The institution hosts an annual "Assessment Day" early 
in
 the spring
 semester (mid-February)
 for which all
 classes except
 those 
with only
 one weekly meeting (including laboratories) are officially canceled. The
 day is
 divided into a 
morning session where students
 are
 encouraged to participate in
 exams
 that 
assess university general-education goals and student
 satisfaction.
 The afternoon session is
 reserved for "departmental activities." Departments
 are encouraged to
 use this time to bring 
students together for
 program
 assessment activities; limited institutional funds 
to support departmental progress are available 
through a series of "challenge grants".

The
 department felt strongly that we had to
 participate in
 some meaningful way so that we sent
 the 
message
 to students that "assessment matters." Our preference
 was to develop an
assessment that
 measured
 our
 students'
 abilities to approach 
and resolve a field‐based
 problem. Unfortunately, February in Minnesota is
 a challenging time 
for fieldwork,
 which
 forced
 us to look to alternative assessments. After much
 debate,
 we settled
 upon developing
 an "assessment
 exam,"
which would 
be
 administered to students each
 year
 on Assessment Day
 and would be
 designed 
to measure
 students' 
progress
 by assessing
 the depth and
 maturity of 
their responses. 
The central idea
 was to
 develop
 questions that students
 who completed
 or were enrolled in 
our introductory 
sequence (physical geology and
 historical geology) could demonstrate
 what
 they learned
 in terms 
of
 simple rock identification or map 
and cross section
 interpretation, while
 we anticipated that upper-level students could 
synthesize 
content 
from
 multiple
 courses to provide 
much
 deeper
 and richer analysis
 in their responses.
 We
 made
 every effort to develop 
an exam 
that
 would challenge, but not
 demoralize, our
students. This,
 in itself, turned 
out 
to be more difficult 
than we originally anticipated,
 given
 the disparity in
 knowledge and
 ability between
 a new
 entering student
 and a student in his/her last semester.

After spending about a year developing
 the exam, we
 administered it
 for
 three
 consecutive years 
on Assessment Day (2005‐2007). We 
made small revisions to the 
exam 
each year
 as we 
saw 
how 
things went;
 revisions were
 generally minor, and
 made for clarification or to reduce 
the length of the exam.
 We coded the
 exams so that graders had
 no knowledge of
 student 
identity.
 Grading
 the exam
 presented an
 entire new
 set
 of challenges, as it was difficult to set
 aside 
a large
 enough block of time following the Assessment 
Day
 to
 work collaboratively 
to ensure internal consistency.
 Somewhat surprisingly, students were anxious 
to
 know 
how
 they 
did.

Our analysis of student performance 
showed that
 our initial hypotheses held true:
students who 
had 
taken more
 geoscience 
courses performed
 better
 on the exam.
 However, we 
found 
that student performance
 at the upper level was not as strong
 as we anticipated it 
would be.
 When
 we interviewed
 students (graduates)
 to
 understand why, we 
learned that senior students
 didn't take 
the
 exam 
seriously. 
They
 completed 
it quickly 
and
 gave perfunctory answers, 
resulting in lower
 scores 
than 
expected. In 
short, our
 data 
analysis revealed that we 
needed
 to find
ways to incorporate our
 assessment strategies
 into the fabric of
 our
 curriculum.


As
 a result of our experiences, 
we have since abandoned our departmental "assessment
 day"
 activities, and 
instead
 use
 that time for
 faculty planning
 toward on‐going assessment.
 We
 have
 opted 
to
 move 
forward with 
two parallel strategies,
 both just
 in their infancy.
 We have agreed
 to create a 
set of exam questions that will be 
incorporated
 into 
the final
 exam of core courses,
 beginning with
 our 
introductory courses. Student responses 
to these questions will 
be
 tracked
 longitudinally to determine if we
 see 
improvements in the 
maturity of 
the response.
 Secondly, we have decided 
to implement 
a student‐learning portfolio into 
our curriculum.
 We are 
piloting
 the
 portfolio 
project in two classes 
this
 spring 
semester
(in 
Earth 
and Life through
 Time, 
the
 second
 course in 
the
 major sequence; and in Sedimentology and
 Stratigraphy, a junior-senior 
level
 writing intensive course).

Truth
 be told, this is a lot of work. 
It's
 difficult to hold
 the
 creative 
tension
 needed 
to
 work 
through 
the development process and not fall back on collecting numbers, whether 
these
 be
 grades
 earned in specific
 courses
 or 
numbers of students
 going 
on 
to graduate 
programs.
 It's
 challenging 
to
 think
 through developing
 exam
 questions 
that will elucidate connections
 through 
the curriculum and
 it's
 equally
 challenging 
to think through how
 to scaffold
 student
 development
 through 
portfolio creation and maintenance. 
Fortunately, our 
faculty are all
 committed 
to assessment as a means of both demonstrating our successes
 and 
improving 
upon
 our weaknesses. 
The process 
is fascinating
 and
 we're all learning 
a good
 deal. 
It's helped
 us,
collectively, really come to true consensus 
on 
what
 we believe 
to be 
the most 
important
 aspects of our
 program 
and 
on our hopes and
 dreams for our 
graduates.
 Perhaps 
most significantly, our assessment process has affirmed 
that we 
agree on ~98% of
 what we
 do,
making 
it
 much easier to resolve 
conflicts when
 and if they arise.


Stay tuned.