Collegiate Learning Assessment (CLA) and CLA in the Classroom

Some of our most desired outcomes for student learning, like critical thinking skills, are the hardest to define and measure.  The goal of this article is to introduce you to the measure that is currently being used in the public baccalaureate institutions of West Virginia, and to stimulate some initial discussions on campus about what avenues might useful for us to explore further in the coming months and years.

The Collegiate Learning Assessment (CLA) measures critical thinking, analytic reasoning and written communication using two different tasks -- a performance task and an analytic writing task. A detailed overview of the test, including a few sample questions, is available at http://www.cae.org/content/pdf/CLABrochure2008.pdf.  More information about the exam is available at the website of the Council for Aid to Education (CAE) at http://www.cae.org/content/pro_collegiate.htm.

The CLA currently is required for all public baccalaureate institutions in West Virginia as part of our participation in the Voluntary System of Accountability (VSA), and is supported through the Higher Education Policy Commission.  At FSU, administration of the test is coordinated through the office of Associate Provost Chris Lavorata/  The test is administered to 100 freshmen each fall, mainly in freshman seminar courses (and including some PC&TC students), and then to 100 seniors in the spring.  Student results at a given institution are adjusted for incoming ACT/SAT scores.   This test has been use at Fairmont State since 2006 and basic results are available on the Institutional Research website (http://www.fairmontstate.edu/institutionalresearch), as part of the VSA link (Voluntary System of Accountability template; see p.5 of the following link: http://www.fairmontstate.edu/images/userimages/wfinley/page_6423/fsucollegeportrait.pdf).

"CLA in the Classroom" (abbreviated [CLA]ssroom) is a new initiative from the Council for Aid to Education (CAE), aimed at encouraging the development and use of critical thinking assessments in classroom settings.  In December 2008, Tim Oxley, Director of the MBA program, and Tony Gilberti, Dean of the College of Science and Technology, attended a "CLA in the Classroom Academy" in Charleston, WV, hosted by the West Virginia Higher Education Policy Commission.  Both men expressed a willingness to lead a brown bag session on [CLA]ssroom for campus colleagues, and both also shared their experiences in the interviews that follow.

Tim Oxley, Associate Professor of Business and MBA Program Director, participated in the  2-day "CLA in the Classroom Academy" (abbreviated [CLA]ssroom) last December and generously shared his perspectives in this interview: 

What was the focus of the workshop?

The workshop had two objectives: First, the workshop demonstrated how participants can perform diagnostic scoring using proprietary CLA[ssroom] Performance Tasks. Secondly and perhaps most importantly, was to engage participants in the process of building a performance task using one of the CLA[ssroom] Performance Tasks as a template. I was surprised to discover the workshop was to be more instructionally oriented than informational/promotional.

What key ideas or approaches made the biggest impact on you?

I was most taken with the authentic assessment approach... [Student performance was measured] by placing the students in a case analysis that mimics a real world experience and requires them to use their higher order thinking skills such as critical thinking, analytic reasoning, problem solving, and written communication. This approach is really a case approach that involves the student actively rather than passively like many case studies. Secondly, I was impressed with the diagnostic tool, or “Student Diagnostic Report”, a booklet of rubrics designed to measure how well the student performed in using their higher order skills in the performance task.

How could we use what you learned at the workshop to improve student learning on our campus?

There may be a number of ways to use the performance task approach to improve students’ learning. Individual faculty members, departments, or schools could develop and use performance task assessments to measure students’ progress toward learning higher order thinking skills; administer similar instruments at various intervals and evaluate progress. The assessments could be used as culminating activities in capstone courses. The issue that may be of primary concern is how to provide feedback to students on their performance. The best method for optimal feedback is face-to-face individualized meetings. This could be daunting.

Comment on the value of the conference to you and to the institution. For instance, should we try to send more people in the future, or have the presenters come to campus?

Notwithstanding the expense, I believe there would be sufficient interest in having one-day training on developing performance task assessments. The [CLA]ssroom organization has two one-day formats that can be delivered on campus. One training event is focused on learning how to develop the performance task and one training event is to learn how to administer and score a performance task. The particular proprietary [CLA]ssroom Performance Task used in the workshop was a retired assessment that would be useful in social sciences, particularly sociology and criminal justice. The scenario requires students to answer three questions in essay form using only evidence provided in the assessment. Workshop participants were given three levels of student responses to score. Obviously each of the responses fit into one of the rubric categories: emerging, developing, mastering.

What did you learn by interacting with participants from other state institutions?

Not only were participants present from other West Virginia institutions, but there were participants from out-of-state institutions. I believe participants shared a common desire and need to effectively measure higher order thinking skills in addition to discipline specific content. Informal conversations surrounding learning outcomes led me to believe FSU is not necessarily behind in our efforts to achieve this goal.

Once the time and energy is invested in developing a performance task assessment, I would suggest it be peer reviewed before using. The assessments themselves are not normed, but others should be involved to make sure the performance task and the diagnostic are measuring what is intended to be measured. One could be lost in developing the task and lose sight of the skills being measured.

What was the focus of the workshop?

The focus of the workshop was to address issues of teaching and learning within the scope of improving student learning. The workshop utilized educational research from the past 25 years to address issues of critical thinking, analytic reasoning, and problem solving. A key focus of the workshop addressed the use of authentic assessment, the use of performance tasks (i.e., outcomes), and grading rubrics.

What made you want to attend?

Assessment and student achievement have been research interests and a focus of my publications for a good part of my career. During the past ten years, there has been an increasing amount of emphasis placed on the topic of assessment. Despite this emphasis, there appears to be no consensus on exactly what topics and processes assessment comprises. Should assessment be concerned with the performance of individual students or groups of students, the quality of instructional practices, or the performance of programs, departments, or the institution itself? My goal for attending this workshop was to continue my education and bring back resources that could be shared with my colleagues.

What one or two (or several) key ideas or approaches made the biggest impact on you?

The workshop helped to reinforce much of what is known about how students learn best. The use of authentic assessment and creating learning materials which are meaningful to students were most valuable. Meaningfulness consists of the relationships between facts, generalizations, rules, and concepts which students view as important and useful in a contemporary society. Making learning meaningful is a matter of selecting the appropriate content and helping students to see its applicability in the situations for which they are concerned. Since American culture has been characterized as a technologically based society, the function of our educational system should provide citizens and students with the insight and understanding to solve the problems resulting from the social, technical, and environmental interface.

How could we use what you learned at the workshop to improve student learning on our campus?

There are a number of items that could and should be shared with our campus community. These could consist of how to incorporate authentic assessment techniques, and methods of identifying job-related and civic-related roles and activities into the subject matter being taught. The CLA has developed an inventory of copyrighted material that illustrates a method for fostering higher order thinking and reasoning skills among students. I found this material to be well-developed, and it can be used to create learning materials that are relevant and meaningful to our students.

Comment on the value of the conference to you and to the institution. Should we try to send more people in the future, or have the presenters come to campus?

This type of workshop would be difficult to duplicate with a large number of participants, as it was activity based and required a significant amount of interaction from the participants. We would be better served by sending people in teams (3-4 people) from the same department to the conference. In this manner they would be able to collaborate both before and after the conference to redesign and existing course or developed material for a new course. It would also be valuable to hold a brown-bag lunch or full day workshop utilizing past conference participants as presenters for faculty on our campus.

What did you learn by interacting with participants from other state institutions?

Each of the other state institutions all have the same interests and problems associated with developing assessment plans and institutionalizing assessment. Additionally, all of the participants were looking for methods to improve the student learning process. In many respects, Fairmont State University was well ahead of many of the institutions at the conference. This is due in no small part to our Assessment Committee and the faculty who are engaged in this process.

What else would you like to share about the workshop or CLA?

While assessment may be a response to external demands or audiences, its focus must remain with the educators and institution. Assessments may further serve multiple purposes, but the primary purpose should be for the improvement of student learning. The CLA illustrated that good assessments will require an enormous amount of data collection and assurances that the information gathered is valid, reliable, and credible. Lastly, the costs associated with an assessment should be weighed against its potential benefits and side effects.