Continuous Improvement Process

Program Student Learning Outcomes (SLOs) and Essential Studies (General Education) Essential Learning Outcomes (ELOs) are measured by a variety of designated activities in each program as defined by the curriculum maps. Some courses in the Essential Studies program measure achievement for both LEAP ELOs and program SLOs. BIOL 151 is an example of the type of course that measures both the Major/Minor and the Essential Studies programs.

As instructors analyze their findings from course-level assessment activities, they are asked to reflect on their instructional and assessment processes to discover anything upon which they could improve. This reflection leads to a set of planned actions which are recorded in the Taskstream report and implemented the next time the course is taught. This information is also reported in each course syllabus as required by the MSU Syllabus Template. The Spring 2016 syllabus for BIOL 151 is an example.

The Annual Report

The Annual Report is completed and submitted in Taskstream by Assessment Coordinators and Division Chairs. It is reviewed in-depth by a set of Coordinators not associated with the area being evaluated. Coordinators from the “home” division are invited to their division’s review session to answer questions and argue the division perspective as needed.

The 14-15 Annual Report Review Schedule demonstrates the review process:

In the annual reporting process, achievement of program SLOs is measured with their assigned course-level activities. This is accomplished in a four phased annual reporting process as described below.

Each division is asked to collaborate to identify important findings for each SLO in each Major/Minor program and enter the results into a Taskstream form for important findings. Essential Studies faculty meet based on the ELO and/or the special focus being addressed, collaborate to identify important findings, and enter the results into a Taskstream form for Essential Studies courses. ELOs and most SLOs generate two important findings; however, only one finding is needed when an SLO is associated with only one program (i.e. Psychology). Important Findings should be conceptual in nature and reflect a synthesis of course embedded and non-course embedded information. Using a single course finding and developing action plans for that course does not constitute a Finding. Annual Report Reviewers verify that the appropriate number of Important Findings are written for each SLO and ELO as shown in the examples for Sci & Math and ELO2 Critical Thinking.

Findings should be supported by at least two sources; one must be a course embedded activity and one must be non-course embedded. Non-course embedded sources require documentation in other sections of the Taskstream assessment cycle (e.g. "Professional Exams"). Annual Report Reviewers confirm that more than one source is used to support the Finding and validate any non-course embedded sources by locating and verifying the source in Taskstream (see worksheets linked above).

Important Findings generate action plans designed to improve student learning on the SLO, not simply to better performance on the activities themselves. Meaningful action plans tied to the findings should be specific, verifiable, and extend to multiple courses. Annual Report Reviewers verify that action plans exist, are relevant to the findings and extend to multiple courses to impact learning on the SLO or ELO (see worksheets linked above).

The Important Findings section is scored using rubrics for Major/Minor and Essential Studies courses. The rubric which evaluates number of findings, number of sources, and quality of action plans. For the “Number of Sources” criterion, reviewers do not need to validate course embedded resources. Non-course embedded activities are verified via the Professionally Developed Exams, Student/Faculty Surveys, Student/Faculty Focus Groups, Locally Developed Student Surveys, Student Focus Groups, Alumni Surveys, State-mandated Program Reviews, Professional Accreditation Reviews, Professional Advisory Board Recommendations, Environment Scans, Brainstorms or Other. Reviewers are asked to be reasonably strict in the validation process. A quality action plan is specific, verifiable and impact learning on the SLO well beyond simply improving performance on a single assessable activity.

Course-embedded activities

For all Major/Minor and Essential Studies courses, faculty members are asked to report if scheduled activities were completed, if acceptable targets were met or exceeded and whether actions were undertaken or planned for each course assigned to measure each SLO and ELO. Annual Report Reviewers verify that actions are undertaken or planned by reviewing information submitted to Taskstream. Reviewers also validate each action plan in the same manner. In the Major/Minor Course-Embedded worksheets and in the Essential Studies Course-Embedded worksheets, a quality action plan is defined as “specific, verifiable, and targets student learning on the SLO rather than just performance on the assessment activity.” Examples of completed Annual Report Course-Embedded worksheets are Science & Math and ELO2 Critical Thinking.

Course embedded activities are scored using a rubric which evaluates percentage of scheduled courses with completed activities; quantity and quality of action plans for all courses assessed. A separate a criterion evaluates percentage of scheduled courses with completed activities; quantity and quality of action plans for courses delivered at a distance.

Non-course embedded resources

Non-Course Embedded resources include information obtained outside of course-level measures, such as locally developed surveys and focus groups, program reviews, alumni surveys, focus groups, student surveys and several other activities. Divisions are expected to maintain this information in Taskstream throughout the academic year in the Non-course embedded details section (shown below). Instructors utilize this information on an ongoing basis to understand and improve student learning in their programs.

The Non-Course Embedded Activities form of the Annual Report defines which of the NCE activities were used to support the Important Findings identified for each SLO and ELO. Annual Report Reviewers attempt to validate each of these NCE resources by locating them in the NCE Details section and recording them on the major/minor NCE Worksheet (See Sci & Math example) or Essential Studies NCE Worksheet (See ELO2 Critical Thinking example).

The NCE Rubric for Major/Minor and Essential Studies NCE activities contains two criteria. One evaluates the number of validated activities influencing plans to improve student learning. The other criterion evaluates use of “Special Emphasis” NCE activities, which are locally developed alumni surveys, student surveys and focus groups and locally developed alumni surveys.

Prior Year Review

The Prior Year Review (PYR) process is a key part of the annual report in that it allows the institution to “close the loop” by evaluating progress on actions planned one year prior to the year of the annual report. Important Findings and Action Plans written in the Annual Report for the previous year are re-visited by divisions and Essential Studies programs to determine progress toward intended actions. Annual Report Reviewers use the PYR Worksheets for Major/Minor programs and Essential Studies programs to verify that planned actions have occurred, that those action had an impact on learning and that the verified impact on learning extends to more than one course. Reviewers verify by checking the Taskstream Status Report section for the activity which asks instructors to describe the results of the action taken. Syllabi with changes made and non-course embedded components could also serve as verification/evidence that planned actions were implemented. The PYR Major/Minor and Essential Studies rubrics evaluate verification of implemented action with reported results on student learning impact and verified extension of that impact to multiple courses.

Conclusion

The Annual Report is a rigorous process which enables the institution to collect information about student learning outcomes, such as results of each assessment activity, which informs plans for improvement of the instruction or the assessment process. The important findings process requires that instructors collaborate about their instructional approaches, consider aspects of education outside of their classrooms, and share information about how their students are performing. A synthesis of understanding at a high level must occur in order for relevant findings and actions to be developed. Through the prior year review, the impact of implemented plans on student learning is evaluated, requiring instructors to carefully consider the actions they choose and the likelihood of implementation and success. This evidence demonstrates that MSU collects information about student learning outcomes and applies that information to the planning, review and assessment processes.