The unit must implement a formal process for using the assessment system and stakeholder feedback to inform unit and program improvement.

A strong assessment system is not only for data collection, review and accountability purposes. The ultimate purpose of UED’s assessment system is to promote continuous improvement of our unit and its programs, including the improvement of candidate performance and meeting our mission to diversify the teacher workforce and improve the achievement of urban learners Birth-Grade 12. The formal process we use to create data-informed plans and decisions to improve our programs and the unit as a whole is described in the narrative below and the overall system design is depicted in Figure 5.1.

Figure 5.1: Overall Assessment System Design

The UED assessment system implements a formal process designed to offer meaningful data to those who work in the UED and our stakeholders which leads to their important feedback that informs improvement efforts in three areas:

• Candidate Performance – Urban learners deserve excellent educators who know how to form positive relationships and nurture their students’ excellence. Our primary focus is to enhance teacher candidate performance with urban learners to improve the achievement of urban learners. The UED is also focused on the performance of our candidates on standardized assessments (i.e. MTLE and edTPA) that are high stakes for teacher candidates and the UED in terms of licensure and unit accountability. The UED Assessment System process provides useful aggregated data analyses of candidate performance on key candidate assessments at required checkpoints (admission; during courses, at degree candidacy and/or entrance to practicum or student teaching; graduation or program completion; and after graduation from the program) to the UED faculty, staff and dean to support a continuous, data-driven, improvement.

• Program Quality – The performance of teacher candidates is determined by  several factors that relate to the quality of program they enroll in and complete ..  Our programs and the courses within programs evolve and improve with data generated from our assessment system and stakeholder feedback in order to ensure candidate performance improves.

• Unit Quality – Strong and effective programs depend on a strong, stable and effective unit in order to be sustained and grow in their capacity to prepare excellent urban teachers. By aggregating program-level data across the unit, we generate a composite view of how our University prepares teacher candidates for the opportunities and challenges they will face in working with urban learners, families, communities and colleagues.  With this view, our process for working for continuous improvement based on data involves re-examining unit operations (e.g., quality of instruction and advising, governance, policies, resources, personnel, program delivery, and other candidate support).

The UED assessment system is based on a cycle of continuous data collection, collective analysis by multiple stakeholders, and efforts at improvement guided by our understanding of research, our own practice, reflection, collaboration and leadership focused on education equity. The assessment cycle (see Figure 5.2) begins with collecting data about candidate performance, our mission or unit/program operations. These data are then analyzed by unit faculty and staff as well as stakeholders which leads to recommendations for program improvements designed to enhance candidate and PreK-12 student learning, achieving our mission, and strengthen unit operations. The cycle begins again with new assessments and data about candidate performance, our mission and unit/program operations.

Figure 5.2 Assessment System Process for Continuous Improvement

The assessment system is designed to collect data that provide multiple measures (see Exhibit 5.1) of important candidate, program and unit quality, including:

1. Candidate knowledge, skills, dispositions and performance (i.e. signature assessment evaluations in courses throughout their programs, MTLE content and pedagogy exams, edTPA Task and rubric scores, advanced practicum and student teaching overall evaluations, and surveys of candidates, alumni and their supervisors);
2. Metrics related to our mission (i.e. demographics of candidates who are enrolled in and complete our program, licensure and job placement rates, feedback from school partners); and
3. Unit and program operations, including quality of faculty instruction, governance, resources, faculty characteristics, program delivery and candidate support services.

These data inform stakeholders’ feedback to the unit which in turn informs unit decisions and actions for continuous improvement in the cyclical process as shown in Figure 5.2. The program assessment data are shared with program coordinators, then the whole faculty and staff in the UED, and then with stakeholders for their analysis and feedback. Data-driven program and unit-level improvements are then proposed, decided upon and implemented consistent with our mission, best practice, professional standards, and the UED Conceptual Framework.

Over the past decade, the UED has implemented and refined our formal process for using our assessment system and stakeholder feedback to inform improvements to programs and the unit. Since our last PELSB site visit in 2018 for continuing unit approval, three examples of how we have implemented our process for continuous improvement stand out:

1. Efforts to close achievement gaps in who completes student teaching by providing financial support through grants
2. Efforts related to improving candidate MTLE and edTPA scores
3. Development of UED Dispositions for Effective Urban Teacher Candidates

Financial support to Complete Student Teaching

By focusing on our mission to increase teachers of color and American Indian teachers (TOCAIT) in urban schools, UED staff, faculty and stakeholders examined data that showed a gap between the percentage of TOCAIT candidates enrolled in our various programs and the percentage who enrolled in student teaching to complete their programs. By talking with teacher candidates, they revealed the most significant barrier to them student teaching was financial. As post-traditional college students, many whose incomes are needed for their families, they were able to juggle work and life responsibilities while taking courses in evenings and weekends as they were earning income. However, doing the full-time work of student teaching for 3 months without pay was proving to be untenable for disproportionate percentages of TOCAIT candidates compared to their white peers.

Stakeholder feedback and recommendation to the UED was to more actively seek and prioritize grant funding to support TOCAIT candidates in completing student teaching. Over the past several years such an intentional focus of our work has resulted in the UED receiving several grants and donations that prioritize support for TOCAIT candidates to complete student teaching while also providing scholarships to all TOCAIT candidates enrolled in coursework as a retention strategy. Most notably, since the Collaborative Urban and Greater Minnesota Educators of Color (CUGMEC) grant became competitively awarded in 2017, the UED has been awarded approximately one-third of legislatively appropriated funds. We have used 95% of awarded grant money for direct support to hundreds of BIPOC candidates, especially student teachers. In addition, we have promoted other grant opportunities and dozens of TOCAIT candidates have received student teaching grants directly from the Office of Higher Education. Several candidates who graduated without student teaching have returned to student teach with this financial support. As a result, the gap has closed between the percentage of TOCAIT candidates who complete licensure programs and the percentage of white peers who complete programs (Figure 5.3). Of the 53 teacher candidates who successfully completed student teaching requirement for licensure during the academic year 2020-2021, 29 (54%) were students TOCAIT. Most remarkably, of the 29 TOCAIT candidates who successfully completed their program in 2020-2021, a total of 18 who did not receive the OHE student teaching grant were awarded tuition scholarships through the CUGMEC grant for all their student teaching credits.

Figure 5.3: UED Program Completers by Race as a Percent (%) (FY14–21)

Efforts related to improving candidate MTLE and edTPA scores

MTLE content and edTPA data reveal a need for more specific interventions to support candidates in improving their outcomes. Data from specific components of MTLE and edTPA reveal areas of mastery for some candidates outlined in the standardized assessments. The data also indicates specifically areas needing additional work in order to improve their outcomes. A base for targeting interventions can be found in the proficient performance of our teacher candidates in the field with urban learners as assessed by cooperating teachers, school administrators and university supervisors using instruments based on the Standards of Effective Practice including Standard 1: Subject Matter. 

In recent years, analyzing the UED assessment system data about MTLE and edTPA scores and recommendations from stakeholders on our Advisory Groups have resulted in changes to the curriculum, changes to policy, and the allocation of intensive faculty and staff resources to support candidates' learning strategies to improve their outcome of these assessments. One example of a primary intervention is tied to stakeholders’ recommendations that candidates have early and ongoing exposure to and practice with the edTPA and MTLE, prior  to , as well as, during  student teaching. 

Prompted by assessment system data and stakeholder suggestions, faculty are now working to scaffold practice with all edTPA prompts and meeting rubric expectations in courses throughout candidates’ programs. We also realized we needed to provide more explicit instruction during student teaching seminars and provide edTPA writing days following the examples of other institutions with more traditional student populations because a major barrier for our candidates has been lack of time to focus on the edTPA while trying to meet the needs of the urban learners they are teaching.

As for the MTLE content exams, much of the content knowledge tested on the exams is gained or covered by candidates several years before they actually take the exams.  Feedback from stakeholders gathered at Advisory Group meetings was to provide more intensive test prep support and to change unit policy and require candidates take diagnostic practice tests early in their programs while being required to take the actual exams before student teaching. In the spring of 2022 the UED made these policy changes.  In the summer of 2022, several faculty reached out to make personal contacts with candidates and learn what support they needed and wanted ranging from test vouchers to online study guides/interactive tools, and personalized tutoring. That summer, resources were also allocated to purchase 200 subscriptions for Study.com as a robust, interactive online test prep tool along with another comprehensive online test prep tool Mometrix.  In fall of 2022, MTLE and edTPA Support Committees of faculty and staff were formed to develop and implement intervention plans for raising student scores on both standardized assessments.

Due to disruptions with the implementation of MTLE and edTPA standing policies by PELSB during COVID, it is too early to determine whether and how much these changes will impact candidates’ scores, but the data is being collected and analyzed as part of the cycle of our assessment system for continuous improvement. 

Development of UED Dispositions for Effective Urban Teacher Candidates

Prior to the new unit rule being implemented, the UED had unit-defined dispositions expected of urban teacher candidates expressed in our UTP Standards of Professional Conduct. However, we did not assess these dispositions at various stages of a candidate’s matriculation through their program. During the 2021-22 school year, the UED faculty, staff and dean started drafting ideas for an assessment instrument based on dispositions stated in the UED Standards of Professional Conduct. Using the formal process of gathering recommendations from stakeholders at our spring 2022 Advisory Group meeting about what dispositions are most important for urban teachers and how to define them, the UED has incorporated and implemented stakeholder feedback in our Dispositions Assessment instrument that we are starting to implement in the fall of 2022 during our two practicums (EDU 311 and EDU 450) and student teaching. Preliminary data from its use will be analyzed in our assessment system and shared with stakeholders in spring and/or fall 2023.