Evaluation

Using data to make strategic decisions and wise investments can be seen as one of the success factors of effective partnerships (Coffman, 2005). Furthermore, the development of appropriate measures to monitor progress and results and to make judgments on performance is an important function of strategic partnerships. Such monitoring is necessary to the sustainability of partnerships (Coble & Williams, 1998).

    Measurement and evaluation as a structured, systematic effort is an integral part of the Office of the Vice President for Education Partnerships.  A set of core principles that include essential elements for an assessment framework has been developed by the office:

1. The work is guided by solid intellectual conceptualizations and empirical evidence of ongoing successful efforts.
2. All efforts will be accompanied by evaluation.
3. Results will be disseminated.

    Major evaluation efforts are underway, as a comprehensive plan is being developed to evaluate ASU’s partnership initiatives. Guided by a clear vision, mission, goals, and core principles, the ASU Office of the Vice President for Education Partnerships developed a strategic roadmap and a master evaluation plan.  The strategic roadmap allows the use of pathways for visualizing relationships and activities of the partnerships. Using the measurable engagements approach, clearly articulated objectives with measures and benchmarks are assigned to each of the specific goals.  The evaluation measures and benchmarks enable the office to track and report meaningful progress of the ALPHA (and other) districts over time in response to the ambitious goals of the University-School Partnerships.

    Through the generous support of the late Bill Carstens, a private donor who made a $750,000 gift (payable at $150,000 for five years), a comprehensive evaluation plan has been implemented to identify what works in ensuring the overall success of the ALPHA Partnership. Evaluation efforts are guided by a team of evaluation experts. University and school partners work together to develop evaluation objectives.

    Evaluation measures have been carefully designed to address areas such as validity, reliability, direction, sensitivity to differentiation across respondents, direct relatedness to what the program can actually achieve, and unbiased data collection. Evaluation efforts are ongoing.

    A primary focus of the evaluation effort is to examine ASU University-School Partnerships’ engagements and effectiveness on participating students’ academic skills and achievement, attitudes toward learning and school, and behaviors in and out of the classroom.  In addition, the evaluation efforts examine both the use of ASU and community resources by teachers and educational leaders and the degree to which professional development has affected practice.  The evaluation efforts engage parents and family members who participate in the ASU University-School Partnership, to assess their understanding of program impact. For example, programs such as Bullying Prevention, Bridges, Creciendo Unidos, and Arizona Quest for Kids, which are being evaluated in the Murphy School District, engage students as well as their parents and family members. Furthermore, the evaluation efforts focus on identifying program factors, if any, that are related to improved outcomes for students, teachers, leaders, schools, and families.

    It is imperative that partnership efforts maintain built-in and centralized information and data systems to coordinate and facilitate monitoring and evaluation of the university’s Pre-K-12 engagement (events, programs, initiatives, people etc). The office’s evaluation activities include data profiling efforts for focus schools and districts in Alpha partnerships. Data have been collected for the last three years, which will be used as baseline data and allow for the monitoring of educational progress over the years. The following are a few examples of demographic and achievement measures that are being compiled for partner districts and schools:

•      Student and Student Body Descriptors (such as total student population, gender and ethnicity distribution, English language learners, and students on free or reduced lunch)
•      FTE Figures (such as number of FTE teachers)
•      Community Descriptors (such as median household income)
•      School Descriptors (such as Title I school designation)
•      Student Achievement Tests (AIMS, SAT9, AZEC)
•      Student Academic Progress
•      School Profiles (AZLEARNS achievement profiles (excelling, highly performing, performing and underperforming).