Restructuring school incentives pilot project summative evaluation report 1992-1993

              Restructuring Schools Incentives Pilot Project
Summative Evaluation Repof
Submitted to the Arizona Department of Education
and
The Joint Legislative Committee
Dr. Susie Cook and Dr. Thomas Haladyna
ASU West
July, 1 993
Table of Contents
List of Figures
List of Tables
Executive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i
Evaluation Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Evaluation Strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
Toward a Definition of Restructuring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4
Evaluation Question One: Findings From Project Schools . . . . . . . . . . . . . . . . 7
Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Goal 1 : Increased Parental Involvement . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 0
Goal 2: Increased Student Achievement . . . . . . . . . . . . . . . . . . . . . . . . . . .1 1
Goal 3: Improved Professionalism and Cooperation . . . . . . . . . . . . . . . . . . . .1 3
School Profiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Goal 4: Improved Efficiency and Effectiveness . . . . . . . . . . . . . . . . . . . . . . - 23
Goal 5: Improved School ClimateILearning Environment . . . . . . . . . . . . . . . .2 4
Goal 6: Meeting Unique Needs of Students . . . . . . . . . . . . . . . . . . . . . . . . . 2 5
Summary of Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2 6
Evaluation Question Two: Improving Restructuring . . . . . . . . . . . . . . . . . . . .2 8
Evaluation Question Three: Arizona's Efforts . . . . . . . . . . . . . . . . . . . . . . . .3 3
Survey of Project Teachers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3 3
Conclusions From Survey . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4 4
Evaluation Question Four: Improving Arizona's Restructuring . . . . . . . . . . . . .4 6
Mitigating Circumstances . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 6
Observations by External Evaluators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4 7
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . -48
References
Figure 1:
Figure 2:
Figure 3:
Figure 4:
Figure 5:
Figure 6:
Figure 7:
Figure 8:
Figure 9:
Figure 10:
Figure 11 :
Figure 12:
Figure 13:
Figure 14:
Figure 15:
Figure 16:
Figure 17:
Figure 18:
Figure 19:
Figure 20:
Figure 21 :
Figure 22:
Figure 23:
Figure 24:
List of Figures
Structural Model for Restructuring . . . . . . . . . . . . . . . . . . 5
School Profile: C . J . Jorgenson . . . . . . . . . . . . . . . . . . . . 15
School Profile: Carminati . . . . . . . . . . . . . . . . . . . . . . . . 16
School Profile: Del Rio . . . . . . . . . . . . . . . . . . . . . . . . . . 16
School Profile: Gilbert . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
School Profile: Glendale . . . . . . . . . . . . . . . ..- . . . . . . . .1 7
School Profile: Kyrene . . . . . . . . . . . . . . . . . . . . . . . . . . 18
School Profile: Laguna . . . . . . . . . . . . . . . . . . . . . . . . . . 18
School Profile: Litchfield . . . . . . . . . . . . . . . . . . . . . . . . 19
School Profile: Mountain View . . . . . . . . . . . . . . . . . . . . 1 9
School Profile: Palmcroft . . . . . . . . . . . . . . . . . . . . . . . . 20
School Profile: Papago . . . . . . . . . . . . . . . . . . . . . . . . . . 20
School Profile: Rancho Viejo . . . . . . . . . . . . . . . . . . . . . . 21
School Profile: Safford . . . . . . . . . . . . . . . . . . . . . . . . . . 22
School Profile: University . . . . . . . . . . . . . . . . . . . . . . . . 22
School Profile: Westwood . . . . . . . . . . . . . . . . . . . . . . . 22
Stages of Overall Restructuring . . . . . . . . . . . . . . . . . . . .3 2
Satisfaction With Classroom Issues . . . . . . . . . . . . . . . . . 34
Satisfaction With Schoolwide Issues . . . . . . . . . . . . . . . .3 7
Satisfaction With Influence
Over Schoolwide Issues . . . . . . . . . . . . . . . . . . . . . . . . . 38
Decision Making Authority
Concerning Classroom Issues . . . . . . . . . . . . . . . . . . . . . 39
Decision Making Authority
Concerning Schoolwide Issues . . . . . . . . . . . . . . . . . . . . . 40
lmportance of Decision Making Authority
Concerning Classroom lssues . . . . . . . . . . . . . . . . . . . . . 41
lmportance of Decision Making Authority
Concerning Schoolwide Issues . . . . . . . . . . . . . . . . . . . . . 42
List of Tables
Table 1 : Stages of Restructuring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Table 2: Levels of Parental Participation . . . . . . . . . . . . . . . . . . . . . . . . . 10
Table 3: Stages of Activity Concerning Student Achievement . . .-. . . . . . . 11
-
Table 4: Teacher Professionalism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Table 5: Profile of Restructuring Activity
by Level of Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23
Table 6: School and Learning Climate . . . . . . . . . . . . . . . . . . . . . . . . . .24
Table 7: Meeting Unique Needs of Students . . . . . . . . . . . . . . . . . . . . . .25
Table 8: Strengths of Project Schools . . . . . . . . . . . . . . . . . . . . . . . . . .27
Table 9: Formative Evaluation Summary . . . . . . . . . . . . . . . . . . . . . . . . .29
Table 10: Summary of Recommendations
Offered to Project Schools . . . . . . . . . . . . . . . . . . . . . . . . . . . .30
Table 11: Teacher Influence in the Classroom . . . . . . . . . . . . . . . . . . . . . .35
Table 12: Textbook Use . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .36
Table 13: Students' Interests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .43
Table 14: BarrierdMitigating Circumstances . . . . . . . . . . . . . . . . . . . . . . - 4 6
Table 15: Teachers' Perceptions of Restructuring . . . . . . . . . . . . . . . . . . . iii
Executive Summary
Summative Evaluation Report
1992-93
This Summative Evaluation Report (1 992-93) includes the following sections:
I. Introduction, Evaluation Plan, Evaluation Strategies;
II. Review of the literature: Toward A Definition of
Restructuring;
Ill. Findings from the project schools, organized as responses
to four evaluation questions:
A. How are the 15 individual schools progressing with
respect to their stated project outcomes?
B. How can each of the 15 school sites improve their
restructuring efforts?
C. How effective are Arizona's efforts to offer incentives to
schools to restructure?
D. How can Arizona improve its statewide effort to
restructure?
IV. Conclusion.
Section I.
The introductory section summarizes Senate Bill 1552 (1990), outlines the
evaluation plan in terms of the four guiding evaluation questions, and explains the
evaluation strategies used to collect data from the 15 project schools, including the
following data sources:
1. Self-evaluation reports submitted by each school;
2. On-site visits by external evaluators;
3. Rubric evaluations of six project goals;
4. 45 focus group interviews conducted with parents,
teachers, and students;
5. Survey data from 390 responding teachers in the 15
schools.
Section II.
This section presents an executive summary of the review of literature
conducted by the external evaluators. The purpose of including an abridged review
in the summative report was to provide for readers the context in which the 15
project schools have directed their restructuring efforts. The most significant finding
from the review of literature was the notion that genuine restructuring centrally
focuses on the improvement of student achievement. The three m_dst significant
contributors to increasing student achievement are: 1. School-based decision making;
2. An aligned model of curriculum/instruction/assessment; and, 3. Professionalism in
the workplace. From the review of literature and through a series of interviews
conducted by the external evaluators during 1991 -92, a structural model (Figure 1,
p. 5) for restructuring was constructed, showing the relationships among the discrete
goal areas outlined in S.B. 1552 (1 990). The development of the model should assist
policy makers with framing future legislation aimed at offering financial incentives to
schools to conduct business differently.
Section I l l .
Organized as responses to the four evaluation questions, this section presents
the findings from the 15 project schools respective of the six goal areas defined in
S.B. 1552 (1990). The rubrics constructed by the external evaluators, based on
Hall's Concerns-Based Adoption Model, are displayed in Tables 1-4
(pp. 9-1 2; p. 14). The findings from the application of the rubrics are displayed in
individual school profiles (Figures 2-1 6, pp. 15-22). The profiles demonstrate the
developmental nature of restructuring as well as the evolutionary stages through
which the project schools have progressed over two years. The data from the rubric
evaluations should assist policy makers with understanding that the success of any
one school is dependent on the character of the culture and values developed prior to
the initiation of the project funding. Each school began its restructuring efforts at
unique points and its progress along the continuum of school improvement to
restructuring is evidence that careful selection of project schools will determine the
success of a statewide effort to restructure schooling.
Tables 5-7 (pp. 23-25) profile the range of restructuring activities aimed at
improving efficiency and effectiveness, improving the learning environment, and
meeting the unique needs of students with respect to four stages of innovation:
Planning; implementation; improvement; and, refinement. These data should assist
policy makers with understanding the wide scope of activities that project schools
have undertaken. Each school demonstrated strengths and these data are summarized
in Table 8 (p. 27).
A critical aspect of the external evaluators' role was to provide formative
information to the 15 project schools that will assist participants with pursuing
genuine restructuring. Both process and product recommendations were offered to
each school. Table 9 (p. 29) summarizes the levels of sophistication at which the
schools conducted their self-evaluation studies (process). Table 10 (p. 28)
summarizes the single recommendation offered to each project school that has the
most potential to impact its restructuring efforts (product).
A survey of teachers (N =390) assigned to the 15 project schools was
conducted, parallel to a survey disseminated by the Arizona Department of Education
to randomly selected schools throughout Arizona. The purpose of the survey was to
determine the teachers' perceptions about decentralization issues. The findings are
displayed in Figures 18-24 and in Tables 11 -1 3 (pp. 33-43). The conclusions, pp. 44-
45, indicated that, even among teachers assigned to schools with funded
restructuring projects, there are perceptions of control exerted by central offices
regarding schoolwide issues and building principals regarding classroom issues. The
most significant finding from the survey data was the extent to which the responding
project teachers felt that their schools had enacted a plan for restructuring. These
data are summarized in Table 15 and reveal a discouraging trend that teachers
perceive efforts in their schools to restructure differently than do their administrators.
Table 15
Percent of Responding Teachers Indicating Schools are Restructuring
iii
Choices
None
Discussed possibilities
Began planning
Enacted plan
Per cent
1
10 -
2 6
62
All 15 project schools perceived that they endured mitigating circumstances or
barriers to restructuring. These data are summarized in Table 14 (p. 46). In addition,
the external evaluators offered seven observations (pp. 47-48) that could impact
future efforts in Arizona to offer incentives to schools to restructure. These
observations collapse into two categories:
1. The appropriateness of S.B. 1552 (1 990) that encouraged restructuring
rather than mandating significant changes; and, -
2. The selection criteria for project participation.
To ensure the success of a statewide effort, the project schools must be
carefully selected based on their respective political climates, their demonstrated
progress in planning for systemic change, their track records with implementing
research-based programs and practices such as multi-age grouping, inclusion models
of regular and special education, year-round education, etc., their institutionalized
abilities to collect data and critically analyze the outcomes, the extent to which their
facilities are conducive to restructured programs and practices, and their financial
commitment and capacity to sustain their efforts both during and after the funding
cycle.
Section IV.
A summary of the Surnrnative Evaluation Report was offered in the context that
Arizona policy makers ought to be proud of their efforts to encourage schools to
restructure. The 15 project schools were commended for asking tough questions
about educating students and responding with heartfelt activities designed to improve
education in public schools in Arizona.
Acknowledgements
The external evaluators, Susie Cook and Tom Haladyna, would like to thank the
following people/organizations for their contributions to the evaluation effort. Any
effort to evaluate people's work is received with mixed emotion. Without the full
cooperation of administrators, teachers, parents, and students in the 15 project
schools, this effort could not have been accomplished. In addition, we would like to
thank: -
The Morrison Institute for Public Policy
The Arizona Department of Education
Dr. Paul Koehler
Ed Sloat
Dr. Barby Carlile
Melanie Chandler
Jill Forney
Lori Niebur
Surnmative Evaluation Report
I. Introduction
Senate Bill 1552 (1 990) offered monetary incentives to schools to restructure.
Fifteen pilot schools were selected to participate in a three-year funding cycle and
were encouraged to examine, and alter if appropriate, their practices, focusing on six
goals:
1 . lncreasing Parental Involvement;
2. lncreasing Student Achievement;
3. lmproving Professionalism and Cooperation of Teachers;
4. lmproving Efficiency and Effectiveness;
5. lmproving the Learning Environment; and
6. Meeting the Unique Needs of Students.
This Summative Evaluation Report (1 992-93) will outline the evaluation plan
and strategies used to collect data from the 15 project schools, offer a summary of
the national review of literature conducted by the external evaluators, and present the
findings from the project schools, organized as responses to four guiding evaluation
questions.
Evaluation Plan
A three-year evaluation plan was designed to measure the success of the 15
pilot school sites that were offered incentives by the Joint Legislative Committee (S.B.
1552, 1990) to restructure. Four evaluation questions guided the three-year plan:
1. How are the 15 individual schools progressing with respect
to their stated project outcomes?
2. How can each of the 15 school sites improve their
restructuring efforts?
3. How effective are Arizona's efforts to offer incentives to
schools to restructure?
4. How can Arizona improve its statewide effort to restructure
schools?
Evaluation Strateaies
To answer the four evaluation questions, the external evaluators collected data
from two fundamentally different sources: 1. Data generated by project personnel in
each of the 15 sites through the process of self-evaluation; and 2. Data generated by
the external evaluators through planned data collection.
Self-Evaluation: The commitment to self-evaluation was ' made after
considerable thought and deliberation and was based on a national review of the
literature (Akpe, 1991 ; Koehler, M., 1990; Pazruski, 1990; Uphoff, 1991 ). Self-evaluation
is a natural aspect of school restructuring because restructuring implies that
decision making is shifted to a broader base of stakeholders (teachers, parents, and
students). These educational stakeholders, then, ought to be responsible for
identifying both the outcomes to be evaluated and the criteria to determine whether
these outcomes have been achieved. With increased responsibility for self-evaluation
comes increased accountability for defining significant project outcomes that are
characteristic of genuine restructuring and for ensuring that sound data collection and
analysis procedures are applied to the project outcomes.
Self-evaluation is a developmental process. Over a two-year period, the
external evaluators have observed that the personnel in the 15 project schools have
defined more appropriate outcomes, developed more relevant means to collect
significant data from parents, teachers, and students, increased data collection and
improved analysis procedures, and focused their projects on improving student
achievement, which is the central focus of genuine restructuring. Perhaps of equal
importance to policy makers and educators, self-evaluation offers schools an
institutionalized plan for evaluating both restructuring efforts beyond the funding cycle
of S.B. 1552 (1990) and other programs and practices that traditionally either are
continuedldiscontinued without benefit of evaluation or require external consultation
at the expense of individual schools.
External Evaluation: While self-evaluation provided one set of data regarding
the attainment of project outcomes, the external evaluation plan had three additional
major purposes:
1. To provide technical assistance to project personnel to help them
complete their self-evaluations;
2. To digest and summarize the findings of the self-studies to compile a
holistic, formative view of restructuring across project schools; and
3. To collect specific data from the project schools that addressed the
major questions of the evaluation plan.
The first two purposes were accomplished throughout the 1992-1 993 school
year, initiated by a technical assistance workshop hosted by the Morrison Institute,
continued through site visits by the external evaluation team, and concluded by a
formative evaluation report prepared by the external evaluators for each project
school. Each school site received a formative report in July 1993 for review, that
included a summary of project outcomes and analyses of the extent to which they
were accomplished. The 15 site reports were compiled into one report. The third
purpose of the evaluation plan was accomplished by the external evaluators through
three major data collection and analysis efforts:
1. A series of 45 focus group interviews conducted with
parents, teachers, and students in each of the 15 project
schools to determine how schools are defining and
measuring student achievement (the central focus of
genuine restructuring);
2. A set of rubrics designed to characterize the developmental
stages of genuine restructuring and to evaluate the extent
to which the project schools accomplished outcomes
organized around the six goals set forth in S.B. 1552
(1 990);
3. An anonymous survey of teachers in the 15 project schools
designed to determine their attitudes about decentralization.
(This survey was designed in parallel form to the survey
disseminated to randomly selected schools throughout
Arizona by the Arizona Department of Education so that
comparisons between the 15 project schools and the
statewide sample could be analyzed for the third-year
summative report.)
These data, combined with the formative evaluation data, are summarized in
subsequent sections of this summative report. Implicit in the presentation of findings
from the 15 project schools is a shared understanding of the nature of genuine
restructuring and its relationship to school reform or school improvement that has
characterized schooling in the United States since the 1970s. The following executive
summary of the review of literature has been included to illustrate the framework in
which the 15 schools have directed their restructuring activities in 1992-93 as well
as to summarize key findings from the evaluation effort conducted during the 1991 -92
school year.
11. Toward a Definition of Restructuring
Mounting public and political pressure to change the way schools operate and
educate students has created formidable challenges for policy makers, researchers,
and educators. Questions about process, definitions, and the implicit relationships
among reform, restructuring, and reconstruction have been largely answered by
prescriptions, how-to manuals, legislative mandates, and trial-and-error efforts in
schools. -
This executive summary of the review of literature will argue that, in order to
effect the wide-sweeping educational change that constitutes restructuring, the
stakeholders in public education must:
1. Create clear language about successful educational practices;
2. Construct a vision of a restructured school;
3. Plan for systemic change; and
4. Refocus the mission of schools on the educative functions.
The term, restructuring, as it applies to schools, has frequently become
synonymous with school choice, teacher empowerment, school-based decision
making, parent involvement, national standards for curriculum and assessment,
accountability, decentralization, or any combination of these practices. Seemingly,
the concept of restructuring or substantively changing the ways schools educate and
operate appears nested in practice. The challenge for schools, then, is to lift the
conceptual framework from practice so that the vision of a restructured school can
drive meaningful changes in organizational structure and classroom activities.
Procedurally, Arizona's Senate Bill 1552 (1 990) offered financial incentives to
15 schools to restructure. Without wholesale mandate, schools were appropriately
encouraged to examine governance, curriculum and instruction, parental involvement,
meeting the unique needs of special student populations, effectiveness and efficiency,
student achievement, the roles of teachers, and the learning environment. An
evaluation of the schools' efforts in 1991-92 revealed that concrete images of
restructured educational practice seemed to focus on three areas:
1. Decision making: Teachers, parents, and students make
substantive decisions about the problems and issues that
confront their local schools;
2. Curriculum, instruction, and assessment: Schools
recognize the inextricable relationship among these three
entities and seek alternative, research-based delivery
strategies; and
3. Professionalism in the work place: Redefining governance
and changes in curriculum and pedagogy generate new
roles and relationships among teachers, parents, students,
and school leaders.
In over 90 interviews conducted with parents, teachers, and students in 1991 -
92 in the 15 pilot schools, comments from these stakeholders did not cluster evenly
or naturally around the six goals set forth in S.B. 1552 (1 990). Rather, the interview
data showed that parents, teachers, and students felt that increasing student
achievement is the central focus of genuine restructuring. The findings offered further
evidence that the three most significant contributors to increasing student
achievement were school-based decision making, an aligned model of curriculum/
instruction/assessment, and improved professionalism among teachers. With these
variables in place, parental involvement increased, the unique needs of students were
more easily met, the climate or learning environment improved, and efficiency and
effectiveness of school operations were improved. These findings offered support for
a structural model (Figure 1) of restructuring that can be fitted to any set of political,
social, and cultural values defined by local stakeholders.
Figure 1
CULTURE
instruction
/ \
School- 7
based I -1 Achievement rofessionalism
Decision / I n the Workplace
making
I
Meeting the
Participation, Unique Need:
I
VALUES
lm~licationso f the Model
To apply the model to schools, student achievement must be appropriately
defined and measured. Schools are currently left with a puzzling dilemma when they
try to demonstrate that change, reform, or restructuring has occurred or has
succeeded in the wake of the national crisis about defining and measuring student
achievement in the context of complex learner outcomes. Further implications of the
model include using these concrete images and appropriate language to effect policy
development at local, state, and national levels. Consistent with the literature, top-down
mandates are the antithesis of genuine restructuring. However, policy makers
ought to establish procedures for encouraging parents, teachers, and students to
conduct business differently. Finally, educators must develop rich descriptions of the
settings in which local schools are restructuring. Without defining and describing the
values to which the stakeholders subscribe, schools will proceed once again down the
long road of change for merely the sake of change and deal with the tragedy of
wasted time and wasted money.
Summary
In reviewing the educational reform literature, it is clear that incremental reform
has obscured the vision for substantially changing the way schools educate students.
Genuine restructuring requires a vision of what a restructured school looks like replete
with clear language about successful educational practices in the classroom.
Stakeholders must define their values within the context of their local cultures, and
policy makers can assist by appropriately encouraging schools with incentives to
engage in meaningful change.
Ill. Findings from Project Schools (1 992-93)
Organized by Four Evaluation Questions
A. Question One: How are the 15 individual schools ~ro~ressiwnaith resDect to their
ect outcomes?
Each of the 15 project schools summarized their settings, defined project
outcomes based on the six goal areas set forth in the legislation, measured their
outcomes, analyzed their findings, and discussed future plans for restructuring in their
self-evaluation studies. In addition, rubrics were designed by the external evaluators,
based on Hall's Concerns-Based Adoption Model, that characterize the evolutionary
stages of restructuring respective of the six goals. Project personnel evaluated
themselves on each of these rubrics on a scale ranging from orientation to renewal.
These self-ratings were validated by the external evaluators based on independent
observations by six evaluators over two years and data gathered during site visits.
It must be noted that for nearly all of the project schools, the ratings on the rubrics
were adjusted by the external evaluators. Either the schools underrated or overrated
their progress. In cases of overrating, it was assumed by the evaluators that
educators are accustomed to marketing their projects to specific audiences and,
therefore, the project personnel perceived that only the highest levels of attainment
on the rating scales would cast them in favor. Actually, few schools participating in
the restructuring project, or in any restructuring effort throughout the nation, would
merit rankings at the highest, renewal, stage. Moreover, rankings above the
mechanical stage are considered significant of genuine restructuring efforts. Finally,
focus group interviews were conducted with parents, teachers, and students in each
of the 15 project schools. The following figures, tables, and discussion summarize
these findings by setting and goal area (ref. S.B. 1552, 1990).
School Profiles
Figures 2-16 illustrate individual school profiles that include three of the goal
areas: Student achievement; professionalism; and, parental involvement; as well as
a measure of overall restructuring. The data are summaries from the rubric
evaluations and presented as individual school profiles to avoid inappropriate
comparisons among schools. Clearly, each school is unique and initiated restructuring
at differing levels, prohibiting comparisons among schools by goal areas. Tables 5-7
show four stages of innovation: planning, implementing, improving, and refining with
respect to the remaining three goal areas: efficiency and effectiveness; the learning
climate; and, meeting the unique needs of target populations.
Demographically, the 15 project schools varied in ethnic and linguistic
composition, organizational configurations, grade levels served, geographic location,
and size. Eleven sites are elementary schools; four are high schools. Of the 11
elementary schools, six are configured to serve grades K-5; three are configured to
serve grades K-6; one serves grades K-8; and, one serves grades K-3. Eight of the
schools are located in Maricopa County; three are in the Tucson arjea; two are in
Yuma; one in Chino Valley; and one in Safford. The largest school participating in the
restructuring pilot project is Rancho Viejo, enrolling 1146 students; the smallest
school is Carminati, enrolling 380 students. Three schools, C.J. Jorgensen, Rancho
Viejo, and Westwood, enrolled a student population of which 85% or more of the
students received free or reduced lunch, indicating the low socioeconomic status of
the students and community. Five schools, C.J. Jorgensen, Palmcroft, Papago,
Rancho Viejo, and Westwood, enrolled students of which 50% or more of the
students were non-White. Each of these variables contributed to the mix of
restructuring efforts underway in the 15 project schools.
Overall Restructurinq
A rubric of seven stages of overall restructuring was constructed based on the
review of literature. Each consecutive stage implies a progression from learning about
restructuring to pursuing restructuring based on considerable experience and success.
All of the 15 project schools demonstrated progress on the identified levels of
attainment from 1991-92 to 1992-93. Ten of the 15 schools attained significant
levels at the routine level or above in 1993. Table 1 displays the seven stages of
overall restructuring. Figures 2-1 6 show the growth in overall restructuring made over
two years by school.
Table 1
Rubric A
Stages of Restructuring Attainable by Site Teams
Directions: Place an X in the column on the right that indicates your level of attainment in 1992.
Place a Y in the same column that indicates where you are in 1993.
Rating
By Year
Levels of
Attainment
Renewal
Integration
Refinement
Routine
Mechanical
Preparation
Orientation
Description
After considerable experience and success, the school is
pursuing its vision of restructuring in the context of
improving student achievement.
Restructured practice, governance, and professionalism of
teachers are institutionalized and consistently evident
throughout the school.
Based upon experience, the restructuring effort is
undergoing evaluation. The mission of the school is focused
on improving student achievement that has been
appropriately defined and measured.
School has engaged in restructuring, accounting for
alternative delivery strategies in curriculum, instruction, and
assessment, school-based decision making, and improved
professionalism of teachers.
School has initiated restructuring with a three-five year plan
for systemic change.
Stakeholders understand the relationship between
incremental reform and genuine restructuring and can
articulate a vision for a restructured school.
School is learning about restructuring and is acquainting
stakeholders with the needs to develop a vision, to clearly
articulate its educational practices, to develop a plan for
systemic change, and to refocus the mission on improving
student achievement.
Goal 1 : Increased Parental Involvement
The evolutionary nature of parental involvement ranges from apathy of parents
to take part in school activities to volunteering in classrooms to involvement in shared
decision making to parents who initiate restructuring activities and are actively
involved in the leadership of the school. Table 2 shows these seven developmental
stages of parental participation. Figures 2-1 6 demonstrate the levels of involvement
by parents by school in 1991-1 992 and 1992-93. Across all 15 schools, parental
involvement was defined as increasing parental participation in school events such as
parentlteacher conferences and special schoolwide activities. Significantly, every
school documented participation rates. Several schools, e.g. University High School,
Glendale High, Kyrene del Sureno, and Litchfield, defined parental involvement as
increasing voluntarism and participation in decision making committees. Some
schools, e.g. Palmcroft, Rancho Viejo, and Westwood, increased parental involvement
and measured the perceptions of parents about their opportunities to be involved in
meaningful activities in the school. Every school, except C.J. Jorgensen, increased
the level of parental involvement from 1991-92 to 1992-93 as evidenced by the
school profiles (Figures 2-1 6).
Table 2
Rubric D
Parental Participation
Levels of
Attainment
Renewal
Integration
Refinement
Levels and Types of Parental Participation
Several parents are initiating leadership of restructuring
activities based on prior experiences and successes.
Several parents are routinely involved in the leadership and
operations of the school.
Parents are routinely involved in all aspects of school life,
including the classroom, and some are beginning to be
meaningfully involved in school-based decision making.
Rating
By Year
Routine
Mechanical
Preparation
Orientation
Several parents are routinely involved in a variety of activities
in this school and understand school-based decision making.
Some parents are involved in this school in various ways such
as working in the library and office and volunteering to help
some teachers in classrooms.
Some parents are involved in this school primarily in special
events, parent-teacher conferences, and attendance at
student performances.
The majority of parents don't take advantage of opportunities
to be involved in school activities.
Goal 2: Increased Student Achievement
Defining and measuring student achievement appropriately are challenges for
all schools. Nationally, there is movement away from using single, standardized sets
of measures of student achievement. Yet, a clear path to measuring complex learner
outcomes remains obscure. The Arizona Student Assessment Program, with its
technical problems, represents the radical changes that are occurring across the nation
in student assessment. There are no appropriate data from the ASAP to assist
individual project schools with year-to-year comparisons. However, several of the 15
project sites utilized sophisticated curriculum-referenced measures that have been
aligned with local curricula to measure student achievement. Glendale High School,
Kyrene del Sureno, Westwood, and Litchfield exemplified appropriate use of pre-post
CRT assessments. Other project schools such as Rancho Viejo, Palmcroft, Gilbert,
Papago, Carminati, and Mountain View High used a variety of indicators to augment
test scores such as dropout rates, attendance, and participation rates in student-based
curriculum programs to document evidence of increased student achievement.
Safford High, Laguna, C.J. Jorgensen, and Del Rio relied more on published tests and
classroom-developed indicators. Because of its uniquely academically gifted student
population, University High School successfully measured student achievement on
traditional standardized instruments such as the SAT and ACT. Significantly, all of
the project schools showed evidence of thinking about student achievement
differently over the two-year evaluation period. Table 3 shows the evolutionary
nature of activities involving measuring student achievement. Figures 2-1 6 show the
levels of activity by project school from 1991 -92 to 1992-1 993. Significantly, 13 of
the 15 schools demonstrated progress in 1993 at the refinement level or above.
Table 3
Levels of Stages of Activity Concerning Defining and Measuring Student Rating
Attainment Achievement BY
Year
We use a variety of indicators of school success including the state's
ASAP, standardized tests, district tests, and outcome measures
developed by our teachers.
Integration We use performance measures in addition to other types of measures
~efinement
Routine
Mechanical
We are making changes in achievement measurement that reflect
student outcomes that ere more meaningful and important.
We use districtlschool-developed paper-and-pencil tests and other
indicators (e.g. attendance, dropout rates, erc).
We use districtischool-developed paper-and-pencil tests and published
tests.
Preparation
Orientation
We use published tests and other indicators (e.g.. attendance, dropout
rates, etc.).
We use published tests exclusively.
Focus G r o u ~In terviews Reaardina Student Achievement
To assist project schools with a clearer understanding of the importance of
defining and measuring student achievement appropriately in a restructured school,
45 focus group interviews were conducted in the 15 project schools. The purpose
of the interviews, conducted with parents, students, and teachers, was to determine
"what" is measured in schools, "how" it is measured, and how it could be measured
differently. The data were clustered to make comparisons of the comments offered
by parents, students, and teachers. Preliminary analysis revealed the following
themes among the comments. (Final analysis will be provided in the third-year
summative report. )
1. Teachers and parents generally disregard and distrust the
results of standardized tests.
2. Teachers generally support the use of the ASAP as a
measure of students' writing and critical thinking skills.
3. Teachers generally considered the administration of the
ASAP over the last two years as a trial effort to work out
the technical problems.
4. Parents generally regarded the ASAP as a more accurate
measure of student performance than a standardized test.
5. Teachers generally support the use of curriculum-referenced
tests, but are unclear as to how they are developed, who
develops them, and how they are used by district/school
administrators.
6. Parents generally are unaware of the content of curriculum-referenced
tests, their purposes, and where they come
from.
7. Teachers generally indicated that they know when students
have learned and that awareness doesn't come from tests.
Rather, the awareness generally comes from affective
means, e.g. facial expressions, behaviors, attitudes,
expressions of motivation, etc.
8. Students generally indicated that they learn best .from
projects, demonstrations, hands-on activities, writing,
science experiments, and math manipulatives.
9. Parents generally indicated that they know when their
children have learned when their students are happy about
going to school and talk about their accomplishments and
school activities.
10. The single common thread among all interview data was
that multiple indicators of student achievement have mole
meaning than any single indicator.
Goal 3: lm~rovedP rofessionalism and Coo~eration
During the last 20 years, educators have focused efforts to improve the
professionalism of teachers on staff development activities. Currently, all of the 15
project schools included staff development in their definitions of this goal area with
documented evidence of participation rates by staff. However, several of the project
schools offered additional evidence of the extent to which teachers and staff members
have increased time and effort to collaborate in planning for the delivery of curriculum
and instruction. More significantly, several of the schools have defined project
outcomes in this goal area in terms of shared decision making, an activity closely
linked with the goal area of improving efficiency and effectiveness. With technical
assistance from the Morrison Institute, most of the 15 schools are in various stages
of implementing a school-based decision making model of governance. For example,
shared decision making is the cornerstone of projects such as University High,
Glendale High, Papago, Palmcroft, and Rancho Viejo, each exhibiting varying degrees
of sophistication. Schools such as Palmcroft and Westwood conducted surveys of
parents, teachers, and students that offered substantial evidence of their progress in
improving the professionalism and cooperation among teachers and staff as well as
progress in shared decision making. Table 4 shows the levels of teacher
professionalism that range from teachers isolated by definition of their teaching
assignments to teachers with redefined roles as active participants in the governance,
curriculum, and planning efforts in their schools. Figures 2-1 6 demonstrate the
progress in professionalism and cooperation by school from 1991 -92 to 1992-93.
Twelve of the 15 schools increased their levels of attainment over the two year
evaluation period. Significantly, 12 of the 15 schools demonstrated progress in 1993
at the routine level or above.
Table 4
Rubric C
Teacher Professionalism
Directions: Place an X in the column on the right that indicates your level of attainment in 1992.
Place a Y in the same column that indicates where you are in 1993. -
Levels of
Attainment
Renewal
Integration
Refinement
Routine
Mechanical
Preparation
Orientation
Teacher Professionalism
Teachers have redefined their professional roles through
collaboration, meaningful contributions to school
governance, and cooperative efforts to change their
educational practices.
Teachers are working together to initiate restructuring
efforts based on prior successes.
Teachers initiate opportunities for professional development
and support is provided. Teachers routinely seek
collaborative relationships with colleagues to plan
curriculum and instruction activities. Teachers are routinely
involved in school-based decision making.
Teachers are routinely involved in school-based decision
making. A variety of restructured practices is integrated
into the curriculum, e.g., multi-age grouping, infusion of
technology, cooperative learning, etc.
Teachers are beginning to become involved in school-based
decision making. Teachers routinely meet to collaboratively
plan for restructured classroom activities.
Teachers plan curriculum activities together and routinely
meet to discuss student progress for purposes of designing
classroom activities.
Teachers are isolated by definition of their teaching
assignments. Little planned effort is made for collaboration
and cooperation.
Rating
BY
Year
School Profiles: Fiaures 2-1 6
The following school profiles, included alphabetically by school name, show the
levels of attainment with respect to seven developmental stages for four areas:
Overall restructuring; student achievement; professionalism; and parental participation.
Each profile demonstrates trends from 1991 -1 992 (darker block) to 1992-1 993
(lighter block) in each of the four areas. The preceding tables 1-4 indicate
descriptions for each of the seven developmental stages, ranging from orientation
(level 1) to renewal (level 7). Again, it must be noted that attainmeni at the routine
level or above is significant of efforts to genuinely restructure. It was expected that
few, if any, schools would attain rankings at level 7. Similarly, few, if any, schools
should have only attained level 1 after two years of project funding.
Figure 2
School Profile: C. J. Jorgenson
Growth from 1992 to 1993
a
7 Renewal
4 6 integration
5 Refinement
3 4 Routine
3 Mechanical
2 2 Preparation
1
1 Orientation
n V
Restructuring Achievement Professionalism Parent Part.
Rubrics by Goals (Senate Bill 1552)
Figure 3
School Profile: Carminati
Growth from 1992 to 1 993
1 1992'
6 1993
5
7 Renewal
4 6 lntegration
5 Refinement
3 4 Routine
2 23 PMreecphaaranticioanl
1 1 Orientation
0
Restructuring Achievement Professionalism Parent Part.
Rubrics by Goals (Senate Bill 1552)
Figure 4
School Profile: Del Rio
Growth from 1 992 to 1 993
1992
1993
6 7
5 7 Renewal
4
6 lntegration
5 Refinement
4 Routine
3 3 Mechanical
2 Preparation
2 1 Orientation
1
Restructuring Achievement Professionalism Parent Part.
Rubrics by Goals (Senate Bill 1552)
Figure 5
School Profile: Gilbert
Growth from 1992 to 1993
1 rn 1992
6
Hi 1993
5
7 Renewal
4 6 Integration
5 Refinement
3 4 Routine
3 Mechanical
2 2 Preparation
1 Orientation
1
n
Restructuring Achievement Professionalism Parent Part.
Rubrics by Goals (Senate Bill 1552)
Figure 6
School Profile: Glendale
Growth from 1 992 to 1 993
7 rn 1992
6 1993
5
7 Renewal
4 6 Integration
5 Refinement
3 4 Routine
3 Mechanical
2 2 Preparation
1 , 1 Orientation
0
Restructuring Achievement Professionalism Parent Part.
Rubrics by Goals (Senate Bill 1552)
Figure 7
School Profile: Kyrene
Growth from 1 992 to 1993
5
7 Renewal
4 6 Integration
5 Refinement
3 4 Routine
2
3 Mechanical
2 Prepamtion
1 I Orientation
A"
Restructuring Achievement Professionalism Parent Part.
Rubrics by Goals (Senate Bill 1552)
Figure 8
School Profile: Laguna
Growth from 1992 to 1993
6 '7
Restructuring Achievement Professionalism Parent Part.
Rubrics by Goals (Senate Bill 1552)
7 Renewal
6 lntegration
5 Refinement
4 Routine
3 Mechanical
2 Preparation
I Orientation
Figure 9
School Profile: Litchfield
Growth from 1992 to 1993
7 1992
6
1993
5
7 Renewal
4 6 Integration
5 Refinement
3 4 Routine
2
3 Mechanical
2 Preparation
1 1 Orientation
n"
Restructuring Achievement Professionalism Parent Part.
Rubrics by Goals (Senate Bill 1552)
Figure 10
School Profile: Mountain View
Growth from 1992 to 1993
Restructuring Achievement Professionalism Parent Part.
Rubrics by Goals (Senate Bill 1552)
Figure 11
School Profile: Palmcroft
Growth from 1992 to 1 993
6 1993
5
7 Renewal
4 6 Integration
5 Refinement
3 4 Routine
2
3 Mechanical
2 Preparabon
1 1 Orientation
0
Restructuring Achievement Professionalism Parent Part.
Rubrics by Goals (Senate Bill 1552)
Figure 12
School Profile: Papago
Growth from 1992 to 1993
rn 1992
6 '7 1993
5
4
5 Refinement
3
2
1
A V
Restructuring Achievement Professionalism Parent Part.
Rubrics by Goals (Senate Bill 1552)
Figure 13
School Profile: Rancho Viejo
Growth from 1992 to 1993
1992
6 '7 1993
5
7 Renewal
4 6 Integration
5 Refinement
3 4 Routine
3 Mechanical
2 2 Preparation
1
I Orientation
n"
Restructuring Achievement Professionalism Parent Part.
Rubrics by Goals (Senate Bill 1552)
Figure 14
School Profile: Safford
Growth from 1 992 to 1 993
5
7 Renewal
4 6 Integration
5 Refinement
3 4 Routine
3 Mechanical
2 2 Prepatation
1
I Orientation
n.,
Restructuring Achievement Professionalism Parent Part.
Rubrics by Goals (Senate Bill 1552)
Figure 15
School Profile: University
Growth from 1992 to 1993
7 BlBi 1992
6
1993
5
7 Renewal
4 6 Integration
5 Refinement
3 4 Routine
3 Mechanical
2 2 Preparation
1
1 orientation
n V
Restructuring Achievement Professionalism Parent Part.
Rubrics by Goals (Senate Bill 1552)
Figure 16
School Profile: Westwood
Growth from 1992 to 1993
7 Renewal
6 lntegration
5 Refinement
4 Routine
3 Mechanical
2 Preparation
1 Onentabon
Restructuring Achievement Professionalism Parent Part.
Rubrics by Goals (Senate Bill 1552)
Goal 4: lm~rovedE fficiencv and Effectiveness
Each of the 15 project schools was asked to indicate their progress with
respect to planning, implementation, improvement, or refinement of 19 areas of
restructured educational practice. The 19 areas on the matrix represent a spectrum
of restructured activities such as aligned curriculum/instruction/assessmentm odels,
shared decision making, alternative school calendars, and alternative student grouping
practices. No school was expected to demonstrate progress in all 19 areas as each
school has selected a focus for its restructuring effort. For example,-palmcroft and
Rancho Viejo are the only two project schools currently implementing or improving
year-round education programs. Table 5 aggregates the data across all 15 schools,
providing a frequency count of the innovations that are underway in the project
schools. Significantly, all of the schools showed progress in their respective areas of
focus over time.
Table 5
Profile of Restructuring Activity by Level of Development
Refining
0
4
3
2
0
5
1
2
5
3
2
4
2
4
2
3
4
2
1
Restructuring Activity
Year-round school
Multi-age grouping
Technology to improve communication
Technology to improve instruction
Technology to improve management
Integration of special and regular education
Community service projects
School-based decision making
Staff development
Interdisciplinary studies
Extended-day kindergarten
Cross-age and peer tutoring
Mentoring programs for faculty/students
Integrated curriculum
Extracurricular/extended-day program
Aligned curriculum-referenced testing
Performance testing
Portfolios assessment
Other:
Implement.
1
2
3
2
6
3
5
4
3
3
5
3
3
1
3
1
3
3
1
Planning
3
1
2
1
2
0
1
4
0
0
0
1
0
2
2
5
3
2
0
improving
1
4
4
9
7
6
4
5
7
4
2
5
3
5
4
6
3
5
0
Goal 5: lm~rovedS chool Climate/Learnina Environment
Each of the 15 project schools was asked to indicate its status with respect to
planning, implementing, improving, or refining its school and learning climate. Project
personnel rated their progress with respect to using multiple measures to determine
the attitudes of teachers, parents, and students regarding the school climate and to
rate the extent to which the relationship between an improved school climate and
improved student achievement is recognized and valued. Notably, several of the
schools, e.g. Palmcroft, Rancho Viejo, Kyrene del Sureno, and Westwood,
administered surveys to parents, teachers, and students that provided a rich data base
for making decisions about restructuring activities. Some of the schools surveyed one
group of stakeholders, e.g. University High surveyed students; Safford High and Del
Rio surveyed parents; Carminati surveyed teachers and parents. These findings
provided substantive evidence of the learning climates. All schools were encouraged
in their formative evaluation reports to systematically survey all stakeholders as a
measure of progress in all goal areas. Table 6 aggregates the data across all 15
schools, providing a frequency count of the extent to which the schools are measuring
and valuing an improved learning environment.
Table 6
School and Learning Climate
School and Learning
Climate
We are measuring our
school climate with
multiple measures of
teachers, parents, and
students.
We recognize the value
of an improved learning
climate and are working
on restructuring
activities (see Rubric E)
to improve our climate.
Improving
3
9
Refining
7
5
Planning
0
0
Implementing
5
1
Goal 6: Meetina the Uniaue Needs of Students
Each of the 15 project schools was asked to indicate the nature of the
innovations that addressed the needs of special student populations with respect to
planning, implementation, improvement, and refinement. Fourteen target populations
were identified on a matrix, ranging from gifted students to students with various
handicapping conditions. Each school enrolled unique student populations. Therefore,
no school was expected to respond in all 14 areas. Several schools, i3.g. Glendale
High, Palmcroft, and Rancho Viejo, have uniquely addressed the language needs of
students whose primary home language is other than English. Other schools, e.g.
Kyrene del Sureno, Gilbert, and Carminati, have addressed unique needs through
multi-age grouping patterns or through the integration of special and regular education
programs. All project schools documented evidence of providing programs for their
respective targeted populations, each exhibiting varying degrees of innovation. Table
7 aggregates the data across all 15 project schools. These findings provide a
frequency count of the schools that are using restructuring funding to meet the unique
needs of students with special challenges.
Table 7
Meeting Unique Needs of Students'
I 'unique needs met through restructuring project
Summarv of Findinas Related to Evaluation Question One
Each project school's restructuring efforts in the six goal areas of S.B. 1552
(1 990) were evaluated with multiple measures. The Formative Evaluation Report
provided to each school summarized specific accomplishments in each goal area,
analyzed those accomplishments in the context of stated projected outcomes as well
as recommendations for future activities, and commended the schools for specific
innovations, procedures, and practices. This Summative Evaluation Report presented
the findings from a variety of data collected by the external evaluators.
The commendations offered to each school in the Formative Evaluation Report
were designed to encourage schools to continue those practices that are research-based
or are showing promise of improving student achievement. Table 8 groups the
commendations offered to schools based on their accomplishments during 1992-93.
These commendations are important because they illustrate the specific strengths of
each school project.
Table 8
Strengths of Project Schools
Prepared clear self-evaluation reports ...% ......
Demonstrated understanding of relationship
between reform and genuine restructuring
Collected meaningful data from parents,
teachers, and students as evidence of
progress in six goal areas
Wove aspects of restructuring into a
cohesive, focused project across all six
goal areas
Demonstrated evidence of significant
innovations in curricular and
instructional practices
Demonstrated exemplary progress with
shared decision making
0 Q Q @ ~ ~ ~ Q ~ - J @ Q ~ @
Q
Q
0..... 0
I
Designed project to meet complex
needs of families through linkages with
social service agencies
Critically analyzed progress in six goal
areas with candor and honesty
Fully cooperated with self-evaluation
and external evaluation efforts
0* I
0 Designed age- and content-appropriate
programs for young children
Defined and measured student achievement
in terms of an aligned curriculum/instruction/
assessment model
Endured significant mitigating
circumstances/barriers to restructuring
~ 0 C
0
("J
f-J
,=J
t
..:..:.:.; ..-..
1
0
........ :.... :..
jgg
....:...; >... :.
3
O Q ( " 1
(-38
.:<<>.
1
I
1
I 0
B. Question Two: How can each of the 15 school sites im~roveth eir restructuring
efforts?
The improvement of individual restructuring efforts in the 15 project schools
will be discussed in two dimensions: 1. A summary of the process of self-evaluation
in which each school engaged; and 2. A summary of the recommendations offered to
each school with respect to specific restructuring outcomes, activities, and results.
-
Process
Because of the evolutionary nature of genuine restructuring, each school's
restructuring activities varied, depending upon their initiation points. However, the
process of studying and evaluating programs and practices was constant across
schools in that the self-evaluation concept was new to all of the project personnel.
In reviewing the self-evaluation reports, which were all formatted similarly based on
the model reports provided for each school, four common themes emerged: Outcome
identification; Outcome assessment; Critical analysis of progress; and, Directions for
the future. These themes are summarized across all 15 schools in Table 9, providing
a profile of the levels of sophistication at which the schools conducted their self-evaluation
studies. The most marked difference among the reports was the extent to
which the schools were willing to honestly discuss their progress. All public schools
are subject to considerable scrutiny and criticism. Therefore, it was assumed by the
external evaluators that few educators are willing to take the necessary risks to
critically analyze their successes and failures in terms of using what they know to
make decisions about future directions. Yet, the ability to collect appropriate data and
use it to make decisions has enormous potential to impact the lives of students,
teachers, and parents. Table 9 highlights those project schools that were successful
in four areas as well as those that can improve.
Recommendations about Restructuring
In each formative evaluation report, recommendations were made to each
project school regarding the nature of their restructuring activities across the six goal
areas. Table 10 summarizes the recommendations offered to each school that have
the most potential to make a difference in the evolution of its restructuring effort.
Table 9
Formative Evaluation Summary
Based on Self-Evaluation Reports -
OUTCOME DEFINITION
Defined prqect outcomes that are
charadenstlc of genuln. restructuring
l l l l l l l l l l l l l I 1
Defined plsxt &comes that are
Oefined proled wtcomes charadenslc !
of tnal-and-ermr pcacttces
OUTCWE ASSESSMENT
I l l l / l ! I ! l ! i l l
Cdledmi'anam mean~ngfudl ata from
pamm, :exhen and students
Did nd sysyslematically cc!lect data
to evaluate prqect wlcoines
I I Cdleded,e&taandsbhW 'B~deiol~/ CI/ b\ la)i@ioj
CRITICAL ANALYSIS
Cntlcally analyzed and drew conclus~ons
that wlll !mcact Mum outcomes
ev~denceo f hav to anzlyze and dispay
Discussed m r e s s regarding gcals !
ard proled outcomes l l !
!
Discussed prcgress based on I I
marketrng prqat o audiem /@I l#q,al 1 s 1 1 . 11 1 1 1 @l 1
DIRECTION FOR FUTURE
Demomtrated dear focus fw
tuture urtcornes 1 1 [$lQi , , /@[&Jd , , 4 1 I i
Demonstrated some d~rectron~ n
the CUlt& Of key ad~ltles
3emanstrated lack of diredlon
because focus has shlRd
Table 10
Summary of Recommendations Offered to Project Schools
It is recommended that:
-
C.J. Jorgensen pursue an understanding of the fundamental differences between
reform and genuine restructuring in order to withstand the substantial internal barriers
that have occurred during the project funding cycle. Successful restructuring can
occur only when the political climate either allows the stakeholders to plan,
implement, and evaluate systemic change or the stakeholders are prepared to
manipulate the political climate.
Carminati pursue the development of curriculum-referenced tests in all content areas
that provide a richer picture of student achievement at all grade levels.
Del Rio pursue the design and development of an aligned
curriculum/instruction/assessment model so that there is a consistent match between
what is taught and what is tested across all grade levels.
Gilbert extend its data collection efforts and analysis procedures so that more
illustrative information is provided regarding student performance on curriculum-referenced
tests and so that evidence of the effectiveness of shared decision making
is available.
Glendale High consider collecting data from teachers and students regarding their
perceptions of the learning environment as well as of specific curricular programs and
practices.
Laguna pursue the design and development of an aligned
curriculum/instruction/assessment model so that stakeholders can systematically
participate in determining what is to be taught at all grade levels, and ensuring that
what is taught is tested and vice versa.
Litchfield pursue the formalization of its shared decision making process and collect
information from teachers as to their perceptions of the extent to which their input is
valued, utilized, and meaningful to the daily operations of the school.
Kyrene del Sureno provide evidence of the effectiveness of its committee structures,
which facilitate shared decision making, by gathering information from the
participants.
Mountain View continue to formalize its process for shared decision making. Efforts
are underway and clear evidence was provided about current progress, indicating that
there is some resistance, as anticipated, from teachers.
Palmcroft pursue the design and development of curriculum-referenced tests that
provide a more accurate measure of student achievement in the context of the local
curriculum and the State Essential Skills.
Papago refine its procedures for displaying data and drawing appropriate conclusions
from the collected evidence across all six goal areas (S.B. 1552, 1990).
Rancho Viejo pursue the design and development of an aligned
curriculum/instruction/assessment model that accurately reflects local curriculum and
the State Essential Skills.
Safford High continue to refine its scopes and sequences at the department level (a
school improvement activity) with an eye to developing an aligned curriculuml
instruction/assessment model that is more characteristic of restructuring.
University High analyze the data collected from students in the Senior Exit
Questionnaire to provide more information about students' perceptions of programs,
curriculum, and the climate.
Westwood pursue formalization of a shared decision making model and analyze its
effectiveness in the context of determining direction for future activities.
Summarv of Findings Related to Evaluation Question Two
The rubric evaluations of overall restructuring offer the clearest evidence of how
the 15 project schools might improve their restructuring efforts. Figure 17 shows
growth trends for all 15 project schools over two years with respect to overall
restructuring. Referring back to Table 1 in which the stages of restructuring are
described, each school has information to formulate its direction, in conjunction with
the recommendations and commendations offered in the Formative Evaloation Report, -
for future years.
Figure 17
Stages of Overall Restructuring
Seven-point benchmarks
A B C D E F G H I J K L M N O
Project Schools
7 Renewal-School is pursuing a new vision
6 Integration-Evident throughout the school
5 Refinement-Focused and improving
4 Routine-Procedures in place
3 Mechanical--Initiating restructuring
2 Preparation-Planning restructuring
1 Orientation--Learning about restructuring
0 Unaware
C. Question Three: How effective are Arizona's efforts to offer incentives to schools
to restructure?
To answer this evaluation question, a survey of restructuring project teachers
(N = 390) was conducted, parallel to a survey disseminated by the Arizona Department
of Education to randomly selected schools throughout Arizona. The purpose of
surveying project teachers was to obtain their perceptions about decentralization
issues. For this Summative Evaluation Report, the data have been aggregated across
all 15 schools by survey item. The findings are presented in a series of figures with
discussion. The third-year summative report will disaggregate the data by school and
will present comparisons between the restructuring project teachers' attitudes about
decentralization and the statewide sample. These comparisons will offer substantive
evidence of the extent to which S.B. 1552 (1 990) has impacted the attitudes of
project teachers about shared decision making and related governance issues.
Survev Desian
The survey of teachers was formatted into three categories: Participant
Characteristics; Current Practices; and, Restructuring. The items clustered under
current practices assessed the teachers' perceptions of their levels of satisfaction
pertaining to specific classroom issues, influence over classroom and student issues,
requirements to use textbooks, and decision making authority. The items clustered
under restructuring assessed the teachers' perceptions of who should have the most
influence in decisions concerning classroom and schoolwide issues, school level
autonomy, willingness to serve on school-based decision making committees, and the
teachers' perceptions of whether restructuring is occurring in their schools. The
surveys were bulk-mailed out to project teachers and returned through the mail by
school. Teachers were not identified by name. The response return rate was 86%.
Findinas
Figure 18 graphs the degree of satisfaction indicated by project teachers
(N =390) regarding six current practices in the classroom. The scale, 0-100,
represents four equal interval quadrants ranging from very unsatisfied (0-25),
unsatisfied (26-50), satisfied (51 -75), to very satisfied (76-1 00). The mean responses
are graphed, suggesting that teachers are satisfied with the selection of core
materials, content, topics, and skills that are taught, the grading system, the discipline
of students, and the use of physical space.
Figure 18
ltenn Satisfaction With Classroom Issues r 1
I
very 0 20 40 XI BO very 100
unsatisfied DEGREE OF SATSMCTW
satisfied
4. On the whole, how satisfied are you with the following
current cbssroom issues?
a. selection of core materials
b. content, topics, and skills that are taught
c. sequence in which conlent, topics.and skills are
taught
d. gtading paradigm or system
e. discipline of students
f. use of physical classroom space
Table 11 indicates the percentage of teachers (N =390) that felt they had
adequate influence over six issues. The responses suggest that the project teachers
significantly felt they had adequate influence over the selection of core instructional
materials, the content, topics, and skills that are taught, grading paradigm or system,
discipline of students, and use of physic.al classroom space.
Table 1 1
Teacher Influence in the Classroom
5. Do you personally feel you have adequate
influence wer the following issues that
aft& your students?
items 1 per cent per cent
a. selection of core instructional materials
b. content. topics, and skills that are taught
c. sequence in which content, topics. and
skills are taught
d. grading paradigm or system
e discipline of students
f. use of physical classroom space
ltem 6 addressed the extent to which teachers felt they had decision making
authority in their classrooms. Of the 390 responding teachers, 85% felt they had
adequate authority and 15% felt they did not. These findings corroborate other
findings that show that 5-1 5% of teachers never feel they have an adequate voice in
determining what they do in schools (Aquila & Galovic, 1988; Rogers, 1971).
However, in restructuring schools, we might hope to find a lower percentage of
teachers who feel helpless to control the decisions made in the classroom about
students and instruction. -
ltem 7 addressed textbook policies. Teachers were asked if they were required
by policy to use specific textbooks as primary teaching tools. Of the 390
respondents, 254 indicated that some policy requiring the use of textbooks existed
in their schools. Table 12 shows the percent of classes/subjects in which these 254
respondents felt textbooks were required for use. These findings indicate a heavy
reliance on textbooks both by policy and in practice.
Table 12
Percent of Classes/Subjects That Require Textbooks
Figure 19 graphs the degree of satisfaction indicated by project teachers
(N = 390) regarding eight schoolwide issues. The scale, 0-1 00, represents four equal
interval quadrants ranging from very unsatisfied (0-251, unsatisfied (26-50), satisfied
(51 -75): to very satisfied (76-1 00). The mean responses are graphed, suggesting that
teachers are satisfied with the subject/class assignments, the hiring of teachers and
administrators, the promotion of personnel, allocation of funding, and the length of
the school day and class periods. However, teachers indicated that they are
unsatisfied with the determination of teachersr salaries in their schools.
I Figure 19
1 Satisfaction With Schoolwide Issues
8 Items
a
I
I I I I ! I 1
0 20 40 60 80 100
DEGREE OF SATISFACTION
I 8. In the following schoolwide issues, on the
I whole, how satisfied are you with the current:
a. subject class assignments
I b. hiring of teachers
c. hiring of administrative personnel
d. promotion of personnel m e. allocation of funds within your school
f. determination of teachers' salaries in your
0 school
g. length of class periods
1 h. length of school day
Figure 20 graphs the degree of satisfaction indicated by project teachers
(N = 390) regarding the extent to which they felt they had adequate influence over
eight issues. The scale, 0-1 00, represents four equal interval quadrants ranging from
very unsatisfied (0-25), unsatisfied (26-501, satisfied (51-751, to very satisfied (76-
100). The mean responses are graphed, suggesting that teachers are satisfied with
their influence over class and subject assignments, hiring of teachers and
administrators, promotion of personnel, allocation of funding, and the length of the
school day and class periods. Teachers felt they were unsatisfied with the influence
they have in the determination of teachers' salaries in their schools.
Figure 20
Satisfaction With Influence Over Schoolwide Issues
very 0 ZP 40 eLl 80 very
1 @I
unsatisfied DEGREE OF SATISFACKN
satisfied
9. Considering your position as a classroom teacher
within your schod, do yw personally feel that you
have adequate influence owr the following?
a. subject class assignments
b. hiring of teachers
c. hiring of administrative personnel
d. promotion of personnel
e. allocation of funds within your school
f. determination of teachers' salaries in your school
g. length of class periods
h. leogth of schod day
ltem 10a asked the teachers to indicate which of seven groups of people
currently have the most influence over decisions concerning classroom issues. Figure
21 shows the percentage of teachers responding yes to the respective groups. Sixty
percent of the 390 respondents felt that the principal currently had the most influence
over classroom decisions while only 25% of the respondents felt that a school site
committee had influence. ltem 10b asked the teachers to indicate which of seven
groups currently had the most influence over schoolwide issues. Figure 22 shows the
percentage of teachers responding yes to the respective groups. -Of the 390
respondents, 63% felt that the district office1 superintendent had the most influence
over schoolwide issues, while 60% of the teachers felt that the principal had the most
influence over schoolwide issues. These are significant findings, suggesting that even
among restructuring school project teachers, there remains a perception that central
office personnel exert considerable control over school-based issues. Even more
significant, only 23% of the teachers felt that the teachers as a group had influence
over decisions concerning schoolwide issues. These findings suggest that shared
decision making at the school level is still in its infancy.
Figure 21
Decision Making Authority
Classroom Issues
10 a. In your opinion, which group (s) CURRENTLY
have the most influence in decisions made concerning
classroom issues?
State BoardIADE
District Office/Supt
Local School Board
Principal
Teachers
ParentsICommunity
Site Committee
0 20 40 60 80 100
N=W Per cent answering yes
Figure 22
Decision Making Authority
Schoolwide Issues
10 b. In your opinion, which group (s) CURRENTLY
have the most influence in decisions made concerning
schoolwide issues?
State BoardIADE
District Office/Supt
Local School Board
Principal
Teachers
ParentsICommunrty
Site Committee
0 20 40 60 80 100
N = W Per cent answering yes
In contrast, items 11 a and b asked the teachers which groups should have the
most influence over decisions concerning classroom issues and schoolwide issues.
Figures 23 and 24 graph the percentage of respondents who felt that decision making
authority should rest with respective groups. Over 90% of the responding teachers
felt that decisions about classroom issues should reside with teachers and 70% of the
respondents indicated that decisions about schoolwide issues should be made by
teachers.
Figure 23
Importance of Decision Making Authority
Over Classroom Issues
1 la. In your opinion, which group (s) SHOULD
have the most influence in decisions made concerning
classroom issues?
State BoardIADE
District Office/Supt
Local School Board
Principal
Teachers
Parentdcomrnunity
Site Committee
0 20 40 60 80 100
N = 390 Per cent answering yes
Figure 24 -
Importance of Decision Making Authority
Over Schoolwide Issues
11 b. In your opinion, which group (s) SHOULD
have the most influence in decisions made concerning
schoolwide issues?
State BoardlADE
District OfficeISupt
Local School Board
Principal
Teachers
Parents/Community
Site Committee
0 20 40 60 80
N - 390 Per cent answering yes
The project teachers were asked to respond to questions about the locus of
decision making with respect to student interests. Table 13 shows the percentage
of teachers that strongly agreed, agreed, disagreed, or strongly disagreed that
students' interests would be best served if classroom and schoolwide decisions were
made at the school site. The high percentages of respondents who agreed or strongly
agreed that students' interest would be better served if decisions about classroom
issues and schoolwide issues were made at the school level offer compelling evidence
to involve more teachers, parents, and students at the school level in-the decision -
making process.
Table 13
Would your students' interests be best sewed if the following decisions werefare made at
the school site?
When asked if they would participate in a school-level decision making
committee, 85% of the 390 respondents indicated that they would. These findings
are consistent with previous findings suggesting that 15% of teachers are not willing
to participate in decision making or any other kind of activity that requires additional
effort.
Finally, when asked if their school had taken any steps toward restructuring,
62% of the responding restructuring project teachers indicated that their schools had
enacted a plan. Interestingly, 26% of the teachers indicated that planning had begun;
10% indicated that possibilities had been discussed; and 1 % indicated there had been
no steps taken to restructure. These findings are very revealing given that these
responses came from teachers who are teaching in schools with funded restructuring
projects!
Conclusions
These findings suggest several themes about the teachers' perceptions of
programs and practices, aggregated across all 15 project schools. These themes offer
evidence from teachers' perspectives of the extent to which Senate Bill 1552 (1 990)
has impacted restructuring in the project schools. A more complete picture of the
impact of the legislation will be presented in the third-year summative report which
will disaggregate these data by school and make comparisons to a statewide sample
of teachers who have not taught in schools with funded restructuring projects.
These findings suggest that significant numbers of responding project teachers felt
that:
1. They are satisfied with current practices pertaining to
classroom issues.
2. They are satisfied with their influence over issues that
affect their students.
3. They are satisfied with current practices pertaining to
schoolwide issues except for the determination of teachers'
salaries.
4. They are satisfied with their influence over classroom
issues.
5. They felt that the Principal still exerts control over decisions
concerning classroom issues.
6. They felt that the central office still exerts control decisions
concerning schoolwide issues.
7. They felt that teachers should have the most influence over
decisions concerning classroom issues.
8. They felt that teachers, a school-site committee, and the
principal should have the most influence over decisions
concerning schoolwide issues.
9. They are willing to serve on school-based committees. - -
10. They perceive the extent to which their schools are
involved in restructuring differently than do their
administrators.
D. Question Four: How can Arizona im~roveit s statewide effort to restructure
schools?
A response to this evaluation question will be addressed by a series of
observations that have been generated from the project schools and by the external
evaluators. The project schools were asked to prepare a set of mitigating
circumstances/barriers that prevented or impeded their restructuring progress. The
data are presented in the following table, indicating a ranked list, from-high to low,
of circumstances most often incurred by the project schools. Based on the frequency
of responses, the circumstances have been divided into major barriers and minor
barriers.
Inadequate evaluation support
Crowded/inadequate facilities
Glitches in installing technology
'Major barriers reflect responses from five or more schools.
Based on the data gathered from the self-evaluation studies, on-site
observations in each project school by a team of external evaluators, the rubric
evaluations, survey data, and the review of literature, the following observations are
offered to Arizona's policy makers regarding the Restructuring Incentives Pilot School
Program:
1. Top-down mandates from policy makers, central offices, or
governing boards to restructure schools are the antithesis
of genuine restructuring. Therefore, encouraging schools
to take risks to conduct business differently by offering
financial assistance that is not available to other schools is
an appropriate incentive.
. Selecting schools that are already engaged in planning for
genuine restructuring to participate in a statewide effort is
a critical aspect of a successful pilot program. The
participation of schools that were simply seeking additional
funding for school improvement or that lack sufficient funds
to carry on traditional programs has been a considerable
barrier to statewide success.
3. Framing legislation in the context of offering incentives to
schools to improve student achievement through shared
decision making, an aligned model of curriculum/instructionl
assessment, and an emphasis on redefined roles of
teachers will significantly impact the ability of project
participants to focus their outcomes on genuine
restructuring.
4. Encouraging schools, through financial incentives, to
implement research-based programs and practices and to
track their successes and failures through systemic data
collection with critical analyses of results will significantly
impact further pilot efforts.
5. Recognizing the demands that genuine restructuring has on
facilities or the impact that modern, carefully designed
facilities has on the opportunities for parents, teachers, and
students to engage in restructuring should be factored into
the mix of selecting schools for participation in a pilot
program.
6. Schools, educators, and communities must have the
capacity to make meaningful changes. The political
climates of schools and districts, the financial
infrastructure, and the stakeholders' willingness to take
risks all must be considered prior to a commitment to
include schools in a pilot project to engage in restructuring.
-
7. The selection process for the participation of schools in a
restructuring project should include a rigorous on-site
interview with parents, teachers, students, and
administrators in addition to an application procedure.
IV. Conclusion
This report has summarized the evaluation process and procedures that were
applied to the Restructuring Schools Incentives Pilot Program during the 1992-93
school year. In addition, the report summarized the findings from the 15 project
schools, organized as responses to four evaluation questions.
The Formative Evaluation Report, a compilation of 15 individual reports, and the
Summative Evaluation Report provide a comprehensive overview of the outcomes,
activities, and results of restructuring efforts undertaken by the 15 project schools.
Policy makers can be proud of their efforts to appropriately encourage genuine
restructuring in Arizona schools. To the extent that the process of restructuring is
evolutionary, the 15 pilot sites ought to be commended for asking tough questions
about schooling and responding with heartfelt activities designed to improve education
in public schools in Arizona.
References
Akpe, C. S. (1991 ). School self-evaluation: An examination of the state of the art
in Nigeria. Studies in Educational Evaluation, 1 7, 1 1 7 + .
Alexander, D., & Alexander, K. (1 991 ). American public school law. St. Paul, MN:
West Publishing Company. -
Anderson, R. (1 972). How to construct achievement tests to assess comprehension.
Review of Educational Research, 42, 1 45- 1 70.
Aquila, F., & Galovic, J. (1988). The principal as a change agent: Encouraging
teachers to adopt change. NASSP Journal, 72, 50 + .
Argyris, C., & Schon, D. ( 1 977). Theory in practice: Increasing professional
effectiveness. San Francisco: Jossey-Bass.
Banathy, B. (1 990). The transformation of education by design. Paper presented at
the annual meeting of the American Educational Research Association, Chicago.
Bank, A,, & Williams, R. (1989). Overcoming the obstacles to site-based
management. Thrust, 17-20.
Betteth, D. (1 988). Restructuring schools: Reform for everyone? Restructuring
teaching: Tower of Babel or shared vision? Teaching Exceptional Children, 2 1, 44-
51.
Cibulka, J. (1990). American educational reform and government power.
Educational Review, 42, 97- 108.
Cole, N. S. (1990). Conceptions of educational achievement. Educational
Researcher, 79, 2-7.
Cook S. J., & Haladyna, T. M. (1 992). Technical report 92: 1: School incentives
restructuring pilot project. Phoenix: Arizona State University West.
Goodlad, J. (1 988). Why our schools don't get much better and how they might.
Teacher Education Quarterly, 17.
Gutherie, J. (I 990). The evolving political economy of education and the implication
for educational evaluation. Educational Review, 42.
Haladyna, T. M., & Cook, S. J. (1992). Evaluation in a restructuring environment.
Paper presented at the annual meeting of the Arizona Educational Research
Organization. Phoenix.
Haladyna, T. M., Nolen, S. B., & Haas, N. S. (1991). Raising standardized
achievement test scores and the origins of test score pollution. Educational
Researcher, 20, 2-7.
Hall, G. & Hord, S. (1 987). Change in schools: facilitating the process. Albany, NY:
SUNY Press.
Holly, P. (1990). Catching the wave of the future: Moving beyond school
effectiveness by redesigning schools. School Organization, 10, 195-21 2.
Joyce, B. (Ed .) (1 990). Changing school cultures through staff development.
Alexandria, VA: ASCD.
Kearns, D. T. (1 988). An educational recovery plan for America. Phi Delta Kappan,
69(8), 656-570.
Koehler, M. (1 990). Self assessment in the evaluation process. NASSP Bulletin, 74,
40+.
Lieberman, A., & Miller, L. (1990). Restructuring schools: What matters and what
works . Phi Delta Kappan, 7 1 ( 7 0).
McCune, S. (1 988). Directions for restructuring schools for the future. Noteworthy.
4 Mid-Continent Regional Educational Laboratory.
National Governors' Association. (1 989). Proceedings of the National Governors'
Association Annual Meeting, 7/24). Washington, DC.
Niebur, L., Cook, S. J., & Haladyna, T. M. (1 992). Restructuring the vision. Paper
presented at the annual meeting of the Arizona Educational Research Association,
Phoenix.
Olson, L. (1992). Quality is job one: Changing schools and classrooms. Teacher
Magazine.
Pazaruski, A., & Rasp, A. Jr. (1990). Self-evaluation: A stimulus for school
improvement. Spectrum: Journal of School Research and Information, 8/31, 27.
Rigden, D. ( 1 990). School restructuring: Designing schools for better student
learning. New York: Precollege Series, Council for Aid to Education.
Rogers, E. (1971 1. Diffusion of innovations. New York: The Free Press.
Sizer, T. R. (1 992). Horace's school: Redesigning the American high school. Boston:
Houghton Mifflin.
Tyack, D. (1 990). Restructuring in historical perspective: Tinkering toward utopia.
Teachers College Record, 92, 1 70- 1 9 1 .
Tye, K. A. (1992). Restructuring our schools: Beyond the rhetoric. Phi Delta
Kappan, 4, 8-1 4.
Uphoff, N. (1 991 ). A field methodology for participatory self-evaluation. Community
Development Journal, 26, 27 1 + . -
Wiggins, G. (1989). Teaching to the (authentic) test. Educational Leadership, 76,
41 -47.