Performance audit, Arizona Department of Education, selected responsibilities and programs

              PERFORMANCE AUDIT
ARIZONA DEPARTMENT OF EDUCATION
SELECTED RESPONSIBILITIES AND PROGRAMS
Report to the Arizona Legislature
By the Auditor General
DOUGLAS R. NORTON, CPA
AUDITOR GENERAL
STATE OF ARIZONA
OFFICE OF THE
AUDITOR GENERAL
July 2,1996
Members of the Arizona Legislature
DEBRA K. DAVENPORT, CPA
DEPUTY IIYD4TOr) GENERIL
The Honorable Fife Symington, Governor
The Honorable Lisa Graham Keegan
State Superintendent of Public Instruction
Transmitted herewith is a report of the Auditor General, A Performance Audit of Selected
Responsibilities and Programs of the Arizona Department of Education ( ADE). This report
is in response to a May 30,1995, resolution of the Joint Legislative Audit Committee. This
performance audit was conducted pursuant to the provisions of A. R. S. 541- 2958. This is
the third in a series of four audit reports regarding public education.
This audit focused on ADE's financial management responsibilities and on programs
concerned with the quality of student education in Arizona. We found ADE needs to
improve its oversight and management of state aid for transportation and basic state aid
monies distributed to school districts. ADE needs to review and propose statutory changes
in the transportation funding formula to address flaws that reward districts with
inefficient operations, and penalize districts with efficient operations. In addition, ADE
should ensure audits are performed of the average daily membership counts used to
determine the amount of funding districts receive. We also found ADE can increase the
usefulness of the school report cards by increasing the amounts hnd types of information
reported. Finally, ADE should review and recommend to the State Board of Education
additional uses for the Arizona Student Achievement Program.
My staff and I will be pleased to discuss or clarify items in the report.
This report will be released to the public on July 3,1996.
Sincerelv.
DOUR. NNort on
Auditor General
Enclosure
I 2 9 1 0 NORTH 44TH STREET rn SUITE 4 1 0 m PHOENIX, ARIZONA 8 5 0 1 8 rn ( 6 0 2 ) 5 5 3 - 0 3 3 3 . FAX ( 6 0 2 ) 5 5 3 - 0 0 5 1
SUMMARY
The Office of the Auditor General has conducted a performance audit of selected
responsibilities and programs of the Arizona Department of Education ( ADE). Ths audit was
conducted pursuant to the provisions of A. R. S. 541- 2958 and in response to a May 30,1995,
resolution of the Joint Legislative Audit Committee. This is the thrd in a series of four audit
reports regarding public education.
Arizona statutes divide state responsibility for education among the State Board of Education,
the Superintendent of Public Instruction, and the Department of Education.' Under statute,
the State Board establishes educational policies. The Superintendent is responsible for
implementing the Board's policies and for managing the Department of Education. The
Department provides services and assistance to schools, districts, and county superintendent
offices.
This audit focused on the Department of Education's financial management responsibilities
and on programs concerned with the quality of student education in Arizona. The Department
distributed approximately $ 2 billion to educational institutions in fiscal year 1995, including
state- appropriated monies, federal grants, and other monies. This audit reviewed ADE's
financial management responsibilities for ensuring that state aid and state and federal grant
monies are distributed accurately, equitably, and timely. In addition, the audit reviewed two
programs relating to quality of education: the school report card program and the Arizona
Student Achievement Program ( ASAP). Anzona, like many other states, is implementing these
programs to assess and improve school performance, and to provide parents with critical
information they need to assess and select schools for their chldren.
Financial Management
Improvements Needed
( See pages 7 through 14)
The Department of Education needs to improve its oversight and management of state aid for
transportation, basic state aid monies, and federal and state grant monies distributed to school
districts.
1 Two other entities outside the scope of this audit also exercise state- level oversight. The State Board for
Vocational and Technological Education exercises general supervision over vocational and technological
education. The State Board for Charter Schools grants charter status to qualifying applicants, exercises
general supervision over charter schools it sponsors, and recommends legislation pertaining to charter
schools.
ADE has not adequately overseen monies used to reimburse school districts for transportation
expenses. Many school districts are reimbursed more than actual pupil transportation costs
because the State's transportation funding formula is flawed. For school year 1994- 1995, the
State paid $ 121 million to districts for transportation, although actual district transportation
costs totaled only $ 102 mlllion. Of the 205 districts that received transportation reimbursement,
145 districts were reimbursed more than actual expenses and 60 had transportation expenses
that exceeded state reimbursements. The current formula primarily reimburses districts for
total annual route miles; thewfore, the more miles a district records as being driven, the more
money it receives. In addition, a 1993 statutory revision further flawed the transportation
funding formula by allowing some districts to bill the State twice for the same miles for a certain
group of students. In one district, this amounted to $ 3.6 million in fiscal year 1996. This could
occur again unless the Legislature addresses this flaw in the funding formula. Moreover, ADE
does not verify transportation expense information submitted by school districts. However,
since the State Superintendent of Public Instruction is statutorily responsible for determining
the accuracy of district transportation information, audits of this information should be
performed.
ADE also needs to improve its oversight of basic state aid monies distributed to school districts.
To improve the accuracy of basic state aid distributions to school districts, audits of districts'
average daily membership ( ADM) counts should be performed. The Legislature appropriated
more than $ 1.4 billion in basic state aid in fiscal year 1995, based primarily on ADM counts.
The Department stopped performing ADM audits in May 1995, even though this function had
identified significant problems with ADM counts. For example, ADM errors resulted in one
district receiving $ 466,622 more in state aid than it should have received. Other states' practices
and continued ADM problems in h o n a support the need to audit ADM counts. ADE could
reinstate an ADM audit function w i hth e Department or expand the work that certified public
accountant ( CPA) firms currently perform in school districts.
Finally, ADE needs to improve its grants administration duties. Currently, the Department is
working to improve its administration of federal and state grant monies to school districts.
The Department's process for administering grants has suffered from poor enforcement of
application due dates, lack of staff responsiveness to district inquiries, and failure to process
some grants in a timely manner. The Department has proposed the establishment of a grants
management office to provide improved grants administration.
Arizona School Report
Cards Can Be More Useful
( See pages 15 through 19)
Recent statutory changes required the Department of Education and schools to implement a
school report card program. Ths program is designed to provide increased school
accountabihty and more information to parents and students to help them make educational
choices. The Department of Education developed the format for and printed the school report
cards, whch contain such required information as school academic goals, programs, and a
summary of student assessment scores. In April 1996, ADE sent the h tre port cards to schools
for distribution to parents. Parents and others interested in school performance, however, may
have dlfhculty using school reports cards to compare student achevement levels between school
districts, because each district selects its own assessment method and establishes its own passing
score. The Department of Education should recommend that the State Board of Education adopt
a standardized statewide assessment and passing score for each grade tested to allow
comparable test scores to be reported on the school report cards.
Arizona's school report cards could be made more useful by increasing the types and amount
of information reported. Other states provide additional achievement information on high
school report cards, such as college placement exam scores and the number of graduates
employed or in college, to help indicate how effective schools are at preparing students for
college or the workforce. Providing additional statistical information and data from more than
one year concerning school safety, student mobility, student- teacher ratios, and dropout rates
could also better Inform parents. Student demographic profiles, which can influence test scores,
can also help make comparisons of schools more fair. In addition, reporting financial
information, such as the amount of money spent on noninstructional items, can encourage
school districts to be more accountable.
Additional Uses for ASAP
Should Be Considered
( See pages 21' through 25)
The State Board of Education has identified problems with the Arizona Student Achievement
Program ( ASAP). It has suspended the State assessment portion of the program and has also
opted to revise the State's academic standards. These academic standards, known as Essential
Skdls, are required by the legislation establishing ASAP. The legislation also requires ADE to
develop assessments to measure student progress in achieving the Essential Skills. The State
Board must begin adopting and implementing these tests during the 1996- 97 school year. When
completed, assessments will be administered in at least four grades selected by the Board.
As the ASAP is being revised, ADE should recommend additional uses for the program to the
State Board of Education to make the program more valuable. Currently, districts use the
program to guide curriculum and measure individual pupil progress. Recent legislation
requires the Board to adopt reading, writing, and mathematics tests and establish passing
scores for graduation, as is done in other states. In addition, some states use their assessment
programs to ensure students gain necessary skills; to increase school accountability; and to
evaluate educational programs. Specifically, some states mandate remedial instruction for
students who are not meeting proficiency standards. Other states use test scores to reward high-performing
schools or to sanction schools with below- standard test scores. Finally, several states
use their exams to help document curriculum strengths and weaknesses.
In addition to considering uses for ASAP, key decisions need to be made in order to enable
the Department of Education and school districts to proceed with test development and
iii
curriculum revisions. First, the Board needs to approve the revised academic standards once
they are completed. Second, the Board must determine at least four grades to be tested, as
required by law. The Board then needs to determine whether it will develop one or more
statewide assessments to monitor progress in each of the selected grades. Finally, the Board
needs to decide whether it will establish passing scores on state assessments in addition to the
graduation exam.
Table of Contents
Page
Introduction and Background ................................... 1
Finding I: Financial Management
Improvements Needed ................................. 7
School Districts Reimbursed
More than Actual
TransportationExpenses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
More Review Needed
ofADMCoun .................................................... 10
ADE Working to Address
Problems Found with Grants
Administration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Finding 11: Arizona School Report
Cards Can Be More Useful .............................. 15
Arizona Is Implementing
School Report Cards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Schools' Student Achievement
Scores Are Not Easily Comparable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Additional Information Could
ImproveUsefulness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Table of Contents ( con't)
Paae
Finding Ill: Additional Uses for
ASAP Should Be Considered ............................ 21
Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
ASAP Is Being Revised . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Additional ASAP Uses
Could Prove Beneficial . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
Key ASAP
Decisions Needed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
Recommendation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
Other Pertinent Information ............................... 27
State Responsibility for
Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
Agency Response
Appendix ............................................... a- i
Figures
Figure 1
Figure 2
Organization of the
Arizona Department of Education
April1996 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Monies Received by the
Department of Education
Fiscal Year 1995 ( unaudited) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
INTRODUCTION AND BACKGROUND
The Office of the Auditor General has conducted a performance audit of selected responsibhties
and programs of the Arizona Department of Education ( ADE). This audit was conducted
pursuant to the provisions of A. R. S. 541- 2958 and in response to a May 30,1995, resolution of
the Joint Legislative Audit Committee. Ths is the thrd in a series of four audit reports
regarding public education.
ADE Executes Board
Policies and Provides
Services to School Districts
ADE operates under the direction of the Superintendent of Public Instruction to execute the
policies set by the State Board of Education.' ADE also provides services to 233 school districts,
1,113 schools, and an additional 46 charter schools. In addition to providing services, the
Department is responsible for distributing funds to school districts. Basic state aid for
kindergarten through 12th- grade education ( K- 12) makes up the single largest component of
the State General Fund appropriations. The Department is responsible for administering and
distributing this state aid, as well as other state and federal education monies. In addition, the
State Board of Education and the Department are responsible for programs to improve school
performance and to keep parents and the public informed. Tlus audit focuses on these
responsibilities and addresses the following specific functions:
Financial Management- ADE is responsible for administering and distributing state and
federal education monies. The majority of the State General Fund K- 12 education monies
are allocated to school districts through a complex statutory funding formula, and state and
federal grant monies are disbursed to school districts through an application process. This
report addresses the need to improve the way school districts are funded for transportation
expenses; the need for audits of the student average daily membership ( ADM) count
component of the funding formula; and the Departments efforts to improve administration
of grant monies.
' Two other entities outside the scope of this audit also exercise state- level oversight. The State Board for
Vocational and Technological Education exercises general supervision over vocational and technological
education. The State Board for Charter Schools grants charter status to qualifying applicants, exercises
general supervision over charter schools it sponsors, and recommends legislation pertaining to charter
schools.
School Report Card Program- ADE has taken responsibility for developing, producing,
and distributing school report cards. School report cards provide parents and the public
with information about individual schools, such as school academic goals, programs, and
a summary of student achevement scores, in order for them to make informed school
choices? The requirement for the school report card program became law in 1994, and ADE
printed the first school report cards completed under the program in April 1996 for
distribution by schools. This audit addresses how certain data is not comparable between
school report cards, and the benefits of report cards providing additional information.
Arizona Student Achievement Program ( ASAP)- The State Board of Education is
statutorily responsible for developing a student achievement program and must approve
any proposed changes or revisions. ADE participates in the implementation of the student
achievement program. Currently, state education officials are in the process of revising
ASAP components in order to improve the program. Ths audit addresses additional uses
for the ASAP, as well as key decisions that need to be made to enable ADE and school
districts to implement ASAP revisions.
Significant Organizational
and Staffing Changes
Occurred at ADE
A new Superintendent of Public Instruction assumed office in January 1995. Since that time,
ADE has undergone four major structural reorganizations. Additionally, the Department
reports that, as of Apd 1996, it is operating with 355 of the 448 FTEs authorized for fiscal year
1996. The Superintendent's new organizational structure has four goals. These goals are
focusing on the Department's academic mission and the effect of ADE's efforts on student
learning; reorganizing divisions to better reflect their missions and responsibilities; providing
organizational stability; and using the strengths of ADE's professional staff.
The Department is organized into seven sections, as shown in Figure 1 on page 3.
1 School report cards are different from pupil report cards. Pupil report cards provide parents with
information about an individual student's progress. School report cards provide parents and the public
I
with information about a school's programs and progress. I
Figure 1
I Organization of the
Arizona Department of Education
April 1996
I Administration ( 7 FTEs) I
Administers all ADE activities. Includes Superintendent of Public Instruc-rintendent,
State Board of Education Administrator, and
Serves as policy advisor to the Superintendent of Public Instruction on all
school fmance issues, ( 2) works with the State Legslature, and ( 3) sets bud-get
policy.
Legislative Policy and Federal Relations ( 3 FTEs) I
In consultation with the Superintendent of Public Instruction, provides rec-ommendations
to, and responds to, requests concerning legislation from the
Governor's Office and the State Legislature.
Constituency Services ( 3 FTEs) I
Maintains agency communication and working partnerships with agencies,
groups, and organizations that impact K- 12 education.
I I
Communications ( 3 FTEs)
Works with the media and other communications groups to raise awareness
and share information on important issues concerning the agency, the
Superintendent's Office, and public education.
Oversees all programs administered by ADE such as School to Work, Aca-demic
Support, Student Services, Student Achievement and Assessment, Pro-fessional
Development, Adult Education and GED Testing, and Charter Schools.
Provides support to the entire Department. Includes Human Resources, Ad-ministrative
Services, School Finance, Budget and Strategic Planning, Financial
Services, and Management Information Systems.
Source: Auditor General staff presentation of data provided by ADE's Human Resources
unit. Data represents ADE FTEs as of April 1996.
ADE Administers and Distributes
Federal and State Education
Monies to School Districts
ADE is responsible for administering and distributing state and federal education monies to
school districts. The Department received approximately $ 2.1 billion in fiscal year 1995. Of that
total, the Department distributed approximately $ 2 billion to educational institutions and
spent approximately $ 23 million carrying out its duties.' See Figure 2, below, for the sources
and their contributions to the total monies the Department received in fiscal year 1995.
Figure 2
Monies Received by the
Department of Education
Fiscal Year 1995
( unaudited)
State Appropriated Monies -
Other Monies
$ 67,595,623 ( 3%) 4
Federal Grants
Source: Auditor General staff analysis of data obtained from Uniform Statewide Accounting System reports
for fiscal vear 1995.
1' Of the remaining balance, approximately $ 23.7 million was reverted to the State General Fund.
4
Scope and Methodology
Tlus audit focused on the Department of Education's responsibilities, including administering
and distributing educational monies and ensuring the overall quality of education. Specific
functions addressed include financial management, the school report card program, and the
Arizona Student Achevement Program.
Tlus audit used a variety of methods to study the functions addressed in this report. Pertinent
information analyzed includes relevant statutes; current reports and studies prepared by the
Arizona Department of Education; information provided by other states; governmental studies;
the former ADE Audit Units ADM audit reports; and private CPA firms' audit reports of school
districts.
Key people interviewed include members of the State Board of Education; the Superintendent
of Public Instruction; Anzona Department of Education staff; Arizona school district officials
and principals; education interest group representatives; nationally recognized experts in the
field of education; education officials in other states; and private CPA firms currently auditing
Arizona school districts.
Ths report presents findings and recommendations in three areas:
1. The need for improvements in the way school districts are funded for transportation
expenses; audits of the ADM count component of the funding formula to ensure proper
distribution of state funds to school districts; and continued efforts to improve grants
administration.
2. The need to increase school report cards' contents to improve their usefulness.
3. The need to consider additional uses for the ASAP and make key decisions to continue
ASAP implementation.
Additionally, other pertinent information presented in ths report addresses the roles and
structures of state- governing entities in education and funding of schools on a real- time basis
( i. e; providing monies either monthly or quarterly per student, at the school the student is
currently attending.)
This audit was conducted in accordance with government auditing standards.
The Auditor General and staff express appreciation to the President and members of the State
Board of Education, the Superintendent of Public Instruction, Arizona Department of Education
staff, and state educators for their cooperation and assistance throughout the audit.
( This Page Intentionally Left Blank)
FINDING I
FINANCIAL MANAGEMENT
IMPROVEMENTS NEEDED
The Arizona Department of Education ( ADE) needs to improve its oversight and management
of monies distributed to school districts. Specifically, problems with the State's transportation
funding formula allowed school districts, in fiscal year 1996, to be reimbursed approximately
$ 19 million more than their actual pupil transportation expenses. In addition, ADE has not
sufficiently monitored school districts' average daily membership ( ADM) counts, the primary
factor used to determine state funding for districts. Moreover, ADE needs to continue
addressing problems with the administration of grant monies to school districts.
ADE administers and distributes state and federal funding for preschool through 12th- grade
education in Arizona. ADE distributed approximately $ 2 billion to educational institutions in
fiscal year 1995. In addition to distributing education monies, ADE is responsible for helping
ensure that those funds are expended appropriately. Statutes require ADE to monitor and assist
districts with financial record compliance.
School Districts Reimbursed
More than Actual
Transportation Expenses
ADE needs to rectrfy the problem of school districts being reimbursed significantly more than
their actual pupil transportation expenses. Statewide, in fiscal year 1996, school districts were
reimbursed approximately $ 19 million more than actual transportation expenses. One reason
ths overreimbursement occurs is that the State's transportation funding formula is flawed, and
can inappropriately reward inefficient school district transportation operations. Another reason
is that ADE does not adequately venfy transportation expense information submitted by school
districts.
ADE is responsible for reimbursing school districts for pupil transportation expenses. Statutes
provide a formula that outlines the type of expenses to be reimbursed and also the rate of
reimbursement For example, school districts are reimbursed expenses for both district- run and
private contractor- provided pupil transportation. The rate of reimbursement is either $ 1.59 per
mile or $ 1.95 per mile, depending upon the daily average route miles per student.' Beginning
The State's transportation formula reimburses districts for transportation expenses by multiplying the
annual bus route miles driven by either $ 1.59 per mile or $ 1.95 per mile. If the daily average route miles
per student is between one- half and one mile, districts are reimbursed $ 1.59 per mile. If the daily average
route miles per student is less than one- half mile or more than one mile, districts are reimbursed $ 1.95 per
mile.
in school year 1994- 1995, actual school district expenses for public transportation are also
reimbursed. In some metropolitan areas, needy students may ride city buses, using passes and
tokens paid for by the district
State reirrzbursanent exceeded actual tmnspo~ Zatione xpenses- In fiscal year 1996, school
districts statewide were paid approximately $ 19 million more than actual transportation
expenses. Two hundred five quahfying school districts were paid $ 121 million, whereas actual
expenses for those districts totaled only $ 102 million. One hundred forty- five of the 205 districts
received payments greater than actual expenses, totaling $ 19 million. One district, for example,
was provided nearly $ 4.5 million more than actual expenses; another district received $ 1.9
million more than actual expenses.
Transportation funding fonnula should be revised - One reason many school districts are
reimbursed more than actual transportation expenses is the State's flawed transportation
funding formula. Arizona's current transportation formula can reward inefficient operations
in some school districts, and can penalize efficient districts. In fiscal year 1996, of the 205
districts reimbursed for transportation expenses, 145 were paid more than actual expenses and
60 were reimbursed less than expenses. Other states' practices can provide ADE guidance in
determining what changes need to be made.
The State's transportation reimbursement formula is flawed for four reasons:
First, the formula can encourage inefficiency because it reimburses districts for total annual
route miles driven. Therefore, the more miles a district records as being driven, the more
money it receives. As a result, districts have no incentive to identify and use the most
efficient traffic routes.
Second, the formula reimburses districts for all daily route mileage, even those route miles
driven without students aboard. An efficiently designed bus routing system attempts to
minimize these miles because it is an extra expense that is not accomplishing the primary
transportation mission of getting students to and from school.
Third, there are problems with the mileage reimbursement rate. The mileage reimbursement
rate is either $ 1.59 per mile or $ 1.95 per mile, depending on the average daily route miles
per student. School districts that minimize the number of miles driven without students
on board may be penalized by receiving the lower reimbursement rate.
Fourth, as discussed on page 9, districts can be reimbursed twice for students transported
using public buses. Currently, districts can be reimbursed for bus fares and reimbursed
for the miles students ride on a public transit system.
This audit identified general guidelines and specific factors that should be used in developing
an equitable transportation funding formula. In determining a new formula, ADE should
decide what transportation activities wdl be encouraged, and then recommend the Legislature
fund these activities at a level that will keep education programs from being negatively affected.
Some guidelines used in other states and recommended by our transportation consultant
include:'
Not funding bus miles driven without students aboard. Without state support for these
nonproductive miles, districts will be motivated to minimize such mileage.
Reconfiguring the mileage reimbursement component to prevent efficient districts from
being penalized and inefficient districts from being rewarded. Another state's formula
awards an annual fixed amount of reimbursement per student that varies depending upon
how far the student lives from school.
Some specific factors that should be considered in a new formula include costs for crossing
guards, and costs associated with transporting students in disabled, desegregation, and gifted
programs. Additionally, other factors not controlled by the district should be considered
including the number of students, district size, student population density, and comparable
costs in comparable districts.
Double pay19zent for public transportation expenses slzould be eli~~ ziruzted- TheL egislature
should consider revising the transportation funding formula to prevent school districts from
being paid twice for the costs of transporting students using public buses. A 1993 statutory
revision to the transportation formula appears to have created a mechanism that allows districts
to be reimbursed twice for transporting these students. As a result, in fiscal year 1996, one
district received approximately $ 3.6 million more than it cost to transport students using public
buses.
Before the 1993 legislation, school districts were paid for students who used public
transportation by including miles students traveled on city buses in the districts' total annual
mileage. However, the current funding formula allows districts to receive payments based on
the highest annual total mileage of the previous three fiscal years, and does not specifically
prohibit districts from receiving payments for both the actual costs of public transportation and
the public transportation miles included in the total annual mileage. Ths creates a double
payment in some cases, and the same double payments will again be made in 1997, unless the
statute is changed. Therefore, the Legislature should consider revising the transportation
funding formula as soon as possible.
I
I
I
I
I
I
I
I
I
I
I
I
I
I
I
I
I
I The Office of the Auditor General retained a consultant to evaluate transportation operations in three
school districts. The consultant also reviewed the State's school transportation reimbursement formula and
ADE's role in verifying school district transportation expenses.
I 9
ADE not vmpjing transportation expenses- In addition to problems with the State's
transportation funding formula, ADE does not vedy district- reported transportation expenses.
Because statutes outline specific transportation oversight responsibilities for the State's
Superintendent of Public Instruction, ADE should audit school district transportation expenses
to ensure that expenses are valid.
A. R. S. 515- 921 requires the Superintendent of Public Instruction to 1) supervise the provision
of pupil transportation services; 2) evaluate transportation programs and routes for the
purposes of determining accurate transportation support levels for each fiscal year; 3) determine
the daily route mileage for each district to determine support levels; and 4) assemble the
information provided by each school district and determine its accuracy.
The Department is not adequately meeting its statutory obligation of determining the accuracy
of transportation information submitted by districts. For district- operated transportation, ADE
requires districts to report annual mileage from bus odometer readings. ADE, however, does
not verify the accuracy of that mileage. For districts that contract for transportation services
or use public transportation, ADE does not perform any review or verification of mileage or
other information districts report.'
Because statutes specifically require the Superintendent to determine the accuracy of school
district transportation information, audits of that information should be performed. Other states
audit their districts' transportation information. For example, Kansas' Department of Education
expects to recover at least $ 395,000 in fiscal year 1996 from audits of 204 of its 304 school
districts' transportation information. Also, beginning with fiscal year 1996, New Mexico's
Department of Education is performing audits of district transportation mformation. ADE could
perform transportation audits with its own staff or contract with certified public accountant
( CPA) h sto conduct the audit work. The Department should determine whch of these two
options would be most appropriate for verifying district- reported transportation information.
More Review Needed
of ADM Counts
Although state aid to school districts is the largest component of the State's General Fund
appropriations, ADE performs little or no review of critical school district information that
determines the amount of funding a district receives. ADE discontinued average daily
membershp ( ADM) audits in 1995, even though student count accuracy problems exist in
Arizona and other state^.^ ADE needs to reinstate an ADM audit function.
1 Private contracting of pupil transportation accounted for at least $ 9 million of transportation costs the State
reimbursed in fiscal year 1996. Districts were also reimbursed over $ 1.1 million for public transportation.
2 ADM is the average enrollment of part- time and full- time students for the first 100 days in session.
I
I
I
I
I
I
I
I
I
I
I
I
1
I
1 I ADE conducted these 44 ADM audits between 1982 and 1994.
2 CPA firms perform the financial audits for those districts receiving $ 100,000 or more in federal monies. The
Office of the Auditor General's Accounting Services Division then reviews the CPA firms' audit results. I Additionally, the Office of the Auditor General's Financial Audit Division reviews internal and other
controls of districts receiving less than $ 100,000 in federal monies.
ADM counts are key factors in determining the amount of state monies school districts receive.
A district's ADM count is the primary factor for determining its funding based on a complex
statutory formula. Variations in ADM accuracy can have a significant impact on the State's
overall budget and on state aid distributions to school districts. The Legislature, in fiscal year
1995, appropriated more than $ 1.4 bdhon for basic state aid, based primarily on ADM counts.
ADM problems identzfied- Despite problems with ADM counts, ADE discontinued its ADM
audit function in 1995. Prior to that time, one responsibihty of the ADE internal audit unit was
to determine whether selected school districts' ADM counts were accurate. To do this, ADE
auditors reviewed the amount of teaching time provided to students, the mathematical accuracy
of ADM reports, district classification of student absences and withdrawals, student residency
classifications, and vocational student count accuracy, among other things. Some of these audits
found sigruficant ADM problems. For example, a 1991 ADM audit report identified a district
that received a state aid overpayment of $ 466,622 due to ADM and other errors. Similarly,
another 1991 report found that a school district potentially received a state aid overpayment
of $ 165,100. The Departments ADM audits also identhed a district that was underpaid $ 23,301
in state aid due to ADM errors. Of the 44 ADM audits reviewed, all but 3 identified problems,
of whch 8 indicated state aid overpayments totaling $ 937,749.'
Although audits of school districts performed by private CPA firms are not intended to ensure
ADM accuracy, some of their audits indicate that ADM problems continue. CPA firms audit
school districts' financial statements and assess compliance with the Uniforwi System of Financial
Records ( USFR).~ T hree of five CPA firms interviewed stated that their audits found ADM
problems even though they perform lunited test work on ADM. One certified public accountant
estimated that 50 to 75 percent of districts have ADM problems.
ADM audits needed- The Arizona Department of Education needs to expand its financial
oversight by reinstating ADM audits. Education officials and other states' practices support
the importance of ADM audits. Eight of 12 western states surveyed conduct audits of student
count data.
Other states have recovered state monies as a result of student count audits. For example,
Colorado recovered $ 1.7 milhon in fiscal year 1995 through similar audits. In addition, a Texas
official estimated that Texas has recovered as much as $ 3 million annually from student count
audits. Kansas expects to recover approximately $ 3.7 million in fiscal year 1996 from student
count audits of 204 of its 304 school districts. Additionally, numerous Arizona state and
education officials, legislative staff, and accounting professionals support ADM audits.
The Department of Education should ensure the accuracy of ADM counts by using either of
two mechanisms:
Audit Function within ADE- ADE could reinstate an internal ADM audit function. Five
of the eight states surveyed that audit student counts have auditors within their
Departments of Education perform these reviews. ADE reports that it could reinstate an
internal ADM audit function at an annual cost of $ 175,000. Ths would cover operating
expenses and five FTE positions.
Expanded CPA audit work- ADE could require that the accuracy of ADM counts be
addressed as part of the districts' financial audits. By working with the Office of the Auditor
General, ADE could ensure that the CPA firms auditing the districts comprehensively
review ADM counts. Additional audit work on ADM counts in the USFR compliance
questionnaire used by the CPA firms could be required by ADE and the Office of the
Auditor General. Two of the eight states surveyed that audit student counts have these
reviews performed by CPA firms as part of annual financial audits.' Department of
Education officials from both states feel that the student count audit work done by the CPA
firms is sufficient to ensure accuracy.
ADE Working to Address
Problems Found with
Grants Administration
Another component of ADE's financial management duties is the administration of state and
federal grants to school districts. Both ADE and our review identified several problems with
grants administration. ADE has stated it will institute a grants management office to address
these problems.
ADEgmttt duties md resymzsibilities- A. R. S. 5515- 204,15- 206, and 15- 207 authorize ADE to
receive and disburse state and federal educational grant monies. School districts use these
monies to deliver various services to Arizona's children. During fiscal year 1995, ADE
administered grants totaling approximately $ 344 million.
The Department of Education is responsible for notifying districts that grant monies are
available. During tlus audit, from September of 1995 through April of 1996, districts applied
to specific ADE units gven administrative charge over program areas for grant monies. These
program units, upon receipt of the application, review the request to ensure that it is complete
and that guidelines are being met If the award is approved, the program unit creates a
payment schedule and the ADE accounting unit begins disbursements to the district.
1 In the eighth state surveyed that audits student counts, the state auditor's office performs these audits, in
conjunction with the annual financial statement audits of the school districts.
Grant processing probhns identified- ADE recognizes that it has problems with the
administration of grant monies. ADE's Grants Review Team recently identified the following
deficiencies:
Application due dates have been poorly enforced;
Grant forms are complex and lack uniformity;
ADE staff have not responded to district inquiries in a timely manner;
Grant processing suffers from a lack of automation; and
Some program units have failed to process applications in a timely manner.
Further review of ADE's grant process and interviews with six school district officials supported
the Department's findings.
ADE's efiovts to address problenzs- ADE has organized a Grants Task Force that meets weekly
to implement the Grants Review Team's recommendations. The Department has also proposed
the establishment of a grants management office, which ADE officials hope will provide
improved grants administration. The office is scheduled to begin operations in July 1996.
A March 1996 status report on the grants management office indicates that the office's goals
and objectives address several of the deficiencies identified by the Grants Review Team and
in our review. However, three areas of concern were not addressed by the status report. A
follow- up discussion with an ADE official helped to clarify ADE's position regarding these
areas:
In order to provide faster award approvals, ADE plans to institute a " substantially
approved form that will provide written interim confirmation of an award while the
Department finishes final application processing. Further, ADE's Director of State and
Federal Programs stated that the new grants management office will have a goal of
processing grant applications within 30 days of receipt
ADE management stated that many delays in disbursing grant monies to districts result
from the Department not receiving grant monies from the U. S. Department of Education
in a timely manner. According to ADE's Director of State and Federal Programs, while ADE
cannot disburse monies that it has not received, recent legislation allows districts to draw
upon other monies until grant monies are received.
Finally, ADE's Director of State and Federal Programs acknowledged that the Department
currently lacks expertise in the administration of grants. However, he said that the new
grants management office will fill this need and thus provide districts with better service
and access to reliable information. He also stated that ADE plans to provide additional
training for grants personnel.
RECOMMENDATIONS
1. The Arizona Department of Education should recommend statutory changes to the State's
transportation funding formula to ensure that state funding is distributed equitably and
to encourage efficiency.
2. The Arizona Department of Education should audit school district transportation expenses
to determine accuracy as required by A. R. S. 515- 921.
3. The Arizona Department of Education should increase its oversight and monitoring of
school district ADM counts. Specifically, ADE should reinstate an ADM audit function. The
Department could implement this recommendation by one of two mechanisms:
a. Forming an internal ADM audit group
b. Expanding CPA firms' audit work on ADM counts by working with the Office of the
Auditor General to require CPA firms to more comprehensively review ADM counts
as part of school district financial audits.
4. The Arizona Department of Education should continue to address problems found with
its administration of grants to school districts.
FINDING II
ARIZONA SCHOOL REPORT CARDS
CAN BE MORE USEFUL
Arizona should modify its school report cards to better inform the public about school
performance and to make the report cards more useful to parents. The State implemented a
school report card program to increase school accountability and to assist parents and students
in choosing schools. Certain data on the report cards, however, are difficult to compare among
schools. ADE could further improve the program by adding information in several categories
to better profile and explain school performance.
Arizona Is Implementing
School Report Cards
Report card programs keep parents and the public informed about schools. Arizona's school
report card program is similar to efforts in other states designed to build public interest and
encourage school improvements. The requirement for the school report card program became
law in 1994, and the Arizona Department of Education ( ADE) has recently printed the first
report cards completed under the program for distribution by schools to parents of enrolled
pupils.
Other states and national organizations indicate report cards can be a powerful tool for
improving school performance and accountability. Report cards are used to focus public
attention and to hold schools accountable for student progress. In addition, the report cards
are designed to help parents select public schools and to pressure schools that are performing
poorly to improve. ADE has taken the responsibdity for developing the format for and printing
the school report cards. A. R. S. 515- 746 requires the report cards to contain at least 13 pieces
of information about schools, including goals, responsibilities, special programs, and test scores.
Statutes also require ADE to modify the report cards as needed each year. A sample school
report card is included in the Appendix ( see pages a- i through a- iv). The report cards are also
accessible through the Internet so that parents can compare information from a number of
schools and choose a school that best meets their child's needs.
Schools' Student
Achievement Scores
Are Not Easily Comparable
Variations in how school districts assess students under the Arizona Student Acluevement
Program ( ASAP) make it difficult for parents to use the school report cards to compare schools'
performance. School report cards include averaged student test scores on district- level
assessments; however, this assessment information is not easily comparable among schools.
The State Board of Education could make the testing data more useful to parents by adopting
a consistent statewide exam and passing score for each grade tested.
Assessr~ zent program diffwmes can confuse report card readers- Assessment scores
summarized on the school report cards do not allow parents to easily compare how well schools
are educating students. School report cards include test scores from district assessments that
measure student progress toward the state Essential Skills. Districts, however, determine how
students will be assessed. For example, one district may develop its own exams to assess student
progress. Another district may assess students using a combination of state- developed tests
and portfolios of student work. Report card readers cannot easily compare student scores
because they do not know whether the assessments used by schools in different districts are
the same.
In addition, districts are allowed to establish the scores students must aclueve to demonstrate
proficiency on the assessments. This practice further limits report card comparability. For
example, ths audit compared two school report cards. School A required students to answer
50 percent of test questions correctly to demonstrate mastery, wlule School B required 80
percent correct responses to demonstrate mastery. School A's report card indicates 95 percent
of its students are mastering most skills, and School B's report card indicates 75 percent of its
students are mastering most skills. A parent comparing the two schools' report cards could
logcally conclude that School A is better educating students. However, since scores the students
must attain to demonstrate mastery are much lower for School A, parents cannot easily
determine which school is more successful.
Statewide aa~ tzcso uld 11zake report cards nzore cmnparable- The Department of Education
could improve the usefulness of reported assessment information by recommending the State
Board adopt a standardized statewide exam and passing score for each grade tested. As
discussed in Finding III ( see pages 21 through 25), the Board is currently revising the ASAP
program, and is in the process of determining how students should be assessed on a statewide
basis. A standardized assessment with a standard mastery level for each grade would enable
parents to compare how well students are progressing at various schools.
Additional Information
Could Improve Usefulness
The Department of Education could further improve the school report cards by increasing the
amount and types of information provided. States with similar programs provide more and
ddferent information to assist parents in comparing schools, to help track school progress over
time, and to increase school accountability.
High school report cards can provide specialized infmmatim- Other states expand the amount
of achievement information presented on high school report cards. For example, scores on
college placement ( SAT or ACT) exams and advanced placement ( AP) tests can help
demonstrate how well schools prepare students for college. Some states also report the number
of students taking a rigorous core of classes. Utah, for example, reports the number of students
whose course work consists malnly of math, sciences, social studies, and English. In addition,
a few states require schools to report the number of graduates employed or attending college.
l k s measure helps indicate how effective the school is at preparing students to continue their
educations or enter the work force.
Move statistical infmtuatim could better infomr parents- Report cards could better describe
school environments by including additional information. For example, ADE should consider
expanding the school safety data reported. It should also consider adding information such
as student mobility rates, student- teacher ratios, and dropout rates. Moreover, ADE should
consider reporting trends in student assessment scores and other measures when the data
becomes available.
School safety- Other states, including Utah, Florida, and California, recommend and use
a number of school safety measures that Arizona may want to consider adding to its school
report cards. Currently, Arizona schools report the number of incidents that involved law
enforcement intervention, but only if charges were filed. ADE may want to consider
requiring schools to report all on- campus incidents that impact school safety, as noted in
the report issued by the Office of the Auditor General on selected Arizona school districts
( Performance Audit Report, 96- 9). Other measures to consider include the number of
students suspended, expelled, or removed to supervised alternative education programs,
and costs of theft and vandalism. California also requires schools to assess safety on campus.
The assessment can include a description of the policies and procedures schools have
developed to ensure student safety.
Student mobility- Florida and New Jersey use a mobility rate to measure disruption in
teaching and learning. The measure identifies the percentage of students entering and
exiting the school during a single school year. Experts indicate the measure is relatively
new, but can help explain differences in test scores among schools.
Student- teacher ratios- This ratio is reported in other states as a measure of educational
quality. A single ratio, however, can sometimes mask the true number of overcrowded
classrooms, particularly in lugh schools. To remedy this problem, experts suggest reporting
student- teacher ratios by course type, or reporting the number of hours students spend in
overcrowded classrooms. If ADE decides to use this measure, the State or school districts
would need to determine the acceptable number of students per classroom.
Dropout rates- High school dropouts have limited employment opportunities;
consequently, education departments in other states have increased efforts to report and
reduce dropout rates. School report cards in other states indicate the percentage of students
in grades 9- 12 who dropped out during the school year. In addition to reporting dropout
rates, California recommends that schools describe the programs they use to promote
attendance and reduce the number of dropouts. If the Department chooses to include this
information on the report cards, it should first address the problems concerning dropout
statistics noted in the report issued by the Office of the Auditor General on ADE dropout
prevention efforts ( Performance Audit Report, 96- 6) to ensure the accuracy of reported data.
Several states include two to five years of data to help identify trends in school performance.
Reports often include district, state, and national averages to make comparison more
meaningful. The Department should consider adding performance trends and other
comparison data to its school report cards when the information becomes available.
Der~ zograplzipc rofiles can make cmtzyarisons more fair- Educational experts generally agree
that demograpluc variables influence student test scores. To help adjust for these factors, other
states include such information on their school report cards as the ethnic distribution of
students; the percent of students receiving free or reduced lunches; and the number of students
with limited English proficiency. These factors vary widely among Arizona schools, and ths
information could be used to evaluate school effectiveness. In other states, ths demographic
information is sometimes used to group schools with similar characteristics for comparison
purposes. Another method for evaluating school progress involves predicting a school's
performance based on demographic factors and comparing predicted performance to actual
results.
Distuictfismsrcial i~ lfon~ zatiocsat n i~ tmeasea ccoi~~ rtabiliiy- fAe w states also report a large
amount of fmancial information on their school report cards. These report cards list items such
as per- pupil expenditures and teacher and administrator salaries. Some states also provide a
listing of expenditures in categories such as instruction, administration, transportation, and
construction. The information can be used to identify districts that spend larger- than- average
portions of their budgets on administration or noninstructional items.
RECOMMENDATIONS
1. The Arizona Department of Education should recommend that the State Board of Education
adopt a consistent statewide examination and passing score for each grade tested to make
comparison of schools' performance more meaningful.
2. The Arizona Department of Education should consider modifying school report cards to
include additional information regarding school performance, student demographics, and
district financial data.
( This Page Intentionally Left Blank)
FINDING Ill
ADDITIONAL USES FOR ASAP
SHOULD BE CONSIDERED
The Arizona Student Achevement Program ( ASAP) could be expanded to further benefit
students, schools, and the State. The State Board of Education is currently modifying
components of the State's student assessment program. During this revisions process, the
Arizona Department of Education ( ADE) should recommend to the Board additional uses for
assessment results. Other key program issues also need to be resolved to enable the ADE and
school districts to implement the revisions and to ensure a quality assessment program.
Background
ASAP, created in 1990, is an effort to improve education by setting hgh academic standards
and requiring students to demonstrate competency. Basic ASAP components include:
A nationally standardized exam such as the Iowa Test of Basic Skds that allows comparison
of Arizona students' exam scores in such topics as reading, language, and math to students
in other states taking the same exam;
The Essential Skills, or academic standards, which outline nine categories of knowledge
and skill the State Board of Education expects students to gain in school. The categories
include language arts, literature, mathematics, science, social studies, music, visual arts,
health/ physical education, and foreign or Native American language;
District assessments of student progress toward the Essential Skills. Districts develop their
own assessments using state- developed tests, district- developed exams, portfolios of student
work, or a combination of tools;
A state assessment to monitor student progress statewide. The State Board of Education
adopted a series of state tests known collectively as Form D. These tests were developed
to measure the same skills as district assessments and to verify district reports of student
progress.
ASAP Is Being Revised
The State spent approximately $ 3 million developing the elements of the original ASAP.
However, a 1994 study performed by the company that developed Form D indicated that the
assessment did not adequately measure the same skills as the district tests; consequently, the
State Board of Education suspended the state tests in January 1995. Shortly after suspending
Form D, the State Board also opted to revise the Essential Skills to make them more
understandable, to put them in a consistent format, and to add new workplace skills. As of
May 15,1996, the State Board was still debating the changes, which will ultimately form the
basis of new state and district assessments, and school curriculum revisions.
Statutes require the State Board to begin adopting and implementing new Essential Skills
assessments during the 1996- 97 school year. In May 1996, the Board hosted a public hearing
to discuss the revised language arts and mathematics standards. The Board anticipated
approving these standards in June and expects to have language arts and math assessments
in place by 1997. Until then, the State lacks a consistent means of monitoring and tracking
student progress toward the Essential Skills.
Additional ASAP Uses
Could Prove Beneficial
As the ASAP is being revised, ADE should recommend to the State Board of Education some
additional uses to make the program more valuable. Currently, districts use the ASAP to guide
curriculum and to measure individual pupil progress. In addition, 1996 legislation requires
the Board to adopt tests and passing scores for students to graduate from hgh school. Other
states use their assessment programs as a graduation requirement for students and for a number
of other purposes.
Arizona graduates will need to da~ zonstratec or~ zpetermj- S tatute requires the Arizona State
Board of Education to adopt tests to determine whether students meet competency standards
for graduation. At least 16 other states also use assessment scores to help determine whether
students meet graduation requirements. To avoid possible litigation, these states incorporate
a number of factors into the design of their graduation assessment programs. The State Board
should consider these factors in developing Arizona's graduation assessment. For example,
other states ensure the tests are valid, reliable, and unbiased. Students also receive multiple
opportunities to pass the graduation exams. Students who do not pass the exams typically
receive additional instruction to correct deficiencies. Some states also offer alternative
graduation assessments to students not passing the initial test. Alternatives include different
test forms and a hands- on learning center where students can demonstrate skills. One state
issues certificates of attendance to students not mastering the skills necessary to earn a diploma.
Assess~ nent uses could be expanded fuvtlzm- ADE should recommend to the State Board of
Education additional uses for the assessments to make the program more valuable. Other states
use their assessment programs to assist students in gaining necessary skills, to increase school
accountability, and to evaluate educational programs. These uses can influence the types of
tests needed and how the tests are implemented. Some of these additional uses include:
Remediation- Several states specifically mandate remedial instruction for any student not
meeting proficiency standards on state tests. These states provide districts with additional
resources for mandated assistance, which can include extended school days, and specialized
instruction. Other states indicate their assessments help teachers identify students needing
additional help, but intervention is not mandated.
School rewards and sanctions - Other states also use test scores to reward high- performing
schools or to sanction schools with below- standard test scores. Kentucky and Texas use test
scores to provide rewards or incentive monies to schools. These states' reward programs
also consider factors such as dropout rates, attendance rates, and whether a school's
graduates are successful; i. e. working or attending college. While most states we contacted
do not offer financial incentives, some publicize successful schools' programs.
Some states have also developed a set of intervention steps to take in schools with below-standard
test scores. Intervention programs often start with offering districts expert
assistance and fmancial resources. If scores fail to improve, states can become increasingly
involved to the point of replacing school management Iowa and New Jersey, for instance,
have used their intervention programs to take over the management of school districts that
have continued to perform poorly. At least five other states have similar intervention
provisions. These provisions enable states to make high- level personnel changes, to withhold
state funding, and to take other action at the school and district level.
Curriculum evaluation- Several states, including New York, Ohio, Oregon, and Utah,
indicate state exams are used to help document strengths and weaknesses of curriculum.
For example, New York's assessment program includes specialized tests to help evaluate
the effectiveness of academic programs in science and social studies.
Key ASAP
Decisions Needed
In addition to considering additional uses for ASAP, other decisions must be made to enable
ADE and school districts to implement ASAP revisions. The Board is statutorily responsible
for implementing a student achievement program and must approve any proposed changes
or revisions.' Board members, however, are still debating issues regarding standards and
assessments. Once the Board makes decisions regarding ASAP, ADE and districts can move
forward with developing a state assessment and revising school curriculums.
b~ tpmfanits sues mlzain & n'ded- The State Board of Education is still considering several
ASAP issues involving the academic standards and state assessment The Board must complete
and approve the Essential Skdls revisions before a test can be developed, piloted, and submitted
A. R. S. § § 15- 203 and 15- 741 require the State Board of Education to adopt and implement academic
standards and achievement tests, and set competency requirements for promotion and graduation.
23
for public approval. While the Superintendent of Public Instruction has outlined her plans for
the program, the Board has not indicated its approval. Board members still need to make
decisions regarding the following issues:
1 Academic standards - Standards outline minimal student competencies and form the basis
of assessment programs. Once the standards are completed and approved by the State Board
of Education, a new state assessment instrument can be developed. The Board indicates
it will first concentrate on approving standards in core areas, such as language arts and
math, so that a basic assessment can be developed. Approval for other standards, such as
the arts and foreign languages, is expected to take longer.
Grades tested- Untd recently, the State was required to assess student progress in grades
3,8, and 12. However, Laws 1996, Chapter 284 requires the State Board to choose at least
four grades in which the state assessment is administered. These grades must be
determined before an assessment can be developed.
Statewide assessment- The State Board of Education needs to determine whether it will
develop one or more statewide assessments to monitor progress in each selected grade.
Under the current program, districts determine the methods used to assess students. The
choice enables districts to customize their assessment programs. However, the State's lack
of a standardized assessment limits its ability to obtain a consistent and clear picture of
student progress statewide.
Passing scores - Statute requires the State Board of Education to adopt competency tests
for graduation, and to establish passing scores for these tests. The Board also needs to
determine if it will establish a passing score for the state assessment( s) administered in other
grades. Establishing a statewide exam and passing score would enable the State Board and
ADE to better monitor student achievement and to report comparable achievement
information on school report cards, as noted in Finding 11 ( see pages 15 to 19).
ADE nsd district nctions hiszge osi Bonrd decisiosts- ADE and school districts must perform
several activities to implement the revised MAP, but cannot move the program forward until
the State Board of Education approves the academic standards. Once the Board has approved
the standards and determined the type of assessment to be developed, the Department can
identify the key focus areas of the assessment, and release a request for proposal, if needed,
detahg the Board's expectations for the exam. Any test produced will need to be piloted and
checked for validity, reliability, and bias to ensure it is a quality assessment instrument. The
Department will also need to determine the level of assistance it will provide to districts to
implement the revisions to the Essential Skdls. The goal is to have the new academic standards
correlated with the Essential Skdls to help districts iden* needed curriculum changes. Districts
will then need to align their curriculums with the new standards, and determine how they will
assess student progress at the school or district level.
RECOMMENDATION
1. The Anzona Department of Education should recommend to the State Board of Education
expanded uses for the ASAP to provide for additional school accountability and to provide
schools with incentives for improvement.
( This Page Intentionally Left Blank)
OTHER PERTINENT INFORMATION
During the course of the audit, other pertinent information was developed relating to the State's
responsibility for education and real- time funding.
for Education
Responsibility divided - Arizona statutes divide state
responsibility for education between the State Board of
Education, the Superintendent of Public Instruction, and the
Department of Education. Statutes direct the State Board of
Education to estabhh educational policy and the Superin-tendent
to execute the Board's policies. The State Board of Education administers the
Department of Education through policy creation. In addition, the Superintendent controls the
conduct of the Department of Education and manages the Department's executive and
administrative functions. The Department of Education provides service and assistance to
schools, districts, and county school superintendent offices.
State Board of Education- The Arizona Constitution establishes the nine- member State
Board of Education. The Governor appoints eight board members to staggered four- year
terms. The ninth board member is the Superintendent of Public Instruction. The
Constitution further provides that the State Board of Education, with other entities, is
responsible for the public school system's general conduct and supervision. Arizona statutes
give the State Board of Education many duties, including supervision and regulation of
public schools; administration and supervision of federal appropriation expenditures;
adoption and implementation of achievement tests; prescription of minimum course
requirements and student achevement standards; establishment of educational policy; and
certification of teachers.
Superintendent of Public Instruction- The Arizona Constitution establishes the
Superintendent of Public Instruction as a publicly elected official in the Executive
Department The Superintendent is a member of the State Board of Education and serves
as the Board's Secretary and Executive Officer. The Superintendent, with the State Board
of Education and other entities, is constitutionally responsible for the public school system's
general conduct and supervision. Arizona statutes require the Superintendent to execute
State Board of Education policies, direct the Department of Education, oversee state schools,
review district budgets, and apportion yearly funding to counties for school districts. The
Superintendent also prints and distributes courses of study as prescribed by the Board and
legal pamphlets as needed.
Department of Education- Arizona statutes direct the State Board of Education and the
Superintendent of Public Instruction to jointly administer the Department of Education.
Statutes also provide that the Superintendent will control the Department's conduct. The
Department has the following statutory duties: monitor and assist districts with financial
record compliance; conduct special education financial and program audits; develop and
distribute standardized school report cards; establish and maintain an environmental
education resource system; and provide technical assistance for chemical abuse prevention
programs, AIDS instruction, and supplemental academic programs for kindergarten through
third grade. Statutes also provide the Department with various school funding responsibili-ties
and many discretionary duties.
Cumt st& m could be problematic- National experts and other states' education officials
identified potential problems with an elected Superintendent of Public Instruction overseen
by a State Board of Education.' Additionally, interviews with Anzona State Board of Education
members, the Superintendent of Public Instruction, Department of Education employees, county
and district superintendents, and school principals suggested that similar problems may occur
in Arizona. The officials noted potential problems with long- term planning ability,
Superintendent accountability to the State Board of Education, and experience requirements
for Superintendent candidates.
Long- term planning- Kentucky previously had a structure similar to Arizona's. A
Kentucky official reported problems with that structure, including limited long- term
oversight and frequent program and personnel changes. Kentucky elected a new
Superintendent every four years and changed education programs after each election.
Similarly, Arizona officials identified potential problems with long- term education
planning, and program and personnel changes following a shift in leadershp. Officials
reported reductions and changes in staff and programs at the Department of Education
following the last election.
Superintendent accountability- California, like Arizona, has an elected Superintendent
of Public Instruction and an appointed State Board of Education. A California official
reported that Cahfornia's Board has no supervisory authority over the Superintendent and
cannot control the Superintendent's actions. Similarly, several Arizona State Board of
Education members indicated that the Board has no authority over the Superintendent and
no ability to require the Superintendent to carry out its policies. The only way Arizona's
Superintendent could be required to carry out the Board's policies would be through a
special action lawsuit Officials and experts further noted that the Superintendent, as an
elected official, is accountable to voters but not to the State Board of Education.
Education experience - National experts report that states with an elected Superintendent
do not usually require the Superintendent to have prior educalion experience. Georgia is
the only state of 15 states with an elected Superintendent that statutorily requires the
Superintendent to have education experience. Of the 35 states with an appointed
Superintendent, 19 statutorily require the Superintendent to have education experience.
1 Different states refer to the Chief State School Officer as either Superintendent or Commissioner. The term
" Superintendent" will be used for all states throughout this part of the report.
In addition, some Arizona officials expressed concerns regarding the lack of an education
experience requirement for Arizona's Superintendent.
According to a national expert, a state with an elected Superintendent is likely to experience
problems. Goodwill between the Superintendent and the State Board of Education will
encourage cooperation and effective functioning. California has an elected Superintendent and
serves as an example of what can happen when goodwill and cooperation do not exist. Several
years ago, California experienced problems when the Superintendent developed his own
agenda and refused to cooperate with the State Board of Education. The Board, with no
supervisory authority, could not control the Superintendent's actions. Ultimately, the California
State Board of Education sued the Superintendent to force him to cooperate.
In contrast to the potential problems that have been identified, Arizona's Superintendent of
Public Instruction identified potential benefits of an elected Superintendent or State Board of
Education. First, an elected Superintendent or State Board of Education provides a voice for
the public in education. Since education receives a large portion of the state budget, it is
important that public opinion is represented. Second, an elected official, whether the
Superintendent or a Board member, is directly answerable to the public for carrying out its
desires.
0 t hs tates ~~ zovinawg ay fvorn Arizona's structure- Nationally, states are changing their
structures to a board of education- or governor- appointed Superintendent of Public Instruction.
National experts report that, since 1900, the most common Superintendent selection method
has changed from election to appointment. In 1900, 69 percent of states elected the
Superintendent while only 22 percent of states appointed the Superintendent. The remaining
9 percent used another selection method. By 1992, this trend had reversed; states with an elected
Superintendent fell to only 30 percent, whde states with an appointed Superintendent increased
to 70 percent. In the past 20 years, 6 states have changed their structures from an elected
Superintendent to an appointed Superintendent. During the same period, no states have
changed their structure from an appointed Superintendent to an elected Superintendent.
Currently, 35 states appoint Superintendents to terms of 3 to 5 years. According to a national
expert, the typical pattern is for the Governor to appoint board members to staggered terms,
and the board to appoint the superintendent. The board members' staggered terms provide
continuity in educational planning.
Real- time funding plan- The Superintendent of Public
Instruction has proposed a " Plan for Education in Arizona"
which calls for, among other ideas, the installation of what the
Department of Education terms real- time funding. Currently,
district funding levels are based on prior- year student counts. Under the real- time funding
proposal, education monies would follow each student to the school he or she attends and
would be allocated on a monthly or quarterly basis.
Since real- time funding proposes hking a certain amount of education dollars to each student,
ADE management regards this plan as an appropriate mechanism for bringing accountability
to schools. In addition, ADE management state that schools will not have to turn students away
due to lack of monies since transferring students will come with the monies necessary to
support their academic needs. Finally, because real- time funding will require tracking and
reporting student level mfomation, the Department views this proposal as an opportunity to
better evaluate student achievement and provide for greater school accountability.
C a m s raised about real- time funding- In order to clarlfy issues and concerns relating to
real- time funding, interviews with education interest groups and with ADE management were
conducted. Following are concerns relating to real- time funding that emerged from those
discussions and ADE's responses:
Budgeting Difficulties- Education association representatives voiced concerns that since
under the real- time funding proposal education monies would be tied to the student, school
funding could fluctuate up and down with the student population. Schools could therefore
find it difficult to plan and to budget.
ADE's Finance Policy Director stated that schools and districts may need to use budgeting
and forecasting techniques similar to those used in the private sector in addition to any
trends that could be gleaned from their prior years of operations. Also, she said that some
fixed cost amount may need to be set aside and provided separately to cover basic
operations.
Complicated Administration- Concerns were voiced regarding the complexity of real-time
funding administration. Specifically, one concern is that the move toward administer-ing
monies at the student level will result in ADE having to manage the accounts of more
than 700,000 individual students, instead of the approximately 233 districts for whom the
Department currently administers monies.
ADE's Director of Special Projects said that the Department has recognized the need for
an automated system to make real- time funding possible. ADE proposes linking schools
to the Department odme to fachtate the tracking, reporting, and administration of student
level mfomation. The Department has initiated a pilot project in Pinal County to test the
feasibibty of transmitting data online. Florida's Department of Education has been using
an online system since 1991. Its 67 school districts transmit student level data to the
Department twice yearly for funding purposes. In contrast, interviews with education
representatives in Minnesota and Delaware revealed that both states use magnetic tapes
and diskettes to transmit student level data rather than relying exclusively on online
systems. Delaware and Minnesota report student level information three times yearly and
twice yearly, respectively.
Abandoned Obligations- Under real- time funding, growing schools would receive
increased funding while schools with declining enrollments would receive less money.
Interviews revealed concerns that these cuts could impact a school's ability to meet fixed
costs and other financial obligations.
ADE's Finance Policy Director acknowledged this issue as one that would need to be
addressed if the State were to move toward real- time funding. She indicated that a reserve
of monies or some baseline funding may need to be established to meet schools' fixed
financial obligations.
Accountability Not Assured- Critics of real- time funding stated that there is no proof that
it will promote school accountability and excellence. One critic of real- time funding said
that children generally do not change schools simply because their parents are dissatisfied.
Another critic stated that schools may actually welcome the departure of some students
if the school cannot adequately meet the needs of their student populations.
The Department acknowledged that it does not know how or if student mobility will
change in reaction to real- time funding. However, ADE management cited the growth of
charter schools as an indication that parents welcome expanded school choices. ADE's
Finance Policy Director stated that discussions with parents have served to further convince
her that parents want and need the option of changmg schools if they believe their chddren's
needs are not being met.
Depavtment planning statewide disc~ rssion to address coszcmzs- ADE management
acknowledged that many of the concerns brought forward will have to be addressed before
real- time funding is implemented in Arizona. ADE management stated that the Superinten-dents
real- time proposal was intended to be a starting point that would provide the foundation
for further discussion. As a part of this, the Department is in the process of scheduhg meetings
with a variety of interested parties to address, among other issues, real- time funding logistics
and concerns.
( This Page Intentionally Left Blank)
AGENCY RESPONSE
( This Page Intentionally Left Blank)
Response to Finding I
While it is me that the Deprtrtment of Education does not verify transportation expenses, it does verify the
transporntion support level as defined by law. The h a 1 route report is compared to the prior year and
discrepancies foluld are questioned. Adjustments are made to route miles when they are fvmd to be in
error.
The Dcpament does not disagree with the principle of reimbursement for expenses. However, the
Department is in full compliance with the transportation support 1eveI formula. l'be law does not require
school districts to expend these Monies on transportation nor wac it the intent of the law to fully reimburse
districts for these costs.
In fiscal year 96, thc first year of dam collection for tokens and passes, the Agency divided the eligible
students into the cost of tokens and passes to determine whether rhe costs were reasonable. No costs wen?
found to be unasvnable.
MCSPONSE TO FINDING 11
ARIZONA SCHOOL REPORT CARDS
CAN BE MORE USEFUL
To preface the points ADE takes issue with, it should be stared that many of the report's comments are
insightful and relate to modifications already slated for inclusion in next year's report cards, such as
expailsion of the school safety section, and inclusion of studcnt mobility and schvvl staffing data. Also, it
should be noted that all the information required under ARS $ 15- 746 is contained in the school report card.
To understand the following comments, thc assumption that tic rcpon cards cxist u~ dinlyto provide the
public a way to COMPARE schools to Jetennine which is school is & shouId be chaIienged. Currently
the ADE does riot rate, or rank, schools. There are a myriad of factors othcr than tests scores which are
available to determine whether a particular school is well- suited for a particular child. Report a dsc ctions
on dcademic goals, insmctional programs, and parenttschool responsibilities providc more profound
insight into the philosophy and character of a school than test scores, student- teacher ratios, and mobility
rates.
Information on Student Achievement of the Essential Skills
In rhe absence of a statewide ASAP assessment of the stare- Board- determined academic content standards,
data on student achievement of the Essential Skills, as reported by schools, is Ihe only available measure of
whether such standards isare being met. To not rcpon mastery information on student achievement of the
current standards would completely neglect the notion of accrdcnlic accountability and would deny students
and parcnts information on tl~ ev ery academic standards thar one day may be used to deny students their
diplomas.
Though determination of student mastery of the essential skills does vary by school. and comparison
benvcen schools is a dubious endeavor, these data are meaningful within a school, and often betwccn
schools in a district. Information on district- determined mastery levels, or cut scores, for s ~ d e n t
assessment are provided in the school rcport card bccausc dislrica arc responsible for determining
curriculum. instruction and local assessment of the Essential Skills. This responsibility requires districts to
ensure that their students have thc opportunity to learn the processes and acquire the knowledge necessary
to achieve state content and performance standwds. 111 the absence of a cute performance standard,
districts must rtly on their professional judgment to determine mastery levels, in the same way tcachers
determine a passing or non- passing grade for a given class. Once an equitable system of standards ha<
been set, then the next level of accountability may be rtchicved. Until thcn, it is ADE's position that this
inrornlation should not be abandoned. The ADE plans to clarify the description of this information, to help
guide parents away from erroneous conclusions about scllool comparison based on these data
Inclusion of Additional Information
ADE plans to put student achievement data in some context by h e inclusion of a mobility fdctor, and the
possible inclusion of the percent of students with limited English proficiency and a poverty measure such
as the perccnr of studens eligible for free meals ( if available). Racelethnicity data will not be included
since a student's race or ethnjcity has no inherent relationship to academic performance. Indeed, it is
against professional standards to test wilh an instrument that may be selectivt. ly biased against a particular
group.
The use of non- mandated tests such t b college entrance exams like thc SAT or ACT is not a good general
indicator of a school's academic achievement level, since the test is voluntary and is not required for
entrance into Arizona uriiversities if the required coursrwork is achieved and the student is in top 50
percent of the graduating ciaus. Thc population of students taking rhese tests would not be a valid indicator
of studcnr preparation for college.
Currently, fmalce data at the school level are not readily available because scl~ oodl istric~ c ontrol finding
and determine budgets as well as personnel salaries. Since schools often have little say in these decisions,
such district- levcl data would nut bc indicative of school philosophy or effectiveness.
( This Page Intentionally Left Blank)
APPENDIX
I
I School R e p o r t C a r d 1995 - 1 9 9 6
- Anasazi Elementary School
I Scottsdale Unified District
11 130 E. Cholla, Scottsdale, Arizona 85259
Principal: Mr. Jeff Quisberg ~ rabes: Kt06
Office Hours: 7: 00 a. m. to 4: 00 p. m. Enrollment: 892
Phone: I , ( 602) 451 - 5020
FAX: ( 602) 860- 4753
1
Anasazi Elementary School is a special place 10 learn, grow and play. In 1995, Anasazi was selected as an A+
School. This honor exemplifies the outstanding efforts of the entire Anasazi community. This year's theme is " One
By One We Make A Difference." Recognizing quality citizenship, requiring diligent study habits, fostering respect
and establishing sound leadership skills are all part of the mission and philosophy of Anasazi.
School Organization and Philosophy
Self- contained Classrooms * Multi- age Classrooms
I Integrated Instruction * Cooperative Grouping
A
Instructional Programs Academic Goals
* Giited
* Extended Day Kindergarten training, enhancement of the 1 * Preschool
* Scottsdale Prevention Institute
* DARE Instruction 1 Fine Arts Partnership for Instruction comprehension skills as a result of
I supplemental language arts lessons.
School Honors I ( I Arizona A+ School
Promotion Rate 1994- 95
Attendance Rate 1994- 95
)
Number of School Days 176
Hours of Instruction 6 hrs. and 50 min.
Fire Safely School of the Year 1995
Odyssey of the Mind to World Champ 1995
Math Olympiad Outstanding Students 1995 I This Arizona School Report Card was prepared by the Arizona Department of Education using a standardized format to combine information
provided by the local school with data Compiled from state records. Information is provided for the 799596 school year unless otherwise noted.
Endhent is the number of students enrolled as of Oct. I, 1994. Be aware that each school has special strengthsand needs and that all schools 8 benefif from the active involvement of parents in their children's education. If you have questions about the report card or need more information,
contact the school office. Look for Arizona School Report Canls at https'/ ade. state. az. udreportcarddon the Internet.
a- i
Appendix
School Site Council
Council Composition Council Duties
1 School Administrator( s)
1 Noncertified Employee( s)
4 Teacher( s)
3 Parent@)
1 Community Member( s)
0 Student( s)
* Budget
* Curriculum Development
* Extracurricular Activities
* Instructional Strategies
* Personnel Decisions
* Staff Development
For more information regarding the school site council, contact Doris Billings at ( 602) 451- 5020
School Responsibilities
To provide a stimulating and challenging academic
program; to provide a climate conducive to learning; to
honor and protect the rights of all students; to
communicate openly and regularly; to encourage parent
participation in the life of the school; to assure the safety
of students; to maintain a highly qualified professional
staff
Parent Responsibilities
Parents are urged to show support of their children in
several areas: provide a supportive home environment
conducive to study 2nd learning; motivate their children
to do their best by taking an interest in their work; be
role models by valuing education and by participating in
the activities of the school and parent organization.
Parents are expected to help their children abide by the .
district's Code of Student Conduct and the school's
rules.
I Scho~ S] a fety ( The school does not have a po! ice officer assigned to the school.)
I Number of incidents occuring on the school grounds that required the intervention
of law enforcement personnel and resulted in charges being filed. 0
I Number of students against whom the school filed charges during the 1994- 95
school year. 0
I Number of individuals other than enrolled students against whom the school filed
charges during the 1994- 95 schobl year. 0
Transportation Policy
Transportation is provided for K- 3 students 314 mile from school, and 4- 6 students 1 mile from school. Mid- day
transportation is provided for eligible kindergarten students. Service is provided for special education students whose
IEPs require it. Safety and welfare of student riders is the first consideration. All vehicles are maintained to provide
safe and efficient transportation. Bus evacuation drills are conducted for all students and staff.
For more information regarding transportation policy, contact Hal Dibler at ( 602) 451- 5050
a- ii
Appendix
Resources Available at School Site
Special Facilities Extracurricular Activities
Technology Lab Student CounciWearbook Staff Art Club
student Publishing Center Afterschool Enrichment Program Odyssey of the Mind
City Park as Extension of Campus Battle of the BooksNVord Masters Peer Tutoring and Mentoring
Outdoor Classroom Garden
Social Services
Day Care
Counseling Services
DARE
Afterschool Program
Crisis Intervention
Lunch Program
Health Services
For more information regarding social services, contact Doris Billings at ( 602) 451- 5020
Academic Achievement Indicators
Students were tested in grades 4, 7 and 10 using standardized, nationally norm- referenced achievement tests. Results
for 1992, 1993 and 1994 are reported in terms of the national percentile rank achieved on the lowa Tests of Basic Skills
for grades 4 and 7 and the Tests of Achievement and Proficiency for grade 12. For each of the three grade levels tested,
the percentile rank score for the school is provided in the subject areas of Reading, Language and Mathematics.
Corresponding national and state scores are provided for comparison.
Studsnts were tested in grades 3, 8 and 12 on their achievement of the Arizona Essential Skills using district- selected
assessments. An Essential Skills Completion Report ( ESCR) was compiled by each school to indicate the percentage of
students meeting achievement levels ( standards) determined by the district for competency in reading, mathematics and
writing ski! ls. Each subject area for each grade has several clusters of skills which are tested separately throughout the
schoo! year until mastery is achieved. For each cluster the standard to be met is given as a percentage of the possible
points on the test. The percentage of students tested is reported and the percentage of students determined to be
proficient is given.
lowa Tests of Basic Skills and Tests of Achievement and Proficiency
Percentile Rank Scores
School Arizona National
Grade 4 - 1992 - 1993 - 1994 1994 7
Reading 67 80 70 48 48
Language 62 69 67 48 49
Mathematics 66 76 73 49 46
Grade 7
Reading - - - 53 48
Language - - 53 48
Mathematics - - 5 1 46
Grade 10
Reading - - 5 1 47 '
Mathematics - - - 47 48
a- iii
Appendix
Student Achievement of Essential Skills 1994- 95
This summary lists the Essential Skills clusters, with the district standard, the percentage of students tested, and the
percentage of students achieving mastery.
GRADE 3
READING
Personal Experience Narrative
story
Informative Report
Communication
Poem
WRITING
Penonal Experience Narrative
Imaginative Story
Report
Communication
Poem
MATHEMATICS
Sorting and Classifying
Conjecturing
hadions
Using Money
Measurement
Patterns
Identifying Shapes
Interpreting Word Problems
GRADE 8
READING
Personal Experience Narrative
story
Informative Report
Communica: ion
Poem
Summary
Essay
WRITING
Personal Experience Narrative
Story
Repod
Communication
Poem
Summary
Specialired Expository Paper
MATHEMATICS
Statistics
Probability
Analyzing Data
Measurement
Geometry
Geometric Principles
Expressions and Equations
Patterns
Mathematical Reasoning
District
Standard
75%
75%
75%
75%
75%
75%
75%
75%
75%
75%
75%
75%
75%
75%
75%
75%
75%
75%
District
Standard
75%
75%
75%
75%
755' 0
75%
75%
75%
75%
. 75%
75%
75%
75%
75%
75%
75%
75%
75%
75%
75%
75%
75%
75%
Percent
Tested
100% -
86%
100% -
100%
89% -
83% - 6
100%
100%
100%
100%
100%
100%
100%
4 00%
Percent
Tested -
-
-
-
- -
-
-
-
-
-
-
- -
-
-
-
-
-
-
-
-
-
Percent
Mastery
11% -
74%
69% -
50%
78% -
43% -
95%
75%
73%
39%
87%
90%
81%
79%
Percent
- Mastew -
-
-
-
-
-
-
GRADE 12
READING
Personal Experience Narrative
Short Story
lnformative Report
Communication
Poem
Summary
Essay
Persuasive Passage
ReviewlEvaluationlCritique
WRITING
Personal Experience Narrative
Short Story
Report
Communication
Poem
Summary
Specialized Expository Paper
Persuasive Paper
Evaluation
MATHEMATICS
Analyzing Data
Progressions
Misuses of Statistics
Geometry
Measurement
Reasoning
Graphing
Statistics
Distrid
Standard
75%
75%
75%
75%
75%
75%
75%
75%
75%
75%
75%
75%
75%
75%
75%
75%
75%
75%
75%
75%
.75%
75%
75%
75%
75%
75%
Note: Under the Family Educational Rights and Privacy Act of 1974,
no individually identifiable references to students may be made.
Therefore, items of data containing information about three or fewer
students have been replaced with an asterisk (-) to protect student
privacy. Some columns contain a dash (-) to indicate ' not
applicable or no data available' because schools may not test all
subject areas or grade levels in the same year.