PERFORMANCE AUDIT
ARIZONA DEPARTMENT OF EDUCATION
DROPOUT PREVENTION EFFORTS
Report to the Arizona Legislature
By the Auditor General
June 1996
Report # 96- 6
DOUGLAS R. NORTON, CPA
AUDlTOR GENERAL
STATE OF ARIZONA
OFFICE OF THE
AUDITOR GENERAL
June 27,1996
Members of the Arizona Legislature
DEBRA K. DAVENPORT, CPA
DEPUTY aYDITOE GENERAL
The Honorable Fife Symington, Governor
The Honorable Lisa Graham Keegan
State Superintendent of Public Instruction
Transmitted herewith is a report of the Auditor General, A Performance Audit of the Arizona
Department of Education's ( ADE) dropout prevention efforts, This report is in response to a
May 30,1995, resolution of the Joint Legislative Audit Committee. The performance audit was
conducted pursuant to the provisions of A. R. S. 541- 2958. This is the first in a series of four
audit reports regarding public education.
According to ADE's most recent reports, over 12 percent of Arizona's high school students and
3 percent of Arizona's middle school students dropped out during the 1993- 94 school year.
We found that ADE needs to take a more active role in helping school districts prevent
dropouts. While Arizona school districts operate over 400 dropout prevention programs,
including alternative schools, counseling programs, truancy control efforts, and peer
mentoring opportunities, neither the districts nor ADE have assessed most of these programs'
effectiveness. Further, ADE provides limited technical assistance to districts regarding dropout
prevention programs. Finally, we found ADE needs to work with districts to improve the
accuracy of dropout data reported by the districts.
My staff and I will be pleased to discuss or clanfy items in the report.
This report will be released to the public on June 28,1996.
Sincerely,
~ btqflasR Norton
Auditor General
Enclosure
2 9 1 0 NORTH 44TH STREET . SUITE 4 1 0 . PHOENIX, ARIZONA 8 5 0 1 8 m ( 6 0 2 ) 5 5 3 - 0 3 3 3 rn FAX ( 6 0 2 ) 5 5 3 - 0 0 5 1
SUMMARY
The Office of the Auditor General has conducted a performance audit of the Arizona
Department of Education's ( ADE) dropout prevention efforts. This audit was conducted
pursuant to the provisions of A. RS. 541- 2958 and in response to a May 30,1995, resolution
of the Joint Legislative Audit Committee. This is the first in a series of four audit reports
we will issue regarding public education.
Arizona's school districts operate a variety of dropout prevention programs designed to
keep public school students in school and provide them with'alternatives to dropping out.
Dropout prevention efforts range from highly structured alternative schools to informal,
school- sponsored counseling and tutoring programs. However, schools, districts, and
ADE need more information to ensure that efforts are effective and directed at the
appropriate schools and students. Most programs have not been evaluated by districts or
the Department of Education. As a result, the real effect of dropout prevention programs
is unknown. Further, proving programs reduce dropout rates is difficult because schools'
dropout counts are inaccurate.
Expanded ADE Role Would
Benefit Dropout Prevention
Efforts Statewide
( See pages 5 through 10)
By taking a more active role, ADE could enhance the quality and effectiveness of dropout
prevention programs statewide. Arizona school districts operate over 400 of these
programs, including alternative schools, counseling programs, truancy control efforts,
peer mentoring opportunities, and after- school and summer school classes. However,
neither the districts nor the State have assessed most of these programs' effectiveness. As
a result, at least $ 25 million in district, state, and federal funds may be supporting
programs of questionable impact. Education experts agree that evaluation is necessary to
assess program impact; however, few districts evaluate their dropout prevention
programs. Further, ADE provides limited technical assistance to districts attempting to
evaluate programs.
In contrast, other states' education agencies have focused more efforts on dropout
prevention programs. These states, recognizing the benefits of addressing the dropout
problem, provide technical assistance to schools operating dropout prevention programs,
and in some cases, evaluate the programs. Additionally, some states have created
innovative methods to encourage schools to implement dropout prevention programs.
Arizona's Dropout Data
Do Not Accurately Reflect
the Number of Dropouts
( See pages 11 through 17)
Arizona's reported school and statewide dropout rates do not accurately reflect the actual
rates of dropouts. More than half of Arizona's dropouts are classified as such because
schools have not been able to locate the students after they leave the district. Many of
these students may have transferred to schools in other districts or states. However,
schools experience difficulty tracking these students and verifying whether students have
transferred to a new school. Other states improved their dropout data by successfully
tracking students using a statewide automated information system. The Department plans
to implement a statewide automated system that would allow easier tracking of students
by electronically connecting all Arizona schools and districts with ADE, but this plan is
in its early stages.
Moreover, schools' inconsistent recordkeeping also distorts dropout data. Schools fail to
consistently follow ADE guidelines for classifying student dropouts. ADE can improve
the accuracy of Arizona's dropout data by providing more training to district personnel
responsible for this recordkeeping.
Table of Contents
Introduction and Background ..........................
Finding I: Expanded ADE Role Would
Benefit Dropout Prevention Efforts Statewide .................................
Numerous Dropout Prevention
Programs Exist Statewide
ButMayBeIneffective ..........................................
Paae
1
Increased ADE Role Could Improve
District Efforts to Address the Dropout Problem ............................................ 8
Recommendations ........................................ 10
Finding II: Arizona's Dropout Data
Do Not Accurately Reflect
The Number of Dropouts ........................... 11
Dropout Figures
PossiblyInflated ........................................ 11
ADE Should Increase Staff Training to Improve Data Accuracy ....................................... 15
Recommendations .............................................. 17
Other Pertinent Information ........................... 19
Arizona's Dropout Rates Not Comparable ............................................... 19
1987 Dropout Prevention Program
Continuedhdefinitely .......................................... 20
Alternate Approach to Dropout Prevention ............................................. 22
iii
Table of Contents ( con't)
Agency Response
Appendix .......................................... a- i
Tables
Table 1 Types of Dropout Prevention Programs
Academic Year 1994- 95 .................................... 7
Table 2 Districts Currently Participating in
the 1987 Dropout Prevention
Program .............................................. 21
Figure
Figure 1 Types of Dropouts
Academic Year 1993- 94 .................................... 12
INTRODUCTION AND BACKGROUND
I - The Office of the Auditor General has conducted a performance audit of the Arizona
Department of Education's ( ADE) dropout prevention efforts. This audit was conducted
pursuant to the provisions of A. R. S. 541- 2958 and in response to a May 30,1995, resolution u of the Joint Legislative Audit Committee. This is the first in a series of four audits
regarding public education.
Arizona's Dropout Problem
According to ADE's most recent reports, over 12 percent of Arizona's high school students
and 3 percent of middle school students dropped out during the 1993- 94 school year.
Additionally, 32 percent of the class of 1993 failed to graduate within 4 years. Further,
national statistics show Arizona ranks poorly at keeping students in school when
compared to other states. The United States Bureau of the Census records information
1 about people not enrolled in schools and without a high school diploma or an alternative
credential, such as a General Educational Development ( GED) certificate. In 1990, the
Census reported a 14 percent dropout rate in Arizona for persons aged 16 through 19. This
1 appears high when compared to other states' rates, which had ranges of 5 to 15 percent. More recently, in 1993, Arizona ranked 39th in the nation, with 16 percent of its 18- to- 24-
year- olds not possessing a high school diploma or an alternative credential. , I
1 The Effects of Dropping
Out Are Substantial
Dropping out of school adversely impacts not only dropouts but society as well. Today's
dropout faces a bleak future, with few economic opportunities. Census figures show that
in 1960, dropouts were 50 percent more likely to be unemployed than non- dropouts;
however, by 1987, dropouts were 200 percent more likely to be unemployed. According 4 to a recent report
" Education, more and more, has become key to a productive and satisfying life. Gone are the
days when a lack of education didn't hurt one's chances fm finding good, steady work.
Opportunities are expanding for those with higher level skills and withering for those without
such skills."'
I
1. Education Testing Service Policy Information Center. Dreams Deferred: High School Dropouts in the United
States, 1995.
Moreover, dropouts impose a considerable burden on society. High school dropouts
represent over 80 percent of the U. S. jail and prison population. They also comprise nearly
half of the heads of households on welfare. Other consequences include foregone income
and tax revenues. One estimate suggests each year's class of dropouts will, in their
lifetime, cost the nation approximately $ 260 billion in lost earnings and foregone taxes?
Students At Risk
of Dropping Out
Students with certain characteristics are more likely to drop out than others. Common
indicators of a student's at- risk status include:
Low or failing grades,
Attendance problems,
Age above average for grade level,
Low socioeconomic status, and/ or
Non- English speaking background.
However, experts suggest that using these indicators to identify potential dropouts is
problematic since more than 50 percent of dropouts may not fit the " at- risK' criteria. Many
dropouts are students who lack the social, economic, ethnic, and racial characteristics
typically associated with dropouts.
Dropout Prevention
Programs in Arizona
State efforts to deal with the dropout problem include two statutorily established dropout
prevention programs. Laws 1987, Chapter 333 authorized certain districts to raise and
expend funds beyond their revenue control limit ( a district's total spending limit based
on its student count and transportation costs) for dropout prevention programs. Currently,
20 districts participate in the 1987 program. ( See Other Pertinent Information, pages 19
through 23, for more information on this program.) In addition, Laws 1988, Chapter 308
established a program supporting at- risk students within grades kindergarten through
third ( K- 3) and seventh through twelfth ( 7- 12). ADE selected 12 districts to participate in
1. LeCompte, M. D. and Dworkin, A. G. Giving Up On School, 1991.
the 7- 12 At- Risk Program based on the severity of their at- risk problem, their proposed
program, and their size and geographic location. In fiscal year 1995- 96, ADE reduced the
funding provided to the original 12 sites in order to provide funding for 15 additional
districts. In addition to these programs, school districts support many other dropout
prevention programs.
However, of the two statutorily established programs, only the 1987 Program will remain
after the 1995- 96 fiscal year. In 1996, the Legislature eliminated the At- Risk Program based
on recommendations in a Program Authorization Review conducted by the Joint
Legislative Budget Committee staff and the Office of Strategic Planning and Budgeting
staff in 1995. The review recommendations noted that the dropout prevention programs'
impact on dropout rates is unclear, and effective strategies for dropout prevention have
not been clearly documented.
Staff and Funding
Although at least $ 25 million is spent on dropout prevention activities statewide; the
statutorily established 1987 and At- Risk programs account for only $ 8 million of this
funding. The Legislature appropriated $ 2.2 million for the At- Risk Program and districts
raised more than $ 5.8 million in local revenues for the 1987 Program during the 1995- 96
school year? The remaining $ 17 million was funded by district budgets, and federal and
private grants.
Due to the At- Risk Program's elimination, no ADE employees will oversee dropout
prevention programs in 1996- 97. The employee who spent half her time collecting
information and monitoring programs will be reassigned. However, other ADE staff
continue to collect and analyze district enrollment and withdrawal information and
prepare dropout and graduation reports.
Scope and Methodology
This audit focuses on Arizona's public school dropout problem. The audit includes
information on state and local efforts to prevent students from dropping out, and the
problems with accurately identifying and capturing Arizona's total number of dropouts.
We examined Arizona's dropout problem from three perspectives. First, all Arizona
districts were surveyed to document the extent of dropout prevention activities
throughout the State. Site visits to several dropout prevention programs were then
conducted to venfy survey responses. Additionally, we conducted an extensive literature
Although the state dropout prevention program was placed in a block grant of $ 22.9 million for fiscal year 1995-
96, ADE expected $ 2.2 million to be spent on dropout prevention programs.
review to identify effective dropout prevention strategies and programs, and also
contacted other states to identify how they reduce dropout rates.'
Second, we examined the accuracy of the school districts' reported dropout numbers. Site
visits were conducted at 15 high schools within 10 districts to determine recordkeeping
practices regarding enrolling, withdrawing, and tracking students and to identify
inconsistencies between schools. The Department's efforts to develop a statewide
information and tracking system were also reviewed.
Finally, we reviewed Arizona's dropout rate calculation methodology and compared it
to the U. S. Department of Education's National Center for Education Statistics methodol-ogy,
and that used in several other states. National and state reports comparing various
calculation methodologies were also reviewed. In addition, ADE staff responsible for
dropout rate calculations were interviewed.
Findings are presented in two areas:
The effectiveness of the numerous dropout prevention programs districts operate is
unknown.
Arizona's reported dropout rates do not accurately represent the State's actual dropout
rate.
Additionally, the Other Pertinent Information section ( see pages 19 through 23), presents
Arizona's inability to compare its dropout rate with other states, the status of the RCL
dropout prevention program, and new strategies for dealing with the dropout problem.
The Auditor General and staff express appreciation to the Superintendent of Public
Instruction, the Arizona State Board of Education, Arizona school districts, and their staff
for their cooperation and assistance throughout the audit.
1. The following states were contacted during this audit: Arkansas, California, Colorado, Florida, Illinois,
Minnesota, Nevada, South Carolina, Tennessee, Texas, Washington, Wisconsin, and Virginia.
4
FINDING I
EXPANDED ADE ROLE WOULD
BENEFIT DROPOUT PREVENTION
EFFORTS STATEWIDE
" Little will be gained by throwing large sums offunds at ' dropout prevention" that is not
directed toward positive results. The real pay- ofi will occur when states use their money to
determine, through rigorous external evaluation practices, what works and to help school
districts adopt valid practices. "'
The Arizona Department of Education ( ADE) could do more to help districts address the
State's dropout problem. Districts annually spend millions of dollars on over 400
programs designed to keep students in school; yet their effectiveness is questionable, since
few of these programs have ever been evaluated. Moreover, ADE provides limited
technical assistance to districts operating dropout prevention programs. ADE should
consider experts' recommendations and follow other states' leads by providing districts
with increased technical assistance in dropout prevention programming and evaluation.
Numerous Dropout Prevention
Programs Exist Statewide
But May Be Ineffective
Although districts operate many types of dropout prevention programs, the effectiveness
of these programs is largely unknown. Neither the districts nor ADE adequately
determine program impacts through evaluation. Consequently, the State's and districts'
well- intended efforts may be misguided.
Districts report numerous eflorts statewide- To determine the extent of dropout
prevention efforts statewide, we conducted a survey of every school district in Arizona.
Overall, 77 percent, or 185 districts, responded. In the survey, district administrators were
asked to include as a dropout prevention program any strategy at any grade level
designed to prevent students from dropping out, or to retrieve students who have already
dropped out of school. Ninety- five district administrators described at least one dropout
prevention effort operating within their district. Additionally, these districts reported the
'. The Southern Regional Education Board. reaching the Goal to Reduce the Dropout Rate. 1991.
5
number and grade level of students served, provided budget information, and desc
ribed program evaluation efforts.
Table 1 ( see page 7') lists the types of dropout prevention programs operating in Arizona
during the 1994- 95 school year.
The survey revealed that districts expend significant funding and efforts on programs.
In fiscal year 1994- 95, districts spent more than $ 25 million on these programs. However,
this figure may understate total dropout prevention program funding as only half of the
districts provided budget information for their programs. Program costs ranged from zero
for volunteer tutors in elementary schools to over $ 160,000 for an alternative school.
Districts utilized state, federal, and private grants, and local district funds to finance
dropout prevention efforts. However, districts funded the majority of efforts with local
tax dollars.
Moreover, approximately 50,000 students, about 1 in every 13 students enrolled in
Arizona's public schools, participate in a dropout prevention effort. This figure is
conservative, since it excludes students served by 31 programs operating school- wide,
such as counseling or attendance incentive programs.
Dropout prevention efforts can serve students at every grade level from kindergarten
through the 12th grade. Fifteen districts reported operating almost 100 dropout prevention
programs aimed solely for students in grades K through 6, and 75 districts operate over
200 programs targeted to students in grades 7 through 12. Additionally, several school-wide
efforts serve students at all grade levels.
Programs may be ineffective- Because districts spend millions of dollars on dropout
prevention programs, it is important to ensure that dollars and efforts are appropriately
targeted. However, without evaluation, the effectiveness of dropout prevention programs
remains uncertain. Experts in dropout prevention strongly recommend well- designed
evaluation as a component of dropout prevention efforts to determine what strategies
work. One expert states, " It is senseless to implement and run educational programs
without systematically assessing their effects on intended beneficiaries."' Researchers
recommend measuring program performance using test scores before and after
participation in the program, and long- term tracking of graduation and dropout rates,
enrollment in academic courses, attendance rates, and participation in extracurricular
activities. Additionally, proper evaluation should include a comparison group that
provides a baseline for determining program effectiveness.
Nattiello, G., McDill, E. L., & Pallas, A. M. Schooling Disadvantaged Children: Racing Against Catastrophe. 1990.
6
Table 1
Types of Dropout Prevention Programs
Academic Year 199495
Alternative School
An exclusive alternative learning environment for
students failing in the conventional school.
Alternative Classroom
An alternative class period held on the conven-tional
school campus. Alternative classes are
designed to help potential dropouts increase their
academic skill level.
Behavior Modification
Any program designed to modify violent, destruc-tive,
or disruptive behavior patterns. Examples
include: substance abuse, gang prevention, in-school
suspension, and intensive discipline pro-grams.
incentives
Students receive positive reinforcement for good
behavior, high academic achievement, or regular
attendance.
Peer Assistance
Students provide counseling and tutoring to fellow
students.
Teen Pregnancy and Parenting Program
Program for teen parents focusing on prenatal
health and parenting skills in addition to the
conventional academic curriculum.
Truancy Control
Programs designed to decrease absenteeism.
Comprehensive Program Tutoring
Students participating receive academic, health, Academic assistance provided to students by
and social services. These programs combine the adults in groups or one- on- one.
services of several state and local agencies
Other
Counseling Programs that did not fit into any of the categories
Advice and guidance provided to students by above.
adults in group or individual sessions.
Extended School Day or Year
Allows students to make up missed or failed
classes or simply increase their number of credit
hours.
Source: Auditor General survey of dropout prevention efforts.
While experts agree that programs must be evaluated to determine program effectiveness,
Arizona's districts evaluate few programs. Although districts reported evaluating 43
percent, or 178, of all dropout prevention efforts, further analysis of the evaluations
revealed that only 20 programs were thoroughly evaluated. Districts evaluated these
programs using outcomes determined through tracking of participants to measure
program effectiveness. The remaining 158 programs used less rigorous evaluation
techniques including student satisfaction surveys, feedback from parents, and teacher
evaluations to measure the effectiveness of dropout prevention programs.
Increased ADE Role Could
Improve District Effort. to
Address the Dropout Problem
By expanding its dropout prevention role and providing technical assistance, ADE could
help districts improve dropout prevention programs. Currently, the Department provides
few services that support dropout prevention efforts in school districts. Experts and
district administrators agree that an expanded ADE role could enhance statewide efforts
to prevent students from dropping out In contrast to Arizona's limited technical assistance
role, several states assist school districts by providing a clearinghouse of dropout
prevention information and by evaluating programs.
ADE provides few smices- Despite districts' extensive efforts in dropout prevention
programming, ADE provides limited technical assistance in this area. Technical assistance
can range from compiling and disseminating information on promising dropout
prevention strategies to assisting with evaluation designs. While larger districts employ
evaluation staff to provide these services, smaller districts may lack such resources. ADE
also lacks these resources, as it currently commits only half of one employee's time to the
two dropout prevention programs it administers. Additionally, this employee's position
requires expertise in program management, not in research evaluation techniques.
Further, the Department does not maintain information on most dropout prevention
programs funded by districts. One principal reported that he had to research dropout
prevention programs himself because ADE could not provide information or technical
assistance.
The state- funded At- Risk Program was recently. eliminated, in part, because ADE's
evaluation efforts failed to identify effective dropout prevention strategies. The At- Risk
Program was designed to provide districts with technical and funding assistance so they
could implement dropout prevention and retrieval strategies. The 1988 enabling
legislation created the program to specifically identify effective strategies that could be
replicated statewide. However, ADE's performance measures did not adequately gauge
program effectiveness. Moreover, the lack of cost and success ratios for each strategy
prevented the identification and replication of successful programs.
State rob is needed- Education professionals recommend an expanded technical
assistance role for departments of education that includes providing evaluation support
and identrfying best practices. Several of these professionals agree that technical assistance
could substantially improve the evaluation of programs at the district level. For example,
the Soutkem Regional Education Board ( SREB) suggests that states provide local systems
with technical assistance for reducing dropout rates.' This can include publishing and
distributing descriptions of successful dropout prevention strategies, conducting
workshops and conferences, and furnishing on- site technical assistance to all school
systems. Furthermore, SREB strongly encourages states to support research and evaluation
efforts of dropout prevention programs.
In contrast to ADE's limited role, other states actively support dropout prevention by
providing technical assistance and disseminating isormation. For example:
Nevada, in response to an increasing dropout rate, recently made dropout prevention
a top priority. The Departmpnt formed an eight- person task force to develop strategies
to combat the State's dropout problem. The group hosted a conference on dropout
prevention to identify common characteristics of dropouts and is surveying districts
to identify programs operating statewide. Further, the Nevada Department of
Education established a Dropout Prevention Advisory Team comprised of district staff,
business leaders, and Nevada citizens. The Team will identify model dropout
prevention programs and disseminate, train, and assist school staff in selecting and
implementing programs that best meet local needs. Districts will then be responsible
for evaluating the effectiveness of programs in conjunction with the Department.
The Tennessee Department of Education provides incentives to districts operating
model programs. An interagency task force evaluates model programs using outcome
data submitted by program directors. The task force recognizes ten model programs
each year. If chosen as a best practice, the district receives $ 6,000 and the program's
personnel provide information to other schools on how their programs work.
Additionally, the Tennessee Department of Education maintains a clearinghouse of
information on dropout prevention. Schools can contact the Department for
information on dropout prevention research conducted nationally, or in other states,
as well as plans for programs operating within Tennessee.
In Florida, the Department of Education performs compliance audits to ensure that
dropout prevention efforts meet state standards. The Florida Dropout Prevention Act
of 1986 established comprehensive dropout prevention programs that serve students
in five program categories: Educational Alternatives, Teenage Parent, Substance
Abuse, Disciplinary, and Youth Services. The Department audits these programs.
Programs must demonstrate service to the proper population of students, and
demonstrate effectiveness by submitting achievement data. This data includes reading
and math test scores, and graduation, dropout, and promotion rates.
While ADE should do more, recent staffing cuts may hinder the Department's ability to
provide technical assistance to districts. ADE recently significantly reduced its staff and
-
1. The Southern Regional Education Board is an education research organization representing 15 states.
9
also moved many employees into new positions. In fact, due to the elimination of the At-
Risk Program, at the end of fiscal year 1995- 96, ADE will cut all staffing for dropout
prevention.
Therefore, ADE should consider increasing its technical assistance capacity for dropout
prevention. The current ADE dropout prevention program specialist told auditors that the
Department would require at least two full- time staff to provide adequate technical
assistance to district dropout prevention programs. For example, ADE commits 1.8 FTEs
to administer the At- Risk Preschool Education Program, which includes monitoring and
technical assistance to programs in 103 school districts. A comparable staffing commitment
to dropout prevention programs, which serve students at every grade level, might allow
ADE to provide technical assistance to all districts.
To help collect and disseminate information, ADE should also consider establishing a
clearinghouse for dropout prevention. A clearinghouse would provide an easily accessible
centralized collection of dropout prevention program descriptions, as well as information
about whom to contact to obtain more in- depth information about those programs. A 1993
Morrison Institute Report indicated that establishing a clearinghouse on dropout
prevention would cost approximately $ 55,000 annually for a library specialist and other
costs, including clerical assistance and administrative overhead. Further, $ 16,000 in start-up
costs is required for computer equipment, software development, and general
equipment.
RECOMMENDATIONS
ADE should develop its technical assistance capacity to assist districts in dropout
prevention programming by establishing:
1. A clearinghouse of information on dropout prevention
2. Expertise in dropout prevention programs
3. Expertise in dropout prevention program evaluation.
FINDING II
ARIZONA'S DROPOUT DATA DO NOT
ACCURATELY REFLECT THE
NUMBER OF DROPOUTS
Detaining dropout prevention programs' effectiveness is hindered by inaccurate data.
In Arizona, more than half the students counted as dropouts represent students who have
" disappeared" from schools that are unable to verify the students' location. Some of these
students may have transferred to other schools. As demonstrated in other states, ADE can
improve dropout data accuracy by implementing a statewide automated student tracking
system to identify students who transfer from one school to another. Further, schools'
inconsistent recordkeeping practices also distort dropout data. ADE can improve the
State's dropout data accuracy by expanded training for recordkeeping staff.
Background
Schools cannot prescribe appropriate dropout programs to targeted student populations
without reliable data. As noted in Finding I ( see pages 5 through lo), districts operate
numerous dropout prevention programs. However, if students who leave school are not
adequately identified or categorized, districts may inappropriately target funding and
programs. For example, a school may be targeting programs to students in the 11th and
12th grade, when their dropout rates are actually higher for 8th- and 9th- grade students.
Programs based upon incorrect dropout data may fail to impact the intended student
population.
Dropout Figures
Possibly Inflated
Arizona's dropout figures may be inflated by transfer students who are inappropriately
classified as dropouts. Over half of all dropouts are students who have disappeared from
the school system but who may be enrolled elsewhere. Arizona schools have experienced
difficulty tracking these students and confirming transfers. Therefore, to improve data
accuracy, ADE should develop a statewide system to track students who move from one
school to another within the State.
ADE requires school districts to document and report the total number of students who
withdraw from their schools and do not return by the yeais end, which the Department
uses to compile a statewide dropout report. Dropouts fall into one of four categories: 1)
chronically ill, 2) expelled or on long- term suspension, 3) confirmed by the student or
parent/ guardian as a dropout, or 4) absent 10 consecutive unexcused days and whose
status is unknown. Status unknown students stopped attending school without stating
their intentions. They may actually have dropped out, or they may have enrolled in other
schools. However, state law requires schools to count these students as dropouts.
Many s t ~ tclsuss ijied as dropouts may not be dropouts- More than half the students
counted as dropouts are students whose status is unknown. In academic year 1993- 1994,
56 percent of grade 9- 12 dropouts, or 14,328, were students whose status was unknown.
Further, in grades 7 and 8,84 percent were in this category. As shown in Figure 1, only
39 percent of high school and 4 percent of 7th- and 8th- grade students were confirmed
dropouts.
Figure 1
Types of Dropouts
Academic Year 1993- 94
0
Dropout Types: Illness Expelled Status Unknown Dropped Out
7
Grade 7- 8 r...'...-..... ".. .... J.. Grad9e12
Source: ADE's Dropout Rate Study 1993- 94: Annual Dropout Rates in Arizona Public Schools- Grades 7
through 12.
According to ADE's dropout report, most of the status unknown students may have
moved or enrolled in other schools without notifying the schools they left. Schools can
reclassify students as transfers and remove them from their dropout count only if they
receive a transcript request from another school or notification from a responsible adult,
such as a parent or school official. Unfortunately, despite a statutory requirement to do
so, elementary schools often fail to request new students' prior records. In contrast, high
schools are more diligent in requesting records because they must verify credits earned
toward graduation.
Statewide tracking system needed- Currently, schools experience difficulty tracking
students who transfer to other schools without notification. However, other states track
many of these students using statewide automated information systems. While ADE is
now developing a similar system, much work remains before students can be electroni-cally
tracked.
Schools' ability to track students is limited- In the absence of a transcript request or
notification, schools try to locate status unknown students to confirm they have either
dropped out or are still in school. Some students can be " recovered" and reclassified as
transfers or confirmed as dropouts through tracking. For instance, one attendance clerk
reduced her school's total number of unconfirmed dropouts by 40 percent. During
academic year 1994- 95, she mailed 178 form letters to parents of unconfirmed dropouts.
Fifty- four returned letters confirming that their children had transferred to another school.
In a second effort, she phoned all parents who had not responded to the letter and verified
that an additional 18 students transferred to other schools.
However, limited resources and the sheer number of students whose status is unknown
at some schools iake tracking difficult. To determine the extent of schools' tracking
efforts, we conducted site visits at 15 high schools across the State.* Although 80 percent
of the schools visited routinely attempt to track students whose status is unknown, some
commit more time than others. For example, the number of classified staff ( attendance
clerks and registrars) responsible for tracking students ranged from one to three, with
teachers and administrators also helping to track students at some schools. Home visits
are made by staff at only five schools. Additionally, one school waits until the school year
is over before trying to find unconfirmed dropouts, which may be months after students
disappear.
Otlm states i~ tzplementinga utoltsated systems - A lthough tracking efforts exist at the
school level, they are time- consuming and still result in large numbers of status unknowns
statewide. A more systematic method of tracking students who transfer from one school
1. A stratified sample of school districts was used to evaluate tracking efforts around the State. The sample
was first stratified by average daily membership ( ADM), total expenditures, and number of dropouts. We
then selected districts that fell in the 25th, 50th, and 75th percentiles for ADM, dropouts, and expenditures.
Overall, 10 districts were selected including unified and union districts, urban and rural districts, large and
small dish. icts, and districts with ethnic and geographic diversity. Fifteen high schools within the 10
districts were then randomly chosen for site visits to illustrate recordkeeping practices around the State.
to another, such as a statewide electronic tracking system, could more efficiently reduce
the status unknown category and increase the accuracy of dropout figures. Several other
states either have a statewide automated student information system in place, or are
planning to implement one. Currently, Texas and Florida use statewide automated
information systems. For example;
Texas uses the Public Education Information Management System ( PEIMS) to track
student enrollment, financial information, and staffing. Using social security numbers
or unique state identification numbers, Texas' state education department conducts
automated statewide searches to track unconfirmed dropouts and to identify transfer
students. In the 1992- 93 school year, the process was expanded to locate students who
received a GED, graduated, were expelled, or were previously counted as dropouts.
In academic year 1992- 93, Texas identified 8,317 students, or 16 percent, of reported
dropouts. The following year, using PEIMS, Texas found 18 percent of its reported
dropouts.
Florida also operates an automated information system. To reduce status unknowns,
Florida's Department of Education ( FDE) conducts an annual search in August.
Students are tracked or matched by social security number or assigned a unique
identification number, date of birth, and another field as a student identifier.
Additionally, each October, districts can request FDE to locate students who failed to
return after summer break. As a result of electronic tracking, FDE found the majority
of unconfirmed dropouts attended another school in the same district. They found
other students enrolled in community colleges.
ADE developing an automated sljstmz- While ADE is currently developing a statewide
automation system, much work remains before it can be used to improve dropout data
accuracy. ADE's State Automation Plan calls for a technological system to connect all
Arizona schools and districts with the Department through the Internet. The system will
incorporate student, financial, and human resource data. However, ADE still needs to
address several details related to student tracking, including:
Student Identification Numbers- ADE is currently writing a policy that would grant
it the authority to collect and use student social security numbers, which must be
approved by the Attorney General's Office. Further, legislation requiring districts to
provide student level information and social security numbers may be necessary.
Other states have had some difficulty obtaining parental consent for the use of social
security numbers for all students. If the same holds true in Arizona, ADE will need to
develop an alternate identification system for the remaining students.
Mechanisms for Tracking Students - Exactly how students will be tracked has not
been determined. In other states, education departments attempt to match the
identification numbers of status unknown students with those of students currently
enrolled in other schools. This would require that ADE staff conduct matches and
inform schools of students found to be enrolled elsewhere.
Cost- While ADE estimates over $ 11 million will be needed to fully implement the
system, the Legislature has appropriated only $ 250,000 for a pilot study. However,
according to ADE management, student tracking will not be part of the pilot.
ADE Should Increase Staff
Training to Improve Data Accuracy
Before ADE can rely on an automated system to track students, it needs to ensure schools
have adequate recordkeeping processes. Schools' inconsistent recordkeeping contributes
to inaccurate dropout counts. To improve data accuracy, ADE should expand and
improve its training for school recordkeeping staff.
Recmdkeeping problems prevent accurate counts - Schools' recordkeeping problems also
distort dropout counts. Our site visits to 15 schools during the audit also included an
examination of recordkeeping practices. The visits uncovered recordkeeping problems at
many schools. The significance of these problems is illustrated when ADE aggregates
inaccurate data from each school. Recordkeeping problems include:
Misuse of categories - Although ADE developed guidelines for classifying students
as dropouts, schools do not always follow them. For example, one school places all
status unknown dropouts into the confirmed dropout category, thus overestimating
the number of confirmed dropouts. Another school automatically counts all entering
9th- grade students who register but fail to attend as " transfers," possibly underesti-mating
the number of dropouts.
Disregard for preregistration and enrollment processes - Certain schools disregard
preregistration and enrollment procedures, resulting in artificially elevated dropout
counts. During the spring semester, high school counselors visit 8th- grade classes and
preregister students. However, according to ADE policy, high schools should not use
preregistration rosters for their final 9th- grade enrollment counts because some of
these students may not have completed the 8th grade, or may have transferred prior
to the end of the school year. When these students fail to attend in the fall, they will
be counted as dropouts. Two high schools reported that middle schools failed to
supply them with a list of graduates as mandated by the policy. Therefore, the high
schools used their preregistration lists for fall enrollment.
Increased stafitraining could improve accuracy- Although ADE plans to implement a
statewide automated student data system, a lack of sufficiently trained school- level staff
may hamper these efforts. Since accuracy is determined by individuals collecting,
I maintaining, and reporting student data, errors will persist with the implementation of
the automation plan if the current recordkeeping problems continue. Increased training
I
I
I could improve the recordkeeping process and thus improve dropout data accuracy.
In academic year 1995- 96, ADE offered 10 workshops to districts and school record-keeping
staff to provide instructions for required reports. Topics covered include the use
of enrollment and withdrawal codes, absence policies, and guidelines for classifying
students as dropouts. Although the recordkeeping staff we surveyed rated this training
highly, ADE reports that only one fourth of school- level recordkeeping staff attended this
training. While approximately 600 staff attended the most recent training, there are over
2,000 school- level recordkeeping staff statewide. Further, some of the attendees may have
been district- level staff.
The majority of recordkeeping training occurs at the school and district levels. However,
one- third of the staff members trained by district personnel perceived their training as
insufficient, and approximately one- third who received on- the- job training at the school
level felt it could be improved. Further, 10 percent of staff surveyed received no training
at all.
Because ADE training is limited, ADE should consider periodically monitoring schools'
recordkeeping practices to further improve its dropout data. Performance reviews of two
other states' dropout prevention efforts included recommendations for monitoring school
recordkeeping practices. Both the Florida and California Departments of Education were
urged to monitor student records to ensure that proper withdrawal codes are being
assigned or to confirm the accuracy of dropout data submitted by schools. Not only could
problems be addressed immediately, training to prevent common mistakes could be
incorporated into future sessions. As discussed in Finding I ( see pages 5 through lo),
limited staff resources may also hinder ADE's efforts to monitor districts' recordkeeping
practices.
RECOMMENDATIONS
1. ADE should continue its efforts to develop a statewide electronic student information
management system that includes a tracking component.
2. To improve consistency in recording student dropout rates, ADE should increase
participation and expand the formal training designed to help staff understand and
implement dropout and withdrawal procedures.
3. ADE should periodically review and verify the accuracy of dropout data submitted
by schools and districts.
( This Page Intentionally Left Blank)
OTHER PERTINENT INFORMATION
During the audit, we collected other pertinent information on 1) Arizona's dropout rate,
which is not comparable with many other states' rates; 2) a program authorizing certain
districts to raise additional funds for dropout prevention programs; 3) and an alternate
approach to preventing students from dropping out
Arizona's Dropout Rates
Not Comparable
Arizona's dropout rates cannot be compared to many other states' dropout rates because
states use different calculation methods. Although the National Center for Education
Statistics ( NCES) recommends a specific calculation methodology to facilitate interstate
comparisons, many state education agencies, including Arizona's, prefer to use different
methods.' ( See Appendix for a complete discussion of Arizona's dropout calculation
methods.) For example:
States use different time frames to classify students as dropouts. NCES recommends
states count dropouts over a 12- month period, which includes data from more than 1
year. However, other states' time frames ( including Arizona's), span only one school
year.
Additionally, states differ in how enrolled students are counted. While some states and
NCES count all students enrolled as of a particular school day, Arizona's and other
states' counts capture all students who enroll during an entire school year.
States also vary in the types of students counted as dropouts. NCES does not count as
dropouts students who leave school during one school year but return by October 1
of the next school year. In contrast, Arizona counts as a dropout any student who left
school, regardless of whether the student returned the next fall.
Despite not conforming to NCES' method for calculating dropout rates, Arizona's method
appears appropriate. Arizona employs a cumulative approach to count enrolled students
during a school year. Advocates of cumulative enrollment counts cite increased accuracy,
especially in states with high student mobility, such as Arizona. In contrast, NCES' point-
1. The National Center for Education Statistics, within the United States Department of Education, collects,
analyzes, and publishes statistics related to education in the United States. Currently, 18 states conform
with NCES' dropout rate calculation methodology.
in- time count ignores the impact of student mobility, possibly leading to distorted dropout
rates. For example, migrant students transferring into a school after the enrollment count
will not be counted as enrolled, but can be counted as dropouts.
Others agree a cumulative enrollment count is more accurate. First, an NCES consultant
stated that " the most representative and valid" dropout rate would use a cumulative
enrollment count Moreover, a task force established to study NCES' dropout rate formula
noted that a cumulative approach may provide the best method to account for student
mobility. However, because the task force believed this method would impose an
increased data burden on states, it is not the method NCES selected.
Arizona may eventually be able to use both its current formula and submit data according
to the NCES definition. To use the NCES definition, Arizona would have to calculate
dropout rates based on two school years, possibly creating an additional data collection
burden. However, with a statewide automated tracking system, this would be a much
easier task.
1987 Dropout Prevention
Program Continued Indefinitely
In 1987, the Arizona Legislature enacted a dropout prevention program spanning three
years to assist students in grades 4 through 12. Laws 1987, Chapter 333 authorized eligible
districts to raise additional funds for dropout prevention programs. Districts that
experienced at least a 1 percent loss of students between their 40th- and 100th- day
enrollment counts were eligible to participate. In addition, these districts were required
to develop and submit to the Board a comprehensive dropout plan for grades 4 through
12, including specific goals to reduce the dropout rate, absenteeism rate, and student
failure rate, and to increase student achievement.
During the program's initial three years, the participating districts changed. Although 31
districts operated programs during at least 1 year, only 14 districts participated
throughout all 3 years. Since district eligibility depended on a net loss of students, districts
could become ineligible even though they still had a dropout problem. For example, a
rapidly growing district with a serious dropout problem may not have recorded a net loss
of students. However, another district could have recorded a loss of students that was not
due to a dropout problem.
Several statutes continued the program after the initial three- year period.
1990 legislation extended the program for fiscal year 1990- 91, but limited participation
to the districts that participated during the prbgram's third year. Moreover, these
districts were allowed to raise funds only to the level raised during the previous year.
1991 legislation authorized the same districts to continue their programs during the
next two years, and continued to limit the amount of funding districts could raise for
their programs. For these two years, participating districts were also required to
submit documentation describing how their dropout prevention programs improved
student performance and retention, which ADE submitted to the Legislature.
w Finally, 1992 legislation extended participation of these same districts until the
Legislature fully funds dropout prevention programs statewide.
For the 1995- 96 school year, the State Board of Education approved 20 districts for
continued program participation. Table 2 contains a complete listing of these districts,
which raised combined revenues of $ 5.8 million designated for dropout prevention
programs.
Table 2
Districts Currently Participating in the 1987
Dropout Prevention Program
Amphitheater Unified School District
Casa Grande Union High School District
Chandler Unified School District
Flagstaff Unified School District
Glendale Union High School District
Holbrook Unified High School District.
Indian Oasis/ Baboquivari Unified
School District
Mesa Unified School District
Miami Unified School District
Mingus Union High School District
Mohave Union High School District
Phoenix Union High School District
Scottsdale Unified School District
Sierra Vista Unified School District
St. John's Unified School District
Sunnyside Unified High School District
Tempe Union High School District
Tolleson Union High School District
Tucson Unified School District
Yuma Union High School District
Source: Information provided to State Board of Education Members for the November 1995
board meeting.
ADE has had little involvement in this program since its inception. ADE is not funded
for program administration, such as monitoring and evaluation, or for providing
technical assistance to eligible districts. It merely collects eligible districts' comprehensive
dropout prevention plans each year and presents their status to the State Board for
annual approval. In 1989,1991, and 1992, ADE was also required to submit reports to the
Legislature illustrating the participating districts' dropout and absentee rates, and other
performance indicators.
Alternate Approach to
Dropout Prevention
Several experts suggest a more comprehensive approach to address the dropout problem
rather than creating specific programs targeting only certain students. According to these
experts, instead of focusing only on the students traditionally deemed at- risk, schools
should consider all students potential dropouts, and address their needs as such. A
Phoenix Union High School District's Task Force on Alternative Education concluded
that the dropout problem may be a reflection of schools' inability to adjust to changing
student populations. For example, current attempts to address the dropout problem
involve establishing more alternative programs while at the same time keeping
traditional schools intact, resulting in an expensive dual school system. The Task Force
recommends phasing out the dual system by enhancing and internally diversifying the
traditional school to meet the needs of a more diverse student population. Further, one
expert believes special programs or services targeted at certain groups of students may
expend resources without developing the organizational capacity of schools. In looking
for cost- effective alternative methods for preventing student from dropping out, the 1995
Program Authorization Review also suggests implementing systemic school reforms
" aimed at improving education for all students rather than ' enrichment' programs aimed
at only ' at- risk' students."
To meet the needs of all students, the dropout prevention literature suggests implement-ing
school- wide improvements, which apply characteristics of dropout prevention
programs to the entire student population. One way to accommodate students' various
learning styles and goals is to incorporate best practices learned in alternative schools.
According to the Morrison Institute's review of Arizona's at- risk programs, the alternative
school model is the most effective and positively perceived delivery system for at- risk
students. Alternative schools meet the needs of a diverse student population by
integrating a variety of instructional, vocational, and support strategies into a comprehen-sive
system. These schools offer an independent program of study for students to earn
credits toward graduation and flexible schedules with classes from the early morning to
late evening.
Additionally, the Phoenix Think Tank, a community- wide coalition comprised of
education, business, government, and community organization representatives, recently
studied best practices in education in the urban Phoenix area. The study asked what is
working " to ensure that ... students enter, re- enter, and remain in school until their
maximum potential and goals are realized." Several school- wide improvements
suggested as best practices include:
Looking for new ways to impart knowledge to the learner by using technology in the
classroom; student- tostudent tutoring; and accelerated and individualized learning,
I Integrating the curriculum by tying several courses to one project; linking classroom
activities to real- world problems; or addressing self- esteem, critical thinking,
I independent learning, and communication skills in courses, and
Employing a holistic approach to the student by involving parents in student learning I both in the classroom and at home; providing social/ support services for families at
the school, and providing personal counseling and follow- up to students.
I
I
I
I
B
I
I
I
u
I
I
I
I
( This Page Intentionally Left Blank)
Agency Response
( This Page Intentionally Left Blank)
State of Arimna
Department of Education
Lisa G r a b Kcegan
SupcrL~ tendcnotf
Public ~ nslludion
The Honorable Douglas R Norton
Auditor General
2910 N- 44th Street, Suite 410
Phoenix, AZ 85018
Dear Mr. Norton:
Tbe Arizona Department of Education hns reviewed your performance audit report
pertdning to dropout prevention efforts. We do not believe that any commcnts and
information are necessary for clarifiurtion. We rare bapically in agreemenl with your
findings und conclusions.
The department is in thc process of implementing some of your recommendations at the
current time and others will follow rrl a later datc.
I wish to compliment your tcam members on their thoruughness and professionalism.
Paul N. Street
Associate Superintendent
APPENDIX
( This Page Intentionally Left Blank)
Appendix
This appendix presents the basic dropout rate formula, as well as the methods Arizona
and NCES use to calculate their dropout rates.
Dropout Rate Formulas
for Arizona and NCES
A dropout rate measures the number of students who drop out of school as a proportion
of the pool from which the dropouts originated. Thus, each dropout rate includes a
numerator ( the number of dropouts) divided by a denominator ( number of students
enrolled).
Basic dropout rate = number of drovouts
number of students enrolled
Calculating a dropout rate first requires identifying the number of students who drop out
during one school year. Arizona's dropout definition attempts to count all students who
drop out over an entire 12- month period. A dropout is a student who was in school at
the end of the prior school year, or at any time during the current school year, but who
was not enrolled at the end of the current school year. Additionally, Arizona's dropout
definition includes certain types of students and excludes others. For example:
Students who transfer to a GED preparatory class are considered dropouts, regardless
of whether or not they earn a GED certificate.
Students who leave school but return before the end of the school year are not
considered dropouts.
NCES' dropout definition differs from Arizona's. While Arizona counts dropouts over
a 12- month period, which usually extends from July 1 of one school year to June 30 of the
next, NCES' reporting calendar for counting dropouts goes from October 1 of the prior
school year to September 30 of the current school year. Therefore, dropouts who return
to school by October 1 of the following year are not counted as dropouts by NCES.
Arizona does count these students as dropouts; however, at one time, ADE calculated
that 14 percent of their dropouts return for the next school year. Moreover, NCES'
definition does not count students who transfer to a state- or district- approved program
a- i
leading to a General Education Development ( GED) certificate as dropouts, while
Arizona's definition considers GED recipients dropouts.
Also necessary to calculate a dropout rate is the total number of students enrolled during
the year. While Arizona's enrollment count is based on a cumulative count of all students
enrolled during a 12- month period, NCES includes all students enrolled on October 1.
Students who enroll after this date are not included in a state's total enrollment.
a- ii
