Performance audit, Arizona State Board for Vocational and Technological Education

              State of Arizona
Office
of the
Auditor General
PERFORMANCE AUDIT
Report to the Arizona Legislature
By Douglas R. Norton
Auditor General
ARIZONA STATE
BOARD
FOR VOCATIONAL
AND
TECHNOLOGICAL
EDUCATION
June 1998
Report No. 98-10
2910 NORTH 44th STREET • SUITE 410 • PHOENIX, ARIZONA 85018 • (602) 553-0333 • FAX (602) 553-0051
DOUGLAS R. NORTON, CPA
AUDITOR GENERAL
DEBRA K. DAVENPORT, CPA
DEPUTY AUDITOR GENERAL
STATE OF ARIZONA
OFFICE OF THE
AUDITOR GENERAL
June 3, 1998
Members of the Arizona Legislature
The Honorable Jane Dee Hull, Governor
Mr. Kenneth R. Bennett, President
Arizona State Board for Vocational and Technological Education
Transmitted herewith is a report of the Auditor General, A Performance Audit of the Arizona State Board
for Vocational and Technological Education. This report is in response to a May 27, 1997, resolution of
the Joint Legislative Audit Committee. The performance audit was conducted as part of the Sunset
review set forth in A.R.S. §§41-2951 through 41-2957.
The report addresses the Board’s ability to fulfill its statutory role in overseeing vocational education. In
order to fulfill its mandate, the Board needs to address and resolve many of the important issues
impacting vocational education. For example, the Board should pursue partnerships with Arizona
employers to better determine how vocational programs can meet their needs and encourage school
districts to implement the Board’s Workplace and Technology Skills standards. Further, the Board needs
to address problems in implementing recent state funding changes that tie funding more closely to
vocational programs’ success in preparing students for work. To provide better oversight, the Board
needs to devote additional time to vocational education issues at its meetings and revitalize its Vocational
Education Advisory Committee.
To further enhance its oversight role, the Board should also ensure improvements are made in the
Department of Education’s (ADE’s) system for monitoring vocational education program outcomes.
While ADE collects information about school district program outcomes, several problems limit the
usefulness of this information in assessing program quality.
As outlined in its response, the Board agrees with all of the findings and recommendations.
My staff and I will be pleased to discuss or clarify items in the report.
This report will be released to the public on June 4, 1998.
Sincerely,
Douglas R. Norton
Auditor General
Enclosure
i
SUMMARY
The Office of the Auditor General has conducted a performance audit and Sunset review of
the Arizona State Board for Vocational and Technological Education, pursuant to a May 27,
1997, resolution of the Joint Legislative Audit Committee. This audit was conducted as part
of the Sunset review as set forth in A.R.S. §§41-2951 through 41-2957.
The Legislature established the State Board for Vocational and Technological Education
(Board) in A.R.S. §15-781.01 to “exercise general supervision over and regulate the conduct of
vocational and technological education in the public school system.” Arizona supports a vo-cational
education system that provides training in 47 occupational areas to approximately
102,000 high school students each year. Examples of these programs include Drafting
Technology, Food Production/Culinary Arts, Agriculture Business Management, and
Building Trades. Spending for these programs totals more than $75 million each year, with
nearly 40 percent distributed by the Board in state and federal aid. Besides approving the
distribution of this money, the Board also approves policies that impact the local operation of
vocational education. The Board’s membership is the same as the State Board of Education’s
and consists of eight members appointed by the Governor and the elected Superintendent of
Public Instruction. The Board delegates day-to-day oversight of vocational education pro-grams
to the Arizona Department of Education’s (ADE) School-to-Work Division.
The Board’s Limited Oversight
of the Vocational Education
System Leaves Important
Issues Unaddressed
(See pages 7 to 12)
Arizona’s Board has not adequately addressed several issues affecting Arizona’s vocational
education system. Effective state-level oversight is needed to ensure program graduates have
the skills needed by Arizona employers and to set state-level policy regarding workforce de-velopment.
Effective oversight of policy implementation is also needed to ensure state mon-ies
are allocated to school districts in a way that recognizes progress toward state goals for
vocational education. However, the Board has spent little time addressing these issues. For
example, while ADE’s staff have developed several ways of involving Arizona businesses in
vocational education, the Board has not pursued partnerships with Arizona employers to
determine how vocational programs could better meet their needs, and it has not actively en-couraged
school districts to implement Board-approved Workplace and Technology Skill
standards. Further, it has not addressed problems in implementing recent state funding
changes that would tie funding more closely to programs’ success in preparing students for
work (as shown by job placement rates). Nor has it reexamined aspects of the state funding
formula that may actually provide less funding for those programs that prepare students for
higher-wage jobs.
ii
To oversee the vocational education system more adequately, the Board needs to devote
more time to vocational education. In at least half of the combined meetings of the Board and
the State Board of Education held in 1996 and 1997, no substantive vocational education top-ics
were discussed. To assist the Board in addressing vocational education issues, it should
consider taking steps to revitalize its Vocational and Technological Education Advisory
Committee. This 19-member committee, which includes business and industry representa-tives,
can provide recommendations to the Board as well as ensure Arizona employers have
a voice in state education policymaking. However, the Committee has not met since August
1996. Members expressed frustration with the lack of discussion of policy issues, little contact
with the Board, and unclear expectations for their participation. To better link the Committee
to the Board, the Board should also consider having one of its members serve on the Com-mittee.
Current Monitoring Practices
Inadequate to Ensure Vocational
Education Program Quality
(See pages 13 to 18)
To carry out its statutory oversight responsibility, the Board needs accurate information
about vocational education programs’ performance. The Board relies on ADE for ensuring
vocational education programs are annually evaluated. However, ADE lacks an adequate
system for monitoring vocational education program outcomes. Although both state and
federal law require ADE to collect information about school district program outcomes, sev-eral
problems limit the usefulness of this information in assessing program quality, despite
the amount of information collected. Although one study estimates that 20,000 pieces of in-formation
must be reported for each program evaluated, little feedback is provided to school
districts based on the evaluation results. In addition, one standard is set so high that half of
all programs, including some model programs, fail. Thus, the standards may hinder ADE’s
ability to distinguish between quality and struggling programs. Moreover, a flawed infor-mation
system has limited usefulness for tracking program outcomes, and ADE’s paperwork
files are often incomplete and are only retained for one year. Due to these problems, ADE
cannot easily identify programs needing help to improve and target its technical assistance
efforts.
Two factors further hinder ADE’s ability to effectively monitor program performance and
assist struggling programs to improve. Turnover in key staff and management positions has
negatively impacted ADE’s long-term oversight of vocational education programs. In fact,
since 1992, ADE has had five different State Directors for Vocational Education. Further,
since 1996, ADE has experienced turnover in four of its six staff responsible for monitoring
program outcomes. Additionally, ADE management has not emphasized the need for units
within the School-to-Work Division to work together or to ensure that school districts im-
iii
plement program improvements. To help address these issues, ADE formed a Process Im-provement
team in August 1997 to recommend improvements to its monitoring system. To
date, the team has implemented several changes, such as increased training for staff on the
evaluation system, increased technical assistance and training for school district officials, and
revisions to ADE’s internal processing of evaluation information.
iv
(This Page Intentionally Left Blank)
v
TABLE OF CONTENTS
Page
Introduction and Background ......................................................... 1
Finding I: The Board’s Limited Oversight
of the Vocational Education System Leaves
Important Issues Unaddressed.................................................. 7
Key Statewide Issues Receive
Insufficient Attention
from Board.............................................................................................................. 7
Board Does Not Effectively
Use Advisory Committee...................................................................................... 10
Recommendations.................................................................................................. 12
Finding II: Current Monitoring Practices
Inadequate to Ensure Vocational
Education Program Quality......................................................... 13
Current Monitoring System.................................................................................. 13
Program Quality Not
Assured by Current
Monitoring Practices.............................................................................................. 14
Staffing and Philosophical
Changes Impede Long-
Term Monitoring.................................................................................................... 17
ADE Beginning to Improve
Monitoring System................................................................................................. 17
Recommendations.................................................................................................. 18
Sunset Factors................................................................................... 19
Agency Response
vi
TABLE OF CONTENTS (concl’d)
Page
Table
Table 1: Arizona Department of Education
School-to-Work Division
Statement of Revenues, Expenditures, and Reversions
Years Ended or Ending June 30, 1996, 1997, and 1998
(Unaudited).......................................................................................... 4
1
INTRODUCTION AND BACKGROUND
The Office of the Auditor General has conducted a performance audit and Sunset review of
the Arizona State Board for Vocational and Technological Education, pursuant to a May 27,
1997, resolution of the Joint Legislative Audit Committee. This audit was conducted as part
of the Sunset review as set forth in A.R.S. §§41-2951 through 41-2957.
A State Board for Vocational and Technological Education (Board) has existed in Arizona for
over 80 years, providing policy oversight for vocational programs. State law, in A.R.S. §15-
781.01, requires the Board to “Exercise general supervision over and regulate the conduct of
vocational and technological education in the public school system.” The Board approves the
spending of state monies, reviews policies that impact the local operation of vocational edu-cation,
and delegates to the Superintendent of Public Instruction the execution of its policies.
Further, the Board is the sole state agency responsible for receiving federal vocational educa-tion
monies available under the Carl D. Perkins Vocational and Applied Technology Educa-tion
Act of 1990, which requires states to have a board of vocational education in order to re-ceive
federal monies.
Vocational Education
in Arizona
Vocational education programs prepare students for an increasingly demanding labor mar-ket
through courses that train them for jobs not requiring a bachelor’s degree. Arizona’s 1,300
programs provide training in 47 occupational areas to approximately 102,000 high school
students in over 100 school districts. Examples of programs include Sales and Marketing,
Food Production/Culinary Arts, Agriculture Business Management, and Building Trades.
School districts are encouraged to implement a comprehensive model of vocational educa-tion.
The Arizona Model for Vocational Technological Education, developed by educators,
state agency staff, and business and industry representatives, incorporates skill development,
sequenced levels of instruction, and specific curricula to expand student skills in problem
solving and leadership, academics, life management, technology, and career development.
Vocational education can provide valuable occupational training for students who seek em-ployment
directly after high school. However, many students who do not continue their
education after high school receive little or no occupational training. Only half of all Arizona
high school graduates begin college and only about half of these students return for their
second semester. Yet, while over 50 percent of all high school students take a vocational edu-cation
course each year, only 26 percent actually complete vocational education programs (a
sequence of courses in a given occupational area). Thus, many students lack adequate train-ing
to obtain high-demand, high-wage jobs after high school.
2
Board Membership
Under A.R.S. §15-781.01, the State Board of Education has also been designated as the State
Board for Vocational and Technological Education. The Board’s nine members include eight
members appointed by the Governor for four-year terms and the elected Superintendent of
Public Instruction. By statute, the appointed members must include the president of a state
university or state college, a member of the State Board of Directors for Community Colleges,
a superintendent of a high school district, a classroom teacher, a county school superinten-dent,
and three lay members. In 1992, the Board formally established a 19-member Voca-tional
and Technological Education Advisory Committee to assist it in its oversight responsi-bilities.
The Committee includes business and industry representatives, vocational education
administrators and teachers, and post-secondary education officials.
Staff and Funding
The State Board of Education, also acting as the State Board for Vocational and Technological
Education, employs two full-time employees who oversee its administrative operations. The
Legislature appropriated $147,500 in fiscal year 1997-98 for these positions and other Board
operating costs.
As the body charged with coordinating vocational education at the state level, and therefore
with allocating state and federal aid, the Board controls nearly 40 percent of vocational edu-cation
monies. Arizona’s vocational education funding comes from the following sources:
n State-designated vocational education monies (14 percent of total)—In fiscal year
1997-98, the Legislature appropriated over $11 million to the Vocational Education Block
Grant. School districts receive state Block Grant monies based on the number of students
enrolled in approved vocational education programs and graduates placed in related
employment, education, or the military.
n Federal monies (25 percent of total)—Arizona received $18.8 million in federal Carl D.
Perkins monies in fiscal year 1997-98. According to A.R.S. §15-784(E), 15 percent (ap-proximately
$2.8 million for fiscal year 1997-98) of federal monies must be passed through
to the community college system. School districts receive the remaining monies based on
a formula that considers school districts’ academically at-risk student populations.
n Locally designated monies for vocational education (61 percent of total)—School
districts also devote substantial additional monies from their general operating budgets
to vocational education. In fiscal year 1996-97 (the most recent figures available), school
districts expended $47.1 million for vocational education out of their general operating
budgets.
3
The Board delegates policy implementation to the Superintendent of Public Instruction. The
Superintendent assigns day-to-day oversight of vocational education programs to the Ari-zona
Department of Education’s (ADE) School-to-Work Division (Division). The Division is
composed of 34 full-time equivalent state-funded staff, including administrative, information
systems, accounting, and clerical personnel, as well as 14 full-time equivalent federally
funded vocational program staff. The Division includes four program areas.
n The Career Pathways Team—provides assistance in each of six major occupational ar-eas
(Agriculture, Business, Family and Consumer Sciences, Health Occupations, Indus-trial
Technological, and Marketing) encompassing all 47 vocational education programs.
It also provides support for vocational student organizations, and the development of
program curricula.
n The Federal Vocational Education Team—ensures that school districts meet the provi-sions
of the federal Carl D. Perkins Act if they are receiving monies from that program.
n The Federal Job Training Team—coordinates ADE’s involvement in other federal job
training programs, such as the Job Training Partnership Act.
n The Grants and MIS Services Team—provides financial and data support for the Divi-sion.
Table 1 (see page 4) depicts the total revenues and expenditures for ADE’s School-to-Work
Division, which includes all federal and state monies for vocational education programs.
Scope and Methodology
Audit work focused on the Board’s ability to fulfill its statutory role in overseeing vocational
education. Since the Board delegates specific oversight duties to ADE’s School-to-Work Divi-sion,
the audit also focused on the Division's ability to monitor and provide technical assis-tance
to vocational education programs. Several methods were used to study the issues ad-dressed
in this audit, including:
n Interviewing seven current and three former Board members (including the last remain-ing
business representative on the Board, whose term expired in January 1998), eight of
the Board’s Vocational Education Advisory Committee members, ADE and School-to-
Work Division staff, and other private sector business representatives;
4
Table 1
Arizona Department of Education
School-to-Work Division
Statement of Revenues, Expenditures, and Reversions1
Years Ended or Ending June 30, 1996, 1997, and 1998
(Unaudited)
1996
(Actual)
1997
(Actual)
1998
(Estimated)
Revenues:
State General Fund appropriations 2 $ 7,191,400 $ 7,202,400 $11,045,300
Federal grants and reimbursements 3 19,193,327 20,202,671 21,476,968
Other              1,275       
Total revenues 26,384,727 27,406,346 32,522,268
Expenditures:
Personal services 1,285,623 1,516,246 1,970,924
Employee related 331,026 332,995 445,208
Professional and outside services 47,798 150,367 287,579
Travel, in-state 73,423 82,276 66,477
Travel, out-of-state 18,932 30,780 36,034
Aid to organizations and individuals 4 24,043,609 24,821,270 28,818,689
Other operating 361,309 510,929 385,196
Capital outlay 63,816 134,105 259,645
Cost allocation 5 78,100 101,015 252,814
Total expenditures 26,303,636 27,679,983 32,522,566
Reversions to the State General Fund 7,603 13,671
Total expenditures and reversions to the
State General Fund 26,311,239 27,693,654 32,522,566
Excess of revenues over (under) expenditures
and reversions to the State General Fund $ 73,488 $ (287,308) $ (298)
1 Fund balances are maintained at the Department level; therefore, only revenues, expenditures, and reversions to the State
General Fund are available at the Division.
2 The amount shown for the 1996 appropriation is the combination of three separate appropriation lines for vocational educa-tion.
Beginning in fiscal year 1997, vocational education cost centers were consolidated into one block grant appropriation.
During fiscal year 1998, the Division received an additional $3.6 million appropriation from a transfer of vocational educa-tion-
related Basic State Aid monies.
3 Amount includes immaterial revenues from prior years.
4 Aid to organizations and individuals consists primarily of monies provided to other entities for vocational education pro-grams.
Most of the support goes to school districts; however, $2,835,555, $3,519,957, and $3,430,454 for fiscal years 1996, 1997,
and 1998, respectively, passed through the Division to other state agencies such as the State Board of Directors for Commu-nity
Colleges.
5 Using a federally approved indirect cost rate, a portion of administrative costs such as administrative staff support, tele-phones,
rent, postage, and insurance is charged against all Department funds except the General Fund. This rate is applied
against applicable programs’ personal services and employee-related expenditures. The rates for fiscal years 1996, 1997, and
1998 were 12.9 percent, 12.1 percent, and 14.9 percent, respectively.
Source: The Uniform Statewide Accounting System Detail Grant Report for the years ended June 30, 1996, and 1997; and the State
of Arizona Appropriations Report for the years ended or ending June 30, 1996, 1997, and 1998. Expenditures for the year
ended June 30, 1998, are an estimate of the Arizona Department of Education.
5
n Collecting and analyzing school district vocational education program performance data
such as the number of students completing vocational education courses and their gains
in academic and occupational skills for fiscal years 1994-1996;1
n Conducting ten site visits of secondary school vocational education programs and inter-viewing
school district officials, program administrators, and vocational education teach-ers;
n Surveying ten other states concerning their oversight of vocational education, efforts to
involve the private sector, and methods of implementing the local evaluation provisions
of federal law;2
n Reviewing Board meeting minutes from 1992-1997, and meeting minutes of the Board’s
Vocational Education Advisory Committee for 1995-1996; and
n Conducting a review of literature concerning vocational education performance meas-ures,
skills needed by high school graduates, business and industry involvement in state-level
policy development, and appropriate oversight of vocational education programs.
This report presents findings and recommendations in two areas:
n The Board needs to provide increased attention to vocational education programs and
address key issues; and
n ADE needs to take steps to improve its evaluation system in order to provide better over-sight
of vocational education programs.
This audit was conducted in accordance with government auditing standards.
The Auditor General and staff express appreciation to the Board members and ADE and
district staff for their cooperation and assistance throughout the audit.
1 Program performance data maintained by ADE from 1,167 programs in 1994 (the first year data was avail-able),
1,187 in 1995, and 1,310 in 1996 was analyzed. Due to program changes at the school district level, data
from only 695 programs could be tracked across all three years.
2 States selected were identified as national leaders in vocational education or in creating effective relation-ships
with the private sector. The following states were contacted: Colorado, Maryland, North Carolina,
Ohio, Oklahoma, Oregon, South Carolina, Texas, Washington, and Wisconsin.
6
(This Page Intentionally Left Blank)
7
FINDING I
THE BOARD’S LIMITED OVERSIGHT OF THE
VOCATIONAL EDUCATION SYSTEM LEAVES
IMPORTANT ISSUES UNADDRESSED
Currently, the Board has not adequately addressed several issues impacting the vocational
education system in Arizona. Effective oversight at the state level is important for ensuring
that the State’s vocational education programs meet employers’ expectations, and teach criti-cal
skills. However, the Board is not providing this oversight, nor is it actively moving to tie
funding more closely to program outcomes or to coordinate its activities with similar educa-tion
efforts underway within the State. The State Board of Education, also acting as the State
Board for Vocational and Technological Education, has recently focused its efforts on other
topics. To help it address these issues, the Board should reactivate its currently inactive Advi-sory
Committee and seek its assistance.
Key Statewide Issues Receive
Insufficient Attention
from Board
As the oversight body for vocational education in Arizona, the Board is responsible for state-level
policy direction for ADE and school districts. Yet several issues, including employers’
concerns with vocational education, funding changes, and possible program duplication, are
not receiving the necessary Board attention. In fact, the Board spends little time discussing
vocational education at its monthly meetings.
Board’s direction for vocational education system is important—State-level coordination of
the vocational education system is an important part of ensuring that high school vocational
education program graduates have the skills necessary for employment. The Board, charged
by statute with supervising vocational and technological education in public schools, has
many ways in which it can provide this guidance. For example, it can seek input from Ari-zona
employers regarding their expectations of vocational education programs. The Board
also plays a central role in determining how best to fund programs, especially when limited
resources must be distributed among many school districts, and in coordinating vocational
education programs with other, similar education programs.
Board neglects important issues—However, the Board has not resolved many of the impor-tant
issues it needs to face in providing oversight and direction. Interviews with business and
industry representatives, vocational education teachers and administrators, and ADE staff
8
identified several issues requiring the Board’s involvement. The Board should provide direc-tion
in these areas in order to fulfill its state and federal mandate to provide oversight for vo-cational
education.
n Arizona employers’ needs—The Board has not established mechanisms to identify em-ployers’
expectations of high school graduates. Since vocational programs are designed to
prepare students for jobs, business and industry can be seen as the ultimate customers of
vocational education programs. Education literature strongly supports the need for edu-cators
to partner with business and industry in the oversight of vocational education.
However, Arizona’s Board has not actively pursued such partnerships.
Employers’ opinions about the current vocational education program illustrate the need
for these partnerships. Some business representatives reported that many graduates lack
the necessary skills for jobs because schools are not necessarily teaching the skills that
students need. They believe education policymakers have devoted significant time to
identifying which skills students need to go on to college, but not those they need to ob-tain
and keep a well-paying job. Numerous people interviewed during the audit, in-cluding
Board members and business and industry representatives, believe the relation-ship
between the educational and private sectors could be improved in Arizona. The
Board needs to seek the help of Arizona employers in determining what priority voca-tional
education should have in the secondary education curriculum and the skills voca-tional
education programs should teach.
n Recent state funding changes—The Board has not ensured complete implementation
of a new method of funding vocational education that ties funding to program outcomes.
In 1996, the Board recommended distributing 20 percent of the state vocational education
block grant monies to school districts based on the number of each school district’s stu-dents
placed in related employment, education, or military service after graduation. This
percentage would be increased 3 percent each year until placement makes up 35 percent
of total state block grant funding by 2003. Thus, school districts who more adequately
prepare students for work, the goal of vocational education, should receive more funding.
However, the Legislature approved distributing only 5 percent of the state monies based
on placement for fiscal year 1997-98 and 10 percent for 1998-99. Staff of the Joint Legisla-tive
Budget Committee recommended reduced percentages because available placement
rate data was incomplete, in order to give school districts an additional year to develop
systems to collect this data. The Board needs to revisit its funding recommendations and
identify ways to help school districts obtain the necessary placement data so that funding
can be more closely tied to program outcomes.
The Board also has not used recent data to determine if its current formula for distribut-ing
the remaining state monies should be revised. The Board funds vocational education
programs based on a ranking of all programs: programs with a higher ranking receive
more money per student than lower priority programs. ADE ranks each of its vocational
9
programs based on a Board-approved formula that considers the number of jobs avail-able
and the average wages earned in Arizona. However, partly because reliable wage
data was not available when the formula was approved, the formula considers demand
for a job to be three times as important as its wages. While demand is important to con-sider,
the current formula may result in giving a disproportionate share of effect on the
money, limiting money for some programs that train students for higher-paying jobs. For
example, media and graphic communication programs, which train students for jobs
paying over $10 per hour, receive less money than child care or food production pro-grams
for which demand is high, but pay is approximately $7 and $6.50 per hour, re-spectively.
Better data on wages is currently available; thus, the Board should reexamine
the relative importance of job demand and wages and determine if a new formula is war-ranted.
n Coordination with the Governor’s Division of School-to-Work—The Board has yet to
determine how another agency’s similar program impacts vocational education and what
coordination between the two programs is necessary. The Governor’s Division of School-to-
Work in the Department of Commerce administers a federally funded program de-signed
to improve the transition from school to work for all students. The Governor’s
Council on Workforce Development Policy provides oversight to this program. One
Board member serves on the Council.
While similar in mission to vocational education, little coordination exists between the
two programs. In fact, despite having a Board member serving on the Governor’s Coun-cil,
the Board last formally discussed this program in May 1995. Some private sector rep-resentatives
and education officials expressed confusion about the difference between this
program and ADE’s vocational education program, partly due to both units having the
same name. Employers wishing to get involved in the education system are not sure
where to direct their efforts. Additionally, members of the Governor’s Council stated that
alignment of policies, messages, and actions between the two programs is essential to im-prove
the transition from school to work.1 Further, as a result of its Program Authoriza-tion
Review of ADE’s School-to-Work program, the staff of the Joint Legislative Budget
Committee recommends in its Proposed Budget Fiscal Year 1999 Analysis and Recommenda-tions
requiring ADE to report to the Legislature after the 1998 regular session regarding
possible consolidation with related programs in other agencies, such as this program.
Thus, the Board needs to identify areas of duplication with Commerce’s program, deter-mine
if consolidation is necessary, and participate in the development of a consistent
statewide workforce development policy.
n Education initiatives—The Board has provided insufficient direction on its most recent
education initiative. In March 1997, the Board passed Workplace and Technology skill
standards as two of eight sets of standards to be met by all public school students. These
standards encompass skills such as oral and written communication, data analysis,
1 Vandegrift, J.A. “Conversations with Arizona’s School to Work Advisory Council,” Morrison Institute for
Public Policy. July 1996.
10
teamwork, and the use of technology. For instance, prior to graduation a student should
demonstrate the ability to work with others from diverse backgrounds and select and use
appropriate technology to organize and send information. However, because it is not spe-cifically
required by state law, the Board does not currently plan to assess students’ per-formance
on these standards and has not provided direction as to how school districts
should implement them. Further, although statute requires school districts to submit
plans for implementation of all Board-approved standards, ADE has initially excluded
the standards that state law does not require to be assessed, including the Workplace and
Technology standards, from this requirement. Several private sector representatives ex-pressed
frustration with the lack of implementation efforts by the Board and ADE. In fact,
the Phoenix Chamber of Commerce took the initiative and began working with Phoenix
school districts to incorporate the Workplace skill standards into their curricula. The
Board should develop ways to encourage school districts to encompass these standards
into their curricula.
n Board rarely discusses vocational education—The lack of Board action in these areas
results from Arizona’s Board spending little time discussing vocational education. The
Board reduced both the time spent on vocational education issues as well as the number
of issues discussed during Board meetings in recent years. In 1990, the Legislature ex-panded
the Board to include four additional members representing business and indus-try.
At that time, the Board met as a separate Board for Vocational and Technological
Education for approximately an hour prior to the all-day meeting of the State Board of
Education. A review of Board meeting minutes revealed that more time was spent on vo-cational
education issues and considerably more issues were discussed when these
members were first appointed.
After the Board’s composition was again changed, the emphasis placed on vocational
education declined. In 1995, the Legislature returned the Board to its original member-ship,
the same as the State Board of Education’s. In 1996, the Board ceased meeting sepa-rately
as the Board for Vocational and Technological Education and instead combined all
meetings with the State Board of Education. Thus, vocational education issues compete
with all other education issues for time on the meeting agenda. In fact, in at least half of
the meetings held in 1996 and 1997, no substantive vocational education topics were dis-cussed.
Instead, most of the combined Board’s time is spent on teacher certification cases
and charter schools.
Board Does Not Effectively
Use Advisory Committee
Because the Board lacks the time to address vocational education issues, an advisory body
could serve a valuable role. Although the Board has established an advisory committee, it
has not met for over a year and a half. To help carry out its statutory responsibilities regard-ing
vocational education, the Board should take steps to reactivate its advisory committee.
11
Advisory Committee currently inactive—While a separate body in Arizona provided assis-tance
to the Board in the past, it has not met since 1996. A 1992 Board policy formally estab-lished
the Vocational and Technological Education Advisory Committee to make recom-mendations
regarding programmatic issues, financial matters, and improvements in voca-tional
education. The 19-member Committee can provide valuable business and industry in-put
since 10 members must come from the private sector and represent the various occupa-tional
areas taught, such as marketing, health occupations, and agriculture. As the Board no
longer includes specific members from business and industry, the Committee represents an
important opportunity for Board members to involve these stakeholders. In the past, the
Committee encouraged businesses to work with local programs, advocated for increased fi-nancial
support for vocational education, and helped develop recommendations for distrib-uting
state monies. However, while the Committee still exists, it has not met since August
1996.
Interviews with Advisory Committee members and others suggest reasons for the Commit-tee’s
current inactivity. Some Committee members expressed concerns that meetings held in
the last few years frequently included discussions of administrative issues more appropri-ately
decided by ADE versus policy issues affecting vocational education. A review of the
minutes from the five meetings held in 1995 and 1996 revealed that the Committee made few
decisions and rarely discussed the future of vocational education. The Committee also
maintained little contact with the Board. One member stated that even when the Committee
did meet, there was no contact with Board members. While a study session with the Board
was requested, it was never scheduled. Further, interviews with current Board members re-vealed
that several knew very little about the Committee. In addition, Board and ADE ex-pectations
for the Committee may not have been well communicated. One member felt that
the Committee never had a sense of what the expectations were or what they should be do-ing.
Others suggested that ADE needed to make a renewed commitment for the Committee’s
involvement before it would be useful to meet again.
Renewed commitment from Board could revive Committee—The Board and ADE will need
to revise the Advisory Committee’s practices in order for it to effectively provide assistance
to the Board. One study suggests advisory committees must meet six criteria to be effective.1
The Board’s Committee currently meets only one of the six criteria fully: that its membership
is composed of people outside the organization. The Board needs to ensure the Committee
can meet the remaining five criteria:
n The Committee meets regularly on a long-term basis;
n The Committee’s input is genuinely desired;
1 Teitel, Lee. “The Advisory Committee Advantage: Creating an Effective Strategy for Programmatic Im-provement.”
Report published by the George Washington University School of Education and Human De-velopment.
1994.
12
n The Committee is comprised of knowledgeable, committed individuals whose interest in
volunteering is sustained by appropriate recognition and rewards;
n The Committee’s processes allow for a sense of engagement and ownership; and
n The expectations for the Committee are clear, consistent, and well communicated.
The Board should consider reconvening its Advisory Committee to provide recommenda-tions
for the current issues facing vocational education. In October 1997, the Board appointed
ten Committee members to new terms ending in 2001. Committee members interviewed ex-pressed
their continued desire to study important issues and provide their opinions. Mem-bers
also hope to improve the link between the private sector and the public education sys-tem.
To better link the Committee to the Board, the Board should consider having a member
also serve on the Committee. In the past, a Board member served as Chair of the Committee.
Recommendations
1. The Board needs to:
a. Involve Arizona employers in policy discussions and ensure programs meet their ex-pectations;
b. Review its current method of funding programs and determine if a new funding for-mula
is warranted;
c. Coordinate with the Governor’s Council for Workforce Development Policy regard-ing
the State’s efforts to prepare students for work; and
d. Assist school districts in implementing its Workplace and Technology Skills stan-dards.
2. The Board should consider reactivating its Advisory Committee to help address these
vocational education issues. Specifically, the Board should:
a. Communicate its expectations to the Committee and identify the issues the Commit-tee
should address;
b. Ensure the Committee meets regularly;
c. Review Committee recommendations for possible implementation; and
d. Consider appointing a Board member to serve on the Committee.
13
FINDING II
CURRENT MONITORING PRACTICES INADEQUATE
TO ENSURE VOCATIONAL EDUCATION
PROGRAM QUALITY
To carry out its oversight responsibility, the Board needs accurate information about voca-tional
education programs’ performance. The Board relies on ADE for ensuring vocational
education programs are annually evaluated, but ADE does not have an adequate system for
monitoring vocational education program outcomes. Although ADE collects information on
local school districts’ program outcomes, several problems, including misleading perform-ance
measures and a flawed standards data system, limit the usefulness of this information.
As a result, ADE cannot easily identify struggling vocational education programs and target
assistance to help them improve. Further, changes in management and staff have influenced
ADE’s ability to effectively evaluate programs and promote improvement.
Current Monitoring System
Both state and federal law require evaluation of vocational programs as part of the programs’
continued eligibility for state and federal funding. At the federal level, the Carl D. Perkins
Vocational and Applied Technology Education Act of 1990 requires states to annually evalu-ate
all vocational education programs that receive federal funding. Specifically, states must
evaluate programs based on students’ academic gain and at least one of several other stan-dards.
At the state level, A.R.S. §15-783 also requires annual evaluations of vocational educa-tion
programs. The Board, which is responsible for determining Arizona’s standards, applies
the same standards to all vocational education programs, regardless of their source of
money. The Board, upon the recommendations of a Committee of Practitioners1, has for-mally
adopted four performance standards:
n Increases in students’ academic skills;
n Students’ mastery of specific occupational skills within each course;
n Students’ mastery of specific occupational skills across a program of three courses; and
n Placement of program graduates into employment, continuing education, or military
service.
1 Federal law requires states to use a Committee of Practitioners comprised of local school personnel, business
representatives, parents, and others to recommend performance standards to the Board.
14
As required by Board procedures, information on programs’ progress in meeting these stan-dards
is gathered by a local evaluation team, formed of volunteer business representatives,
vocational teachers, and a representative of student special populations. The local team sub-mits
the results of its work to ADE, where School-to-Work Division staff determine the pro-grams’
eligibility for state and federal monies.
Program Quality Not
Assured by Current
Monitoring Practices
While the quality of vocational education programs varies, ADE’s current monitoring system
is of little use in identifying which programs need help. First, Arizona’s performance stan-dards
may actually impede ADE’s ability to gather useful program outcome information.
Second, although ADE maintains a database and paperwork files on local evaluation infor-mation,
the system has limited usefulness for tracking program performance and targeting
technical assistance because it does not allow ADE to readily distinguish between quality
programs and those that need help to improve. Finally, ADE can do more to provide techni-cal
assistance to struggling programs and ensure they improve.
Vocational education programs vary in the quality of training provided—Vocational educa-tion
programs across the State differ in the training provided. During the audit, ADE per-sonnel
could identify a few vocational education programs that are considered model pro-grams
as well as those needing improvement in one or more areas. Programs may lack ade-quate
equipment, a properly certified teacher, or the ability to provide hands-on experience,
or they may not ensure students learn sufficient academic skills. The presence of such prob-lems
demonstrates the need to be able to identify all struggling programs and provide re-sources
to help them. However, ADE’s monitoring system cannot identify programs as being
in particular need. The monitoring system’s effectiveness is hindered by problems with the
standards against which programs are measured and with shortcomings in the information
compiled in ADE’s database.
Performance standards may actually complicate measurement—By adopting standards that
require school districts to collect extensive information, and by setting one of the standards
unreasonably high, ADE may reduce the accuracy and usefulness of the information it col-lects.
Arizona’s requirement that local evaluation teams annually report program progress
on four standards exceeds federal legislation that requires measurement of only two stan-dards:
students’ academic gain and another standard chosen by the state. According to sev-eral
school district officials, gathering the necessary information for evaluating programs
based on four standards is a paperwork burden given the limited feedback ADE provides
based on the evaluation results. One Morrison Institute for Public Policy report found that
vocational education program officials must gather approximately 20,000 individual pieces
15
of information for each program evaluated.1 School districts must gather extensive informa-tion
on all students in vocational education classes, including academic gain test scores, en-rollment
numbers, and information that allows school districts to track students’ placement
in employment, further education, or the military after graduation. In fact, due to the compli-cated
nature of reporting requirements, several smaller school districts contract with regional
consultants to help them with this paperwork burden. The consultants also provide technical
assistance and software to document local evaluation information.
In addition, Arizona’s academic gain standard is set so high that it obscures ADE’s ability to
obtain useful information. Arizona’s academic gain standard requires 100 percent of students
in a vocational education course to learn basic and advanced academic skills, measured by
performance on tests administered at the start and end of the school year. However, students
may not show academic gain for several reasons, including failure to take the tests seriously
because they may not impact students’ grades or transferring to another school at year-end
before taking the final test. Even one student’s performance on such tests will cause a voca-tional
program to fail the academic gain standard. In fact, many programs, including some
programs considered model programs, fail this standard. Auditor General staff analysis of
ADE’s computer standards database revealed that 74, 53, and 49 percent of all programs
failed the academic gain standard during fiscal years 1994, 1995, and 1996, respectively. Of
the 695 programs operating from 1994 through 1996, 37 percent did not meet the academic
gain standard in any of the three fiscal years. Because so many programs fail the standard,
ADE cannot use it effectively in targeting technical assistance. In contrast, setting the stan-dard
at a lower level and then increasing it as programs improve could allow better identifi-cation
of where assistance is needed and provide information on trends in program perform-ance.
Standards data system insufficient to indicate program quality and track performance—The
problems with the performance standards are further compounded by problems with the
database ADE uses to record program performance. First, the database captures only limited
information reported from school districts’ local evaluations. Although school districts are
required to report the percentage of students demonstrating academic gain and occupational
skill mastery, the database records only whether or not the program met the performance
standards. For example, a district where only 25 percent of students demonstrated academic
gain cannot be distinguished from one where 99 percent demonstrated gain. Also, the data-base
records that a program failed a standard if no information was submitted.
Second, ADE has not established a consistent format for the contents of the standards data-base.
Therefore, the database does not allow ADE staff to track programs’ progress over time
or easily generate useful reports. The database was developed with little emphasis on infor-mation
continuity, report generation, or program tracking. As a result, programs cannot eas-ily
be tracked without controlling for changes in program identification codes or definitions
1 Vandegrift, J.A, et al. “Improving Quality and Accountability in Vocational Technological Programs: An
Evaluation of Arizona’s VTE Model and Performance Standards.” Morrison Institute for Public Policy, Oc-tober
1994.
16
of the performance standards themselves. Since ADE retains local evaluation paperwork at
the Department for just one year, the database is the only long-term source for information
about vocational education program performance. Finally, even the paperwork files that are
available are incomplete, severely limiting ADE staff’s ability to obtain useful information
and track program performance over time. Auditor General staff attempted to review pa-perwork
submitted by school districts for 17 culinary arts and child care and guidance pro-grams.
Auditor General staff attempted to determine how these programs performed on the
academic gain standard in relation to model programs. However, 12 of these 17 program
files reviewed did not contain complete paperwork, which ADE policies require school dis-tricts
to submit.
ADE does not adequately target technical assistance—In part because of their inability to
identify struggling programs using the database or paperwork files, ADE staff do not proac-tively
target technical assistance to school districts that may need it most. While ADE staff
provide assistance to vocational programs that they discover are experiencing problems
through site visits, or which request help, this may not be a sufficient way to determine
which programs are truly struggling and where to target technical assistance. Programs’
long-term performance may better indicate which programs should be targeted for technical
assistance. Federal legislation requires ADE to provide technical assistance to school districts
by helping them develop improvement plans. A school district must write a local improve-ment
plan specifying corrective measures the first year a program fails to meet one or more
of the four performance standards. In subsequent years, a joint improvement plan must be
written in consultation with ADE staff, until deficiencies are corrected and a program meets
all standards.
However, ADE staff do not help school districts develop joint improvement plans unless the
district requests assistance. In fact, since ADE does not track evaluation results it cannot de-termine
whether programs are consistently failing standards and operating under either local
or joint improvement plans. Auditor General staff analysis of performance standards data
indicates that approximately 57 percent of all vocational education programs operated under
improvement plans for fiscal years 1994 through 1996. Although the Arizona Vocational
Education State Plan states that programs operating under improvement plans for five years
will become ineligible for funding, ADE officials do not plan to enforce this penalty since
they cannot currently track this information and they believe a better approach is to focus on
program improvement rather than program closure. Further, many of the improvement
plans reviewed for the audit did not adequately specify corrective measures and ADE staff
do not systematically follow up to ensure school districts implement needed improvements.
ADE’s State Director for Vocational Education believes the Division should do more to effec-tively
target technical assistance, help school districts with their improvement plans, and en-sure
that school districts implement needed program improvements.
17
Staffing and Philosophical
Changes Impede Long-
Term Monitoring
Changes in School-to-Work Division staffing and management philosophy have affected the
oversight of vocational education programs. Since 1992, ADE has had five different State Di-rectors
for Vocational Education, who have placed varying emphasis on monitoring voca-tional
education program performance. In addition, turnover in key Division staff positions
may also negatively impact the monitoring of vocational education program performance.
Since 1996, the Division has experienced turnover in four of its six state supervisor positions.
State supervisors are responsible for monitoring vocational education program performance
in each of Arizona’s six vocational education program areas, such as business education or
industrial technological programs. They must develop an extensive knowledge of programs
in their areas in order to track programs’ performance, provide technical assistance, and en-sure
that programs improve. However, due to the limited database information and incom-plete
paperwork files, these staff must learn about programs by visiting sites and meeting
with district program staff. Therefore, turnover in these positions significantly reduces this
knowledge base.
Further, management has not historically emphasized the need for units within the Division
to work together, to share information, or to ensure that school districts implement program
improvements. According to the current State Director, ADE’s performance monitoring sys-tem
is now focusing more on improving programs and providing adequate resources to
support improvement efforts, both of which were missing in the past.
ADE Beginning to Improve
Monitoring System
To address program monitoring issues, ADE’s State Director for Vocational Education
formed the Performance Standards Process Improvement Team in August 1997. The team is
charged with evaluating, recommending, and implementing improvements to the monitor-ing
system. The State Director believes that a systematic evaluation and improvement of
ADE’s internal processes is necessary to adequately support program improvement at the
school district level. This team has already made and implemented several changes, such as
increased training for state supervisors, increased technical assistance and training for school
district officials, and revisions to ADE’s internal processing of evaluation information. Other
issues that the team has recognized, but not yet addressed, include increased use of ADE’s
database information, long-term tracking of program performance, and follow-up on school
district improvement plans. The Process Improvement Team can continue to provide valu-able
input to streamline and improve the monitoring system. In addition to the efforts al-ready
underway, it can review the appropriateness of the current standards and suggest
changes to the evaluation database.
18
Recommendations
1. The State Board for Vocational Technological Education, in coordination with ADE staff
and the Committee of Practitioners, should review the academic gain performance stan-dard
to make it more realistic and to facilitate distinction between quality and struggling
programs.
2. The Board should require ADE to revise its computer database to:
a. Record more specific measures of program performance, such as the actual percent-age
of students who demonstrate academic gain, instead of absolute compliance (i.e.,
“yes” or “no”);
b. Standardize definitions of performance standards and formatting program identifica-tion
codes across years; and
c. Generate reports that track programs’ performance over time.
3. The Board should require all school districts to submit all necessary information for their
local evaluations.
4. The Board should ensure that ADE increases the technical assistance it provides to school
districts, including:
a. Assisting in the development of written program improvement plans; and
b. Following up to ensure improvements are made.
19
SUNSET FACTORS
In accordance with Arizona Revised Statutes (A.R.S.) §41-2354, the Legislature should con-sider
the following 12 factors in determining whether the State Board for Vocational Tech-nological
Education should be continued or terminated.
1. The objective and purpose in establishing the Board.
A State Board for vocational education has been an element of Arizona education for
over 80 years. In 1917, the federal Smith Hughes Act, which provided federal monies
for vocational education, required states to have a vocational education board. Tradi-tionally,
the board functioned as part of the State Board of Education. However, in
1990, under A.R.S. §15-781.01, the Legislature established the State Board for Voca-tional
and Technological Education as a separate entity from the State Board of Edu-cation.
The Board was established to provide oversight for vocational and technologi-cal
education in Arizona’s public schools and to comply with the requirements for re-ceiving
federal monies. The legislative intent in establishing the Board was to encour-age
the development of model programs that would increase students’ opportunities
for vocational education in high-demand, high-wage jobs.
According to A.R.S. §15-781, vocational and technical preparation means “an organ-ized
set of specialized courses which is directly related to the preparation of persons
for occupations that normally do not require a baccalaureate or advanced de-gree…
and which is designed…to provide a pupil with sufficient skills for entry into
an occupation.” Programs concentrate on applied learning to promote specific job
skills, general employability, work attitude, reasoning, problem-solving skills, and
academic knowledge. Examples of programs available to Arizona students include
Administrative Support, Building Trades, Sales and Marketing, Culinary Arts, and
Health Occupations.
Currently, the Board provides policy oversight for vocational programs by reviewing
and approving discretionary spending of federal monies, changes in statewide priori-ties,
and other policies that impact local operation of vocational education as man-dated
by state and federal law. For example, the Board acts as the sole Arizona agency
responsible for federal monies available under the federal Carl D. Perkins Vocational
and Applied Technology Education Act of 1990. Day-to-day oversight of vocational
education programs is delegated to the Arizona Department of Education’s School-to-
Work Division, which is responsible for providing technical assistance, calculating
and distributing funding, and ensuring school districts annually evaluate their pro-grams.
20
2. The effectiveness with which the Board has met its objective and purpose and
the efficiency with which it has operated.
The Board has not effectively met its statutory objective to provide “general supervi-sion
over and regulate the conduct of vocational and technological education in the
public school system.” While the Board approves spending of federal and state mon-ies
for local programs, it has neither discussed nor provided direction on important
issues facing vocational education in Arizona. For example, while Arizona’s employ-ers
are key customers of vocational education, the Board has not actively identified
the concerns of these stakeholders. Additionally, the Board has not determined if its
new method of funding vocational education adequately ties funding to program
outcomes, as the Board intended. The Board also has not discussed the possible du-plication
between vocational education and a similar education program operated by
the Department of Commerce. Finally, the Board has not provided direction on how
school districts should implement its recently approved Workplace and Technology
Skill standards. To help it address these issues, the Board should consider reconven-ing
its Vocational Education Advisory Committee (see Finding I, pages 7 through 12).
3. The extent to which the Board has operated within the public interest.
The Board has operated in the public interest by ensuring that Arizona secondary
school students have access to vocational education programs. The Board ensures
state and federal monies dedicated to vocational education are distributed to ap-proved
district programs. Additionally, the Board directs a portion of federal monies
to the Arizona community college system for post-secondary vocational education as
mandated by state law.
However, the Board could do more to prepare students who directly enter the Ari-zona
workforce instead of going to college or joining the military. In the Superinten-dent
of Public Instruction’s 1996 Plan for Education in Arizona, she states, “Only a por-tion
of high school graduates desire to go on to college. Our current academic prepa-ration
means that those who do not [go on] are likely to enter the job market without
any meaningful workforce preparation.” For example, only half of all Arizona high
school graduates attend college, and half of those students are reported to drop out of
advanced education by their second semester. However, as mentioned in the intro-duction,
only 26 percent of high school graduates complete secondary vocational
education programs. The Board could seek to expand the vocational education op-portunities
for students who may not go on to college by promoting vocational edu-cation
among all students, including college-bound youth, encouraging school dis-tricts
to provide additional career counseling, working to increase the number of vo-cational
education classes qualifying for academic credit, and requiring that students
take more vocational education courses for graduation.
21
4. The extent to which rules and regulations promulgated by the Board are con-sistent
with the legislative mandate.
The Board has no unmet obligation to promulgate rules and regulations. The Board
has authority under A.R.S. §15-781.01 to make rules for its own government, but has
not adopted such rules. The Board must rely on rules and regulations adopted by the
State Board of Education that relate to vocational education programs and students.
Additionally, vocational education policies and procedures are addressed in the Ari-zona
Vocational Technological Education State Plan, a document required by federal
law. This Board-approved plan covers the operations and regulations of the State’s
vocational education program over a three-year period.
5. The extent to which the Board has encouraged input from the public before
promulgating its rules and regulations and to the extent to which it has in-formed
the public as to its actions and their expected impact on the public.
The Board has not promulgated rules and regulations covering vocational education.
However, the State Board of Education adopted rules in 1986 regarding vocational
education prior to the creation of a separate vocational education Board. The State
Board of Education appears to have adequately encouraged public input prior to
adopting these rules by adhering to the Arizona Administrative Procedures Act.
Further, monthly Board agendas are mailed to all newspapers and radio stations in
the State, as well as every school district superintendent, county school superinten-dent,
and charter school.
The Board also widely encouraged input for its 1994-1996 Vocational Technological
Education State Plan. This state plan has been extended due to delays in re-authorizing
the federal vocational education program and remains in effect. The plan
was addressed in public hearings across the State, with public notice and availability
to the written plan at least 10 days prior to all hearings. In addition, public hearings
were published in news media and special notices were mailed to potentially inter-ested
organizations, individuals, and institutions advising them of the public hear-ings.
6. The extent to which the Board has been able to investigate and resolve com-plaints
that are within its jurisdiction.
The Board is not a regulatory agency and does not have the authority to investigate or
resolve complaints. Therefore, this factor does not apply.
22
7. The extent to which the Attorney General or any other applicable agency of
state government has the authority to prosecute actions under enabling legisla-tion.
This factor is not applicable because the Board does not have enforcement or over-sight
responsibilities.
8. The extent to which the Board has addressed deficiencies in the enabling stat-utes
which prevent it from fulfilling its statutory mandate.
The Board has not pursued any changes in its enabling statutes. However, in 1997, the
Board supported a modification in state appropriations to consolidate all vocational
education funding into a single block grant. This should allow the State to streamline
administrative requirements, increase the proportion of monies passed on to schools,
and enhance program flexibility at the local level. Currently, state block grant monies
are distributed based on the number of students enrolled in vocational programs and
graduates placed in employment, further education, or military service related to their
vocational training.
9. The extent to which changes are necessary in the laws of the Board to ade-quately
comply with the factors listed in the Sunset review.
Based on a review of Board laws, no changes in statute appear necessary.
10. The extent to which the termination of the Board would significantly harm the
public health, safety, or welfare.
Termination of the Board would not pose a direct, immediate harm to the public
health, safety, or welfare. However, termination of the Board, without specifying an-other
entity to act as the State’s Board of Vocational Education, would be detrimental
to the State of Arizona in the annual loss of approximately $19 million of federal Carl
D. Perkins grants. As shown in the Introduction (see pages 1 through 5), this repre-sents
approximately 25 percent of total Arizona vocational education funding.
11. The extent to which the level of regulation exercised by the Board is appropri-ate
and whether less of more stringent levels of regulation would be appropri-ate.
The Board is not a regulatory body. As a result, this factor does not apply.
23
12. The extent to which the Board has used private contractors in the performance
of its duties and how effective use of private contractors could be accom-plished.
The Board has not used private contractors in the performance of its vocational edu-cation
oversight duties.
(This Page Intentionally Left Blank)
Agency Response
(This Page Intentionally Left Blank)
May 29, 1998
Mr. Douglas R. Norton
Auditor General
State of Arizona
2910 North 44th Street, Suite 410
Phoenix, Arizona 85018
Dear Mr. Norton:
Attached is a response submitted on behalf of the State Board for
Vocational and Technological Education to the report of the Performance
Audit and Sunset Review of the Board. We appreciate the opportunity to
respond to the observations and recommendations contained in the report.
I would also like to take the opportunity to compliment you on your staff
who served as the audit team. The team was diligent, thorough and
cooperative. The members of the Board and the staff were kept well
informed of the work of the team and the staff met regularly with the team
to discuss issues. Our perspectives on the issues were respectfully
considered and there were no surprises contained in the report.
Sincerely,
Kenneth R. Bennett, President
State Board for Vocational and Technological Education
Performance and Sunset Audit Response
State Board for Vocational and Technological Education
May 29, 1998
FINDING I.
Recommendations. The Board needs to:
1.a. Involve Arizona employers in policy discussions and ensure programs
meet their expectations.
Response: The finding of the Auditor General is agreed to and the audit
recommendation will be implemented. Arizona employers are actively involved with Vocational
Technological Education programs and provide valuable input for program improvement. The
Board will implement policy that involves employers in policy discussions and ensures programs
meet their expectations.
1.b. Review its current method of funding programs and determine if a new
funding formula is warranted.
Response: The finding of the Auditor General is agreed to and a difference
method of dealing with the finding will be implemented. This area is under constant review and
the Board will involve the Vocational Education Advisory Committee in studying the system and
use this information for funding recommendations to the Board. The Arizona Legislature has
taken recommendations from the recent PAR review of the School-to-Work Division and local
district and is considering increasing the funding level for placement to 10% of the Block Grant.
The Department of Education has recently developed, with the assistance of local districts, a
process for collecting placement data. The local districts have welcomed the system and they
have commented that the new system will decrease their workload while obtaining valuable
placement outcome data. Currently, the newest wage data has been factored in to the priority list
that will take effect in FY99.
1.c. Coordinate with the Governor’s Council for Workforce Development
Policy regarding the State’s efforts to prepare students for work.
Response: The finding of the Auditor General is agreed to and a different method
of dealing with the finding will be implemented. The Commerce School To Work Program
purpose is to improve the transition from the school to work system. A member of the State
Board of Education serves on the Workforce Development Council, the Department staff has
been actively involved with the Commerce Office, and joint delivery of services have been
coordinated by the two offices. The Department will continue intensive involvement with the
Commerce School To Work Division to ensure that there is not a duplicate effort between the
Department’s first chance and Commerce’s second chance system. Although a State Board of
Education member serves on the Council as a liaison, the Department will facilitate the process
by providing support for the Board members’ participation.
3
1.d. Assist school districts in implementing its Workplace and Technology
Skills standards.
Response: The finding of the Auditor General is agreed to and the audit
recommendation will be implemented. The Board will involve the Vocational Education
Advisory Committee in developing recommendations as to the implementation of the Workplace
and Technology Skills standards. The Board will consider the recommendations and create
policy related to the implementation of the standards.
2. The Board should consider reactivating its advisory committee to help
address these vocational education issues.
Response: The finding of the Auditor General is agreed to and the audit
recommendation will be implemented. The Board will consider re-establishing a Vocational
Education Advisory Committee. The Board will examine the make-up of the existing committee
and consider changes to its composition, number and membership to best serve the needs of the
students. Upon establishing a Vocational Education Advisory Committee, the Board will
communicate its expectations to the committee and identify the specific issues the committee
should address, ensure the committee meets regularly, review the recommendations of the
committee and consider appointing a Board member to serve on the committee.
FINDING II.
Recommendations.
1. The State Board for Vocational Technological Education, in coordination
with ADE staff, should review the academic gain performance standard to make it more realistic
and to facilitate distinction between quality and struggling programs.
Response: The finding of the Auditor General is agreed to and a different method
of dealing with the finding will be implemented. The Carl D. Perkins Vocational and Applied
Technology Act requires a Committee of Practitioners to assist the Board in the development of
core standards and measures of performance. This Committee will address this issue and prepare
recommendations for consideration by the Board.
2. The Board should require ADE to revise its computer database to:
2.a. Record more specific measures of program performance, such as the
actual percentage of students who demonstrate academic gain, instead of absolute compliance
(i.e., “yes” or “no”).
Response: The finding of the Auditor General is agreed to and a different method
of dealing with the finding will be implemented. The School-to-Work Division has initiated this
recommendation with the undertaking of the Business Process Improvement (BPI). The initial
phases are expected to be complete within the next 90 days. The Department has targeted July 1
4
as a start-up date for the initiation of the School-to-Work automation project. The Department is
aggressively pursuing data collection and analysis for all state and federal programs.
2.b. Standardize definitions and formatting across years.
Response: The finding of the Auditor General is agreed to and a different method
of dealing with the finding will be implemented. It is unclear in the finding which issues the
Auditor General is addressing, however, the Department will continue to see standardization of
definitions and format to strengthen consistency over time. There is new federal legislation
pending which may require modification of the performance standards.
2.c. Generate reports that track programs’ performance over time.
Response: The finding of the Auditor General is agreed to and the audit
recommendation will be implemented. Over the course of the BPI, consideration will be given
to data analysis and ensuring accountability of vocational technological programs. The
Department is establishing technical language and consistent objectives to evaluate programs
over time.
3. The Board should require all school districts to submit all necessary
information for their local evaluations.
Response: The finding of the Auditor General is agreed to and the audit
recommendation will be implemented. The Board will solicit recommendations from the
Vocational Education Advisory Committee for the requirement to submit all necessary
information, while being sensitive to the demands placed on local district personnel for
submission of additional documentation and paperwork.
4. The Board should ensure that ADE increases the technical assistance it
provides to school districts, including:
a. Assisting in the development of written program improvement plans; and
b. Following up to ensure improvements are made.
Response: The finding of the Auditor General is agreed to and the audit
recommendation will be implemented.