Performance audit, jobs for Arizona graduates program

              January 1988 Report No. 88- 1
The Office of the Auditor General has conducted a performance audit of the Jobs for Arizona
Graduates program. This performance audit was conducted in response to Chapter 334, Subdivision
77, of the 1987 Session Laws, which directs the auditor general to prepare an evaluation of the Jobs
for Arizona Graduates program with recommendations relating to the organization and
management, program content, student selection criteria, program placement results, and cost
effectiveness of the program.
The Jobs For Arizona Graduates ( JAG) program was established in 1980 as a State affiliate of Jobs
for America's Graduates, Inc. ( JAM GI. JAM G, the parent organization of J A G , was established in
1979 as a means to use research and demonstration activities as a national strategy to reduce youth
unemployment. Arizona's program began in 1980 with 28 participating high schools and 1,448
students. In 1987- 88, seven schools and 161 students participated in the program.
The JAG Program Is Not Successful
In Placing High School Seniors In Jobs
The Legislature should consider terminating the JAG program because i t does not appear successful
in placing high school seniors in jobs. The Jobs for Arizona Graduates program attempts to target
high school seniors defined as " at risk" and provide the students with necessary skills to get and
maintain jobs. " A t risk" students are those who do not already possess marketable skills, are
enrolled in general academic programs, and have limited or no employment experience.
r Our analysis of 1985- 86 JAG participants shows that the program has had limited success.
Very few " at risk" students were placed in jobs. Only 11 percent ( 36) of the 328 students in our
sample could be defined as " at risk" students, and only 17 of the " at risk" students were placed
in jobs.
0 Many of the jobs did not meet the program goals which suggest that all job placements should
be full- time positions that pay more than minimum wage. Almost half of the students who
received jobs were employed in part- time positions. An estimated 30 percent of all JAG
1985- 86 graduates received minimum wages or less, and " at risk" students received lower
average wages than the " not at risk" students.
Previous employment appeared to be the most significant contributor to job placement. Most
of the jobs found by the 1986 graduates appear to be the result of their own experience rather
than any skills learned in the JAG program.
The Cost Of Placing Students Through The JAG
Program Appears High
Although the cost effectiveness of JAG is difficult to measure, the cost of placing students through
the program appears excessive. An analysis by our Office found that program costs per student are
viewed in several different ways depending upon the definition of students benefiting from the
program. For example, based on the assumption that any student who enrolled in the program
benefited to some degree, even those who dropped out of school or were not placed in jobs, the cost
per student for the 1985- 86 school year would be calculated at $ 1,084. However, if job placement
for " at risk" students is the only definition or criteria for program success, the cost per " at risk"
student placement leaps to $ 21,350 per student.
COST OF 1985- 86 JAG PROGRAM FOR
VAR l OUS STUDENT GROUPS
Number
Cost Per
Student
ALL STUDENTS 571 $ 1,084
P l aced 297 $ 2,085
" AT RISK" STUDENTS
Completing the Program 63 $ 9,828
P I aced 29 $ 21,350
Source: Prepared by the Office of the Auditor General from analysis of 1985- 86
JAG data.
JAG Did Not Follow Legally Required Procedures
When Reimbursing N A U $ 367,202 For
Its Services As Fiscal Agent
Fragmented responsibility and the reliance on informal agreements led JAG to pay Northern
Arizona University ( NAU) $ 367,202 for its services as fiscal agent without following legally
required procedures. Since JAG was implemented in 1980, the Authority Board, Arizona
Department of Education ( ADE), and NAU each played a role in the program's operation. No one
entity had complete program responsibility. Because of the fragmented responsibility for JA G's
operation, a formal agreement was essential to clarify responsibilities. Further, a November 1987
Legislative Council opinion concluded that an intergovernmental agreement was needed between
JAG and NAU to provide for a means of compensation. Because JAG and NAU had never
developed such an agreement, no legal basis existed for JAG to pay N A U for i t s services.
The lack of interagency agreement also affects the ownership of program property. According to
Legislative Council opinion, the informal written agreement between NAU and ADE does
constitute a written contract. However, a formal intergovernmental agreement is needed to
transfer property. In the absence of a valid agreement, NAU cannot retain ownership of JAG
purchased furniture.
DOUGLAS R. NORTON. CPA
AUDITOR GENERAL
STATE OF ARIZONA
OFFICE OF THE
AUDITOR GENERAL
LINDA J. BLESSING, CPA
DEPUTY AUDITOR GENERAL
January 29, 1988
Members of the Arizona Legislature
The Honorable Evan Mecham, Governor
The Honorable C. Diane Bishop, Superintendent of Public Instruction
Dr. Eugene M. Hughes, President, Northern Arizona University
Transmitted herewith is a report of the Auditor General, a Performance Audit
of the Jobs for Arizona Graduates ( JAG) Program. This report is in response to
Chapter 334, Subdivision 77 of 1987 Session Laws.
The report recommends that the Legislature terminate the JAG program
because the program has not succeeded in placing high school seniors in jobs.
The program placed very few " at risk" students in jobs and had limited success
in securing full- time positions paying more than minimum wage. The cost of
placing students through the JAG program also appears to be excessive. In
addition, we found that the JAG program paid Northern Arizona University
$ 367,202 for its services as fiscal agent without following legally required
procedures.
My staff and I will be pleased to discuss or clarify items in the report.
Respectfully submitted,
D O U ~ IS?. Norton
Auditor General
Staff: William Thomson
Mark Fleming
Jerome Miller
Kurt Schulte
Jane Burgess
Enclosure
2700 NORTH CENTRAL AVE. 0 SUITE 700 0 PHOENIX, ARIZONA 85004 @ ( 602) 255- 4385
SUMMARY
The Office of the Auditor General has conducted a performance audit of the Jobs
for Arizona Graduates program. This performance audit was conducted in response
to Chapter 334, Subdivision 77, of the 1987 Session Laws, which directs the auditor
general to prepare an evaluation of the Jobs for Arizona Graduates program with
recommendations relating to the organization and management, program content,
student selection criteria, program placement results, and cost effectiveness of the
program.
The Jobs For Arizona Graduates ( JAG) program was established in 1980 as a State
affiliate of Jobs for America's Graduates, lnc. ( JAM G). JAMG, the parent
organization of JAG, was established in 1979 as a means to use research and
demonstration activities as a national strategy to reduce youth unemployment.
Arizona's program began in 1980 with 28 participating high schools and 1,448
students. In 1987- 88, seven schools and 161 students participated in the program.
( see pages 7 - 12)
The Legislature should consider terminating the JAG program because it does not
appear successful in placing high school seniors in jobs. The Jobs for Arizona
Graduates program attempts to target high school seniors defined as " at risk" and
provide the students with necessary skills to get and maintain jobs. " A t risk"
students are those who do not already possess marketable skills, are enrolled in
general academic programs, and have limited or no employment experience.
0 Our analysis of 1985- 86 JAG participants shows that the program has had
limited success. Very few " at risk" students were placed in jobs. Only 11
percent ( 36) of the 328 students in our sample could be defined as " at risk"
students, and only 17 of the Itat risk" students were placed in jobs.
a Many of the jobs did not meet the program goals which suggest that all job
placements should be full- time positions that pay more than minimum wage.
Almost half of the students who received jobs were employed in part- time
positions. An estimated 30 , percent of all JAG 1985- 86 graduates received
minimum wages or less, and " at risk" students received lower average wages
than the " not at risk" students.
a Previous employment appeared to be the most significant contributor to job
placement. Most of the jobs found by the 1986 graduates appear to be the
result of their own experience rather than any skills learned in the JAG
program.
The Cost Of Placing Students Through The JAG
Program Appears High ( see pages 13- 15)
Although the cost effectiveness of JAG is difficult to measure, the cost of placing
students through the program appears excessive. An analysis by our Office found
that program costs per student are viewed in several different ways depending upon
the definition of students benefiting from the program. For example, based on the
assumption that any student who enrolled in the program benefited to some degree,
even those who dropped out of school or were not placed in jobs, the cost per
student for the 1985- 86 school year would be calculated at $ 1,084. However, if job
placement for " at risk" students is the only definition or criteria for program
success, the cost per " at risk" student placement leaps to $ 21,350 per student.
COST OF 1985- 86 JAG PROGRAM FOR
VARIOUS STUDENT GROUPS
Number
ALL STUDENTS 571
P laced 297
" AT RISK" STUDENTS
Completing the Program 63
P laced 29
Cost Per
Student
Source: Prepared by the Office of the Auditor General from analysis of 1985- 86
JAG data.
JAG Did flot Follow Legally Required Procedures
When Reimbursing N A U $ 367,202 For
Its Services As Fiscal Agent ( see pages 17 - 20)
Fragmented responsibility and the reliance on informal agreements led JAG to pay
Northern Arizona University ( N A U) $ 367,202 for its services as fiscal agent without
following legally required procedures. Since JAG was implemented in 1980, the
Authority Board, Arizona Department of Education ( AD€), and NAU each played a
role in the program's operation. No one entity had complete program
responsibility. Because of the fragmented responsibility for JA G's operation, a
formal agreement was essential to clarify responsibilities. Further, a November
1987 Legislative Council opinion concluded that an intergovernmental agreement
was needed between JAG and NAU to provide for a means of compensation.
Because JAG and NAU had never developed such an agreement, no legal basis
existed for JAG to pay N A U for its services.
The lack of interagency agreement also affects the ownership of program property.
According to Legislative Council opinion, the informal written agreement between
NAU and AD€ does constitute a written contract. However, a formal
intergovernmental agreement is needed to transfer property. In the absence of a
valid agreement, N A U cannot retain ownership of JA G purchased furniture.
TABLE OF CONTENTS
Page
INTRODUCTION AND BACKGROUND . . . . . . . . . . . . . . . . . . . 1
FINDING I : THE JOBS FOR ARIZONA GRADUATES PROGRAM
I S NOT SUCCESSFUL I N PLACING
HIGHSCHOOL SENIORS I N JOBS. . . . . . . . . . . . . . . . . 7
JAG'S GOAL I S TO SECURE EMPLOYMENT
FOR " A T R I S K " STUDENTS . . . . . . . . . . . . . . . . . . . 7
PLACEMENT RATE FOR
" ATRISK1' STUDENTS ISLOW . . . . . . . . . . . . . . . . . 8
MANY JOBS WERE
PART- TIME, LOWWAGEPOSITIONS . . . . . . . . . . . . . . . 11
JOB PLACEMENTS
NOT ATTRIBUTABLE TO JAG PROGRAM . . . . . . . . . . . . . . 11
RECOMMENDATION . . . . . . . . . . . . . . . . . . . . . . . 1 2
FINDING I I : THE COST OF PLACING STUDENTS THROUGH THE
JOBS FOR ARIZONA GRADUATES PROGRAM APPEARS HIGH . . . . . . 13
PLACEMENT COSTS FOR " AT RISK" STUDENTS MAY BE EXCESSIVE , . 13
OTHER PROGRAMS COST LESS . . . . . . . . . . . . . . . . . . 1 4
RECOMMENDATION . . . . . . . . . . . . . . . . . . . . . . . 15
FINDING I l l : JAG D I D NOT FOLLOW LEGALLY REQUIRED PROCEDURES
WHEN RE I MBURS l NG NAU $ 367,202
FOR ITS SERVICES AS FISCAL AGENT . . . . . . . . . . . . . . 17
RESPONSIBILITY I S FRAGMENTED . . . . . . . . . . . . . . . . 17
PROGRAM OFFICIALS FAILED TO FOLLOW LEGALLY
REQUIRED PROCEDURES . . . . . . . . . . . . . . . . . . . . 18
CURRENT ORGANIZATION UNDER ADE IMPROVES CONTROL . . . . . . 2 0
RECOMMENDATIONS . . . . . . . . . . . . . . . . . . . . . . 2 0
. -
AGENCY RESPONSES . . . . . . . . . . . . . . . . . . . . . . . . . 2 1
APPEND l X
LEGISLATIVE COUNCIL OPINION
LIST OF TABLES
Page
TABLE 1 JAG EXPENDITURES
FISCAL YEARS 1985- 86 THROUGH 1987- 88
( unaudited) . . . . . . . . . . . . . . . . .
TABLE 2 ARIZONA JAG STUDENTS WHO WERE
" NOT AT RISK" AND " AT RISK" FOR UNEMPLOYMENT
TABLE 3 RESULTS OF " AT R I S K , " " NOT AT RISK,"
AND UNKNOWN STUDENT PLACEMENTS . . . . . . . . . . . . 10
TABLE 4 COST OF 1985- 86 JAG PROGRAM
FOR VARIOUS STUDENT GROUPS . . . . . . . . . . . . . . 14
INTRODUCTION AND BACKGROUND
The Office of the Auditor General has conducted a performance audit of the Jobs
for Arizona Graduates program. This performance audit was conducted in response
to Chapter 334, Subdivision 77, of the 1987 Session Laws, which states " It is
legislative intent that the auditor general prepare an evaluation of the Jobs for
Arizona Graduates program to be submitted to the legislature by January 31, 1988,
with recommendations relating to the organization and management, program
content, student selection criteria, program placement results and cost
effectiveness of the program."
National Affiliation
The Jobs for Arizona Graduates ( JAG) program is affiliated at the national level
with Jobs for America's Graduates, Inc. ( JAMG). Jobs for America's Graduates was
started in 1979 as a means to use research and demonstration activities as a national
strategy to reduce youth unemployment. Each state program is reviewed annually
by the national Jobs for America's Graduates for compliance with the JAMG Model
for program and curriculum, the existence of a student career association, and the
accurate maintenance of student job placement data. While the JAMG provides
oversight at the national level and enforces program compliance, i t provides no
funding to states in the program.
In 1980 Arizona became the second state to join Jobs for America's Graduates.
Today there are 12 states in JAMG, with four having joined the program in the last
11 months. The organizational structure of the JAMG program in each state is
unique, and depends largely on funding sources and scope of the program within the
state. O f the eight states with the program longer than one year, six function as
part of a private, non profit corporation, while only two are administered by the
state departments of education.
The scope of the program within the state can also vary. For instance, in Missouri
the program is concentrated i n the St. Louis area, while in Massachusetts the
program is everywhere outside the Boston area. In Delaware the program is
statewide. The program in Arizona has operated primarily in the Phoenix and
Tucson areas.
Funding sources also differ in each state, but most do rely heavily on Federal
funding received through the Job Training Partnership Act ( JTPA). JAG has relied
on a variety of funding sources over the years. with the largest being a State
appropriation. Contrary to other states, JTPA funding has never figured
prominently in JA G's operations.
History of Jobs for Arizona Graduates
Jobs for Arizona Graduates began in 1980 with 18 school districts and 28 high
schools participating in the program. In fiscal year 1982- 83 the program increased
to 21 districts and 35 schools with some districts operating more than one program.
The total student enrollment was 1,448. The current fiscal year 1987- 88
participation is seven schools with 161 students enrolled. Program funding followed
a similar trend, and ranged from $ 1.5 million available in fiscal year 1980- 81 to the
current fiscal year 1987- 88 funding of $ 350,808.
Prior to fiscal year 1986- 87, an Authority Board, the Arizona Department of
Education ( ADE) and Northern Arizona University ( NAU) all had roles in the
administrative oversight for the program. "' It was NAU's understanding that
the Authority Board was the controlling body for the program. NAU reduced the
Board's power to that of an advisory function when it took control of the program in
July of 1986. However, on July 1, 1987, ADE was given sole responsibility for the
program. Significant changes have occurred since that time. N A U's affiliation with
JAG terminated when the program was transferred to ADE. Also, the role and
power of the Authority Board has been diminished significantly to the point where i t
now serves only an advisory function.
In the early years of the program, school districts were not required to provide any
funding support to maintain the program. All money for program operations was
provided to participating schools. Sources for those funds were a Federal grant,
See F i n d i n g 111, page 19, f o r a d e s c r i p t i o n o f the r o l e o f the A u t h o r i t y Board, the
Department o f Education and N o r t h e r n A r i z o n a U n i v e r s i t y .
State appropriations and private donations. Also, job specialists "' were JAG
employees and were not required to have vocational education certification. ADE
now requires a strong commitment to JAG by the school districts. Each school
wishing to participate in fiscal year 1987- 88 must make a $ 25,000 ( cash or in- kind)
commitment to the program, and participating schools are also required to submit a
grant application to ADE outlining whether this commitment will be cash or in- kind.
Each school has a payment schedule established with A DE for receipt of this funding.
The State appropriation money can only be used for job specialists' salaries and
benefits. A l l other program expenses must be funded by the school. Also, the job
specialists are now employees of the schools, and are required to have vocational
education or academic certification.
Table 1 ( page 4) shows funding and staffing levels for the last three fiscal years.
JAG Curriculum - The JAG program curriculum focuses on the skills necessary for
students to be successful in the job market. The job specialist attempts to identify a
student's interests and abilities and develop valuable job competencies. In addition,
the curriculum is designed to teach skills necessary to obtain a job.
The curriculum, the pre- tests and the post- tests, are grouped into three sections.
a Career Development Competencies - Development of the student's awareness
about " hidher own special aptitudes, abilities, interests, life goals and desired
life styles." Also included is information about " the world of work, identifying
those occupations which are consistent with hidher own goals.
a Job Attainment Competencies - " Those skills and abilities involved in
successfully obtaining a job,'' such as constructing a resume, conducting a job
search, arranging for and completing a job interview, and completing
applications.
Job Survival, Leadership, and Self- Development Competencies - Competencies
required for job success ( appearance, expectations, time management), for
managing personal finances ( budgeting, using credit), and for functioning
effectively in team or group projects ( team membership, peer relationships,
organizational commitment).
The job s p e c i a l i s t works out ( if the high school and i s responsible f o r student job
placement and r e t e n t i on.
TABLE 1
JAG EXPEND l TURES
F l SGAL YEARS 1985- 86 THROUGH 1987- 88
( Unaud i ted)
Actual
1985- 86
FTEs 20 ( a)
Personal Services $ 51 2,731
Other Operating 24,707
Professional Services 10,337
In- State Travel 6,921
Out- of- State Travel 2,357
Capital Out lay 1 ,647
I n d i r e c t & A d m i n i s t r a t i v e Costs 60,439
Lump Sum
Actual
1986- 87
Budgeted c b )
1987- 88
( a) Includes two JAG Employees a t NAU.
( b) Only two f u l l - t i m e p o s i t i o n s a r e budgeted f o r Central O f f i c e s t a f f , since a l l job
s p e c i a l i s t s are now employees o f the schools. The $ 350,808 amount includes $ 225,000
t o be a l l o c a t e d t o the school d i s t r i c t s f o r job s p e c i a l i s t s a l a r i e s and $ 125,808 i s
a v a i l a b l e f o r other operating costs. $ 238,522 i s a v a i l a b l e t o the program from
cash/ in- kind matches provided by p a r t i c i p a t i n g d i s t r i c t s .
( c ) The budget amount does not i n c l u d e the cash/ in- kind match amount.
Source: Auditor General analysis of JAG program expenditures and 1987- 88
JAG budget.
Students are tested before entering the JAG prqram ar; d after completing each
competency section. The differences between the pre- test and post- test scores are
a measure of the gains achieved within the program.
Methodology
To evaluate JAG program content, program placement results, student selection
criteria and program cost effectiveness, as required in Chapter 334 of the 1987
Session Laws, student data from the 1985- 86 school year was used. ( 1 )
In 1985- 86 15 schools participated in the JAG program: Carl Hayden, South
Mountain, and Deer Valley High Schools in Phoenix; Marcos de Niza in Tempe;
Coronado High School in Scottsdale; Mesa High School; Chandler High School;
Tolleson High School; Casa Grande High School; Coolidge High School; Santa Cruz
Valley Union High School in Eloy; and Cholla, Palo Verde, Pueblo and Rincon High
Schools in Tucson.
Data were collected from three sources. Grade point averages and attendance
records were obtained from the high school permanent records. The JAG student
files maintained at the high schools contained most of the detail needed for
analysis. Data were also obtained from the JAG program files currently maintained
at NAU.
Data collected from JAG student files included the hours of contact between the job
specialist and each student, pre- and post- test scores, and information on vocational
education classes, special skills, training, and previous employment. Additionally,
information regarding contacts between the job specialist and student during the
nine- month follow- up period "' was available in the student's file.
This 1985- 86 student data was- used because it i s the l a t e s t year i n which the
program operated i n i t s e n t i r e t y .
( 2) The job s p e c i a l i s t s are required to monitor the students j o b performance f o r nine
months a f t e r placement.
Information regarding each student's availability for work and placement
information was obtained from the JAG main files. Data were collected regarding
student outcomes and placement for up to three jobs, with salaries, promotions,
raises, etc., being noted. Demographic data were also available from the JAG files
which included: sex, race, marital status, dependents, i f any, handicapped status,
citizenship, birth date.
Various types of statistical analyses were conducted on this data to provide us with
information to address the program aspects specified in the Session Law.
Audit Scope
Our audit of the Jobs for Arizona Graduates program addressed the four statutorily
mandated areas.
Finding I
a The program's effect on student job placement
m The effect of student selection criteria and program content on JAG
participants
Finding II
a Overall cost effectiveness of Arizona's program
Finding Ill
e Evaluation of the organization and management of the program
This audit was conducted in accordance with generally accepted governmental
auditing standards.
The Auditor General and staff ' express appreciation to the administrations of
Northern Arizona University, the Arizona Department of Education, and the high
schools involved for their cooperation and assistance during the course of our audit.
FINDING I
THE JOBS FOR ARIZONA GRADUATES PROGRAM IS NOT
SUCCESSFUL IN PLACING HIGH SCHOOL SENIORS IN JOBS
The Jobs for Arizona Graduates program ( JAG) does not appear successful in placing
high school seniors in jobs. Although the program's goal is to secure full- time,
permanent employment for those students who lack necessary skills, training, and
experience, analysis of the 1985- 86 data indicates that the placement rate for these
targeted " at risk" students is very low. Many students who did obtain jobs found only
part- time, low wage positions. In addition, analysis shows that the job placements
that did occur were not attributable to the JAG program.
JAG'S Goal Is To Secure Employment
For " A t Risk" Students
The goal of the JAG program is to secure full- time, permanent employment for those
high school seniors who lack the necessary skills, training and experience to make a
successful transition into the working world. Standards and program outlines
established by Jobs for America's Graduates ( JAMG), the national organization, do
not specifically set forth criteria for the selection of students. The Program
Operations Handbook ( Volume 1, 1983) states, however, that " students selected for
the program should be those seniors in the high school that are most likely to be
unemployed following graduation." This same Handbook, revised in 1985, also states
that " targeted students primarily include those students referred to as general
education students and those most likely to lack the skills and/ or motivation and
preparation necessary to find and retain employment upon graduation." Throughout
JAM G literature there is a consistent indication that students should be those high
school seniors who:
e are enrolled in general academic programs ( i. e., not taking vocational education
classes);
e do not already possess employability skills;
e have limited or no employment experience.
In order to operationalize this definition, we chose individual 1985- 86 student
participants who had not taken vocational education classes, had no special
employability skills, and had not been previously employed. For our analysis, student
participants with these characteristics are defined as " at risk."
Placement Rate For " A t Risk''
Students Is Low
The JAG program in Arizona appears to place few " at risk" students in jobs. An
analysis of sample data from the 1985- 86 student files (' I indicates that most of
the students in the JAG program were not the types of students the program says i t
targets. They either had taken vocational education classes, had special
employability skills, or had previously been employed and were, therefore, not at high
risk of unemployment. Table 2 ( page 9) shows that the number of students who
actually were " at risk" is very low at each of the nine schools.
Only 36 ( 11 percent) of the students were actually " at risk" for unemployment as
defined by JAMG. Even if those students whose risk status is unknown ( 26) are
presumed to be " at risk," the total of " at risk" students increases only to 62 ( 19
percent). Most students ( 266, or 81 percent) are not the at risk students as defined by
JAM G. Thus, the number of targeted students assisted by JAG was minimal since the
majority of students during 1985- 86 were not " at risk."
Because the number of " at risk" students enrolled in the JAG program is low, these
students' placements are also very low. Table 3 ( page 10) illustrates that only 17 " at
risk" students were placed, compared to the 156 " not at risk" students who found
jobs. Overall, " at risk" students who were placed account for only 5 percent of the
entire sample of 328 students.
This low success rate contrasts sharply with the rate for Arizona reported by the
national JAMG program. JAMG reported in 1986 that 63 percent of its Arizona
students found jobs and that 90 percent found jobs or experienced a " positive
o u t c ~ r n e(.')~ ~
( ' 1 Sample data are based on nine schools where complete student f i l e s were a v a i l a b l e .
The a v a i l a b l e data f o r a l l 15 schools p a r t i c i p a t i n g i n the JAG program during 1985- 86
were also examined as p a r t o f the analysis, and no s u b s t a n t i a l d i f f e r e n c e s were found
between schools w i t h complete student data and those w i t h on1 y p a r t i a l data.
Therefore, t h i s sample appears t o be h i g h l y representative of the e n t i r e 1985- 86
student population.
(') P o s i t i v e outcomes include students f i n d i n g jobs a t the end o f the JAG program and
students who decide t o e n t e r c o l l e g e , post- secondary t r a i n i n g o r the m i l i t a r y .
TABLE 2
ARIZONA JAG STUDENTS
WHO WERE " NOT AT RISK" AND " AT RISK"
FOR UNEMPLOYMENT
Total Students " Not A t Risk"
High School Enrolled i n JAG Students
Casa Grande 4 7 36
Chand 1 e r 4 4 34
Coo l i dge 9 8
Co ronado 5 5 4 7
Deer Valley 3 5 3 1
Marcos de Niza 35 2 5
Pueb 1 o 3 7 28
Santa Cruz Val ley 24 2 1
To l l eson 42 2
m
Totals 328
Percentages 100% 81 %
" A t Risk"
Students Unknown
5
1
Note: These are sample schools o n l y . See footnote ( 1 page 8, for
d e s c r i p t i o n of sample data.
I
Source: Prepared by O f f i c e of the Auditor General from analysis of 1985- 86
JAG data.
TABLE 3
RESULTS OF " AT RISK,"
" NOT AT RISK," AND UNKNOWN STUDENT PLACEMENTS
Total " Not A t Risk" ffAt Risk" Unknown
High School Placements Placements Placements Placements
Casa Grande 25 2 1 1 3
Chand I e r 2 4 20 4 -
Coo l i dge 3 3 - -
Co ronado 30 28 1 1
Deer Val ley 2 1 18 3 -
Marcos de Niza 22 2 1 1 -
Pueb lo 11 9 2 -
Santa Cruz Val ley 17 15 2 -
To l Ie son 24 2 1 3 --
Totals 177 2i! i 17 4 0
Note: These are sample schools o n l y . See footnote , page 8, f o r
d e s c r i p t i o n of sample data.
a
Source: Prepared by O f f i c e of the Auditor General from a n a l y s i s of
1985- 86 JAG data.
However, these figures overstate the program's success in two ways. First, JAM G is
unaware that a large proportion of Arizona students are not " at risk." Second, JAM G
bases its percentages on the number of students " available for placement." ( ' )
During 1985- 86 approximately 31 percent of the total student enrollment was not
available for placement, and most of these unavailable students had dropped out of
school during the year. Thus, the nationally reported placement rate for JAG
students appears much higher than its actual accomplishment.
Many Jobs Were Part- time,
Low Wage Positions
Many of the jobs filled by 1985- 86 JAG graduates do not meet the program's goal.
According to JAG literature, placements are supposed to be full- time positions that
pay more than minimum wage and provide an opportunity for advancement.
However, our analysis shows that almost half of the total placements ( 47 percent)
were in part- time positions.
An analysis of the hourly wages received by the 1985- 86 JAG graduates indicated
that many of these jobs were low paying positions. An estimated 30 percent of the
graduates earned the minimum wage of $ 3.35 per hour or less. The data also
indicated that the " at risk" students received lower average wages ($ 3.93 per hour)
than did " not at risk" students ($ 4.09 per hour). These results differ from JAG'S
report, issued in March 1987, that its 1986 graduates earned an average wage of $ 4.23
an hour for a 40- hour week.
Job Placements Not Attributable
To JAG Program
Although some 1985- 86 JAG students found jobs, those jobs do not appear to be a
result of the JAG program. Analysis of the 1985- 86 data indicates that the
curriculum '*) does not enhance the employability skills of " at risk" students, and
has no significant relationship to job placement. Previous employment appears to be
the only significant contributor to job placement.
Students unavailable f o r pla'c& nent are those who dropped out of school, d i d not
graduate, are pregnant and not seeking employment, are recuperating from a serious
i l l n e s s or i n j u r y , o r are imprisoned.
(') See pages 4 and 5 f o r JAMG's d e s c r i p t i o n of t h e c u r r i c u l u m .
Analysis of the 1985- 86 JAG student data revealed that the curriculum fails to
enhance the employability skills of " at risk" students. The differences between pre-and
post- test scores in the three curriculum areas - Career Development, Job
Attainment and Job Survival - were assessed separately for " at risk" and for " not at
risk" students. While students " not at risk" significantly improved their
scores in all three areas, " at risk" students showed no improvement in any area.
Thus, the " at risk" students did not significantly increase their knowledge of
employment skills.
In addition, our analysis found that the curriculum does not provide knowledge that
facilitates job placement. The student data showed no significant relationship
between higher post- test scores and job placement. On the contrary, those students
who were placed in jobs had statistically significant lower post- test scores than did
students who were not placed in jobs. ( 2 )
Only one variable, previous employment, appeared to contribute significantly to JAG
graduates' success in finding employment. ( 3) Most of the jobs found by the 1986
graduates appear to be the result of their own experience rather than any skills
learned in the JAG program.
Not only is the JAG program ineffective, the strong lack of effect raises strong
doubts as to whether the program can be made effective. In essence, a new program
with a different target group and a different curriculum would be needed.
RECOMMENDATION
The Legislature should consider terminating the Jobs for Arizona Graduates program
because i t does not appear to contribute to increased employment opportunities for
" at risk" students.
Career Development -- $ = 6.40.,- g, < .001
Job Attainment -- = 3.36, g, < .001
Job Survival -- L = 3.93, & < .001
( 2) = 2.90, g, < .01
( 3 ) C h i - s q u a r e = 5 . 3 0 , ~ < . 0 5 , K e n d a l l ' s T a u b = . 1 5 , p , < . 0 1
FINDING I1
THE COST OF PLACING STUDENTS THROUGH THE
JOBS FOR ARIZONA GRADUATES PROGRAM
APPEARS HIGH
Although the cost effectiveness of the Jobs for Arizona's Graduates ( JAG) program is
difficult to measure with any certainty, the cost of placing students through JAG
appears high. Analysis of placement costs for the " at risk" students shows that the
cost per placement for the 1985- 86 school year may be excessive. Other programs
appear to train and place students at less cost than JAG.
Placement Costs For " A t Risk"
Students May Be Excessive
The cost of placing " at risk" students may be excessive. An analysis by our Office
found that in 1985- 86 the " at risk" students represented about 11 percent of the
population we sampled, and their placement rate was approximately 5
percent. When these percentages are applied to the entire JAG student
population, we calculate about 63 " at risk" students in the total population of 571,
and about 29 " at risk" placements. ( See footnote 2, page 8 for explanation.)
The cost of serving these " at risk" students during the 1985- 86 year appears high.
Total costs for the JAG program were $ 619,139. Table 4 shows that the program
costs per student can be viewed in several different ways depending upon the
definition of students benefiting from the program. For example, i f any student who
enrolled in the program benefited to some degree, even those who dropped out of
school or were not placed in jobs, the cost per student for the 1985- 86 school year is
$ 1,084. However, i f job placement for " at risk1' students is the only program success,
the cost per " at risktf student placement leaps to $ 21,350 per student. ( 2)
S e e p a g e s 7 t o 1 3 o f F i n d i n g I .
(') Jobs f o r America's Graduates ( JAMG) estimates t h a t n a t i o n a l l y average costs range
from " less than $ 700 per p a r t i c i p a n t " to " l e s s than $ 1,400 per placement."
TABLE 4
COST OF 1985- 86 JAG PROGRAM
FOR VARIOUS STUDENT GROUPS
ALL STUDENTS
P l aced
" AT RISK" STUDENTS
Completing the Program
P laced
Nurnbe r
571
297
Cost Per
Student
Source: Prepared by the Office of the Auditor General from analysis of 1985- 86
JAG data.
According to JAM G, i f the number of students per job specialist drops below 35, the
result will be " unacceptable costs per placement." In Arizona, during the 1985- 86
school year, five of the 15 participating schools had student enrollments of 35 or
fewer, and two other schools had enrollments of only 37 students.
0 ther Programs
Cost Less
Review of other programs suggests that JAG costs are high. Although, we could
identify no programs in Arizona that are directly comparable to JAG, two other
programs - the Mesa Youth Placement Service and Arizona vocational education
programs - do provide some basis for comparison.
An accurate assessment of the cost effectiveness of JAG requires comparing its costs
to those of similar programs. However, no programs in Arizona could be identified
that encompassed both essential elements of JAG: classroom instruction and job
placement. Two programs that each provide one of these elements ( classroom
instruction and job placement) were identified and their costs estimated f o r a
comparison with JAG.
14
Mesa Youth Placement Service - The Mesa Youth Placement Service ( MY PSI costs
about $ 151 per student placement. MYPS is a job placement service funded by the
Mesa High School District, City of Mesa, Mezona Foundation and Department of
Economic Security. It places students aged 12 to 19 in full or part- time, permanent
or temporary positions, and provides career counseling as well as skills workshops.
The job placement function of MYPS can provide somewhat of a comparison to the
job placement element in JAG. MY PS has an estimated 1,620 placements per year
and its 1987 expenditures were approximately $ 245,000, which results in cost per
placement at $ 151 per student. JAG's cost per placement for all students is $ 2,083
( see Table 4 page 14).
Arizona vocational education programs - A ri zona's vocational education programs
cost less than $ 293 per student. " ' Vocational education, like JA G, provides
classroom instruction. Vocational education in Arizona is taught in high schools,
community colleges, and other private sector secondary schools. Funding for
vocational education is provided by State appropriations and Federal grants as well as
by local school districts. The cost per student of Arizona's vocational education
programs was computed by adding the 1986 funding from all three sources, for a total
State expenditure of more than $ 65 million. In school year 1986- 87, an estimated
128,100 students were enrolled in public secondary schools and another estimated
97,000 in colleges. There is no estimation, however, of the number of students
enrolled in private secondary schools and in other educational institutions. Thus, the
$ 293 estimated cost per student in the program is a high figure, but is still less than
half of JAG's cost of $ 1,083 per participant ( see Table 4).
RECOMMENDATION
If the Legislature decides to continue funding student job placement programs, which
specifically target at- risk students, consideration should be given to developing and
funding less costly programs. Existing programs such as the Mesa Youth Placement
Service should be studied as a possible alternative to JAG.
, -
The amounts used t o c a l c u l a t e t h i s cost were obtained from t h e A r i z o n a Department o f
Education.
FINDING Ill
JAG DID NOT FOLLOW LEGALLY REQUIRED PROCEDURES WHEN
REIMBURSING NORTHERN ARIZONA UNIVERSITY
$ 367,202 FOR ITS SERVICES AS FISCAL AGENT
Fragmented responsibility and reliance on informal agreements led Jobs for Arizona
Graduates ( JAG) to pay Northern Arizona University ( NAU) $ 367,202 for its services
as fiscal agent without following legally required procedures. Oversight of the
program operations has always been fragmented, with several enti ties in partial
control. Informal agreements governed these relationships, which resulted in NAU
not following legally required procedures to receive reimbursement for its role as
program fiscal agent. However, current program organization centralizes oversight
and has increased the level of management control.
Responsibility Is Fragmented
Since JAG was implemented in 1980, there have been three entities responsible for
parts of its operation. No one entity had complete program responsibility.
a Authority Board - The JAG Authority Board was established by Laws 1981,
Chapter 196. It was mandated by statute to " establish a program for curriculum
development, identification, screening, selection and training of Arizona high
school graduates for jobs in this state." This statute expired effective June 30,
1982, and no continuing statute was ever developed. The Authority Board
maintained the ultimate decision- making responsibility for the program ( despite
no continuing statutory authority) until program responsibility was transferred
to N A U in 1986.
0 Department of Education - The Arizona Department of Education ( ADE) was
given a line- item appropriation by the Legislature to provide vocational
education assistance through JAG from fiscal year 1982- 83 through 1985- 86.
Program responsibility was transferred to ADE during this period since the
Authority Board had legally expired. During this period, A DE's responsibilities
were more than merely acting as a conduit for the distribution of money. This
responsibility concluded on May 5, 1986, when the Legislature transferred both
the 1986- 87 fiscal year appropriation and 20 full- time equivalent positions for
JAG from ADE to NAU.
a Northern Arizona University - Northern Arizona University served as fiscal
agent for the program from July 1, 1980 to June 30, 1986, and had full program
responsibility from May 1986- to June 1987. According to NAU personnel, its
duties as fiscal agent were to provide financial record keeping and personnel
administrative services. N A U administrative staff saw their role as that of the
" banker" for the program, though this role was never defined by a formal
agreement. In June 1987, the Legislature chose not to fund the program at
NAU, and transferred i t to ADE. ADE elected to serve as its own fiscal agent.
JAG Program Officials Failed To
Follow Legally Required Procedures
The fragmented responsibility for JAG'S operation increased the need for clear,
formal agreements among the various agencies. However, this did not occur. For
example, an intergovernmental agreement was needed between JAG and NAU to
define fiscal agent duties and means of compensation, yet one was never developed.
As a result, more than $ 367,000 in indirect and administrative cost reimbursement
was paid to NAU in an unauthorized manner for fulfilling the fiscal agent role.
Similarly, a valid agreement was also needed, but not developed, governing
disposition of JAG property.
Intergovernmental agreement needed - Lacking an agreement for payment has
resulted in more than $ 367,000 of unauthorized payments to NAU. However,
because both ADE and NAU appear to be satisfied with the services, further legal
action does not appear warranted.
NAU operated as program fiscal agent through an informal, unwritten agreement
with JAG and ADE. The indirect cost charges made by NAU to compensate for
costs incurred as fiscal agent were paid from the ADE line- item appropriation for
the JAG program. NAU charged the JAG program a yearly fee of approximately 5
percent of the total program budget. This amount was intended to cover the
" intangible" administrative costs. The exact amount of this fee was determined
through the budget process and approved by the authority board. According to the
NAU accountant, the amount of 5 percent was consistent from year to year, since
JAG was a continuing grant program. During NAU's tenure as fiscal agent i t
charged JAG more than $ 367,000.
According to a November 1987 Legislative Council opinion, an intergovernmental
agreement was necessary for NAU to be reimbursed for i t fiscal agent duties. The
opinion states:
" From the inception of the authority board to May 5, 1986, the indirect
costs have been part of an agreement between one state entity, NAU and
another, first the authority'- board then DOE. Interagency agreements are
governed by title 11 chapter 7, article 3, Arizona Revised Statutes. The
agreements which gave rise to the indirect cost charges qualify as interagency
agreements and they must meet the requirements of A. R. S. section 911- 952.
" The interagency agreement provisions apply to public agencies which
include departments, agencies and boards of this state. A. R. S. section 11- 951.
DOE is a department of this state and the authority board and the board of
regents, the governing board of N A U, are boards of this state. A l l three qualify
as public agencies."
The opinion further states:
" Because the powers of the three agencies encompassed the duties of JAG, the
agreements involved the exercise of common powers and were interagency
agreements subject to A. R. S. section 51 1- 952.
" Interagency agreements must be in writing and specify the duration and
purpose of the agreement, the manner of financing the project, the methods
used to accomplish the goals of the project and any other necessary matters.
The agreement must be submitted to the attorney general for the agency and be
filed with the secretary of state or the county recorder depending on the scope
of the agreement. A. R. S. 51 1- 952."
" The agreements concerning ' N AU's role as fiscal agent for JAG were not
in writing, were not submitted to the attorney general, the attorney for the
advisory board, and DOE, and were not filed with the secretary of state.
Because these requirements were not met no monies, including the indirect
costs, could be paid under the contracts. A. R. S. section 911- 952, subsection J."
( Emphasis added)
In the absence of an agreement NAU and ADE did not follow legally required
procedures for NAU to receive compensation for its fiscal agent duties.
Technically, both NAU and ADE could be held liable for repayment of the funds
involved for this statutory violation. However, according to the Attorney General's
Office, because both ADE and NAU agreed on the services provided and the actual
cost incurred, no cause appears to exist for legal action in this matter.
Agreement needed to govern transfer of JAG property - The lack of interagency
agreements also affects ownership of program property. In the absence of a valid
agreement, N A U cannot retain ownership of JA G purchased furniture.
NAU is claiming ownership of all JAG furniture based on an informal agreement
with ADE. This agreement states that any furniture or property purchased on
vocational education grants and the Jobs for Arizona Graduates grant has its title
vested in the Department of Education for the first three years the project is in
existence. If the project lasts more than three years, the title to the property will
be relinquished to the university. The JAG program has been in existence for eight
years. Therefore, N A U is claiming ownership of all JAG furniture.
According to a November 1987 Legislative Council opinion, the informal agreement
between NAU and ADE does constitute a written contract. However, the opinion
concludes that:
" the agreements between NAU and DOE and JAG are subject to the
requirements of A. R. S. section 511- 952. This section allows agencies to dispose
of property pursuant to an interagency agreement, . . . The contract was not
approved by the attorney general or filed with the secretary of state, however,
so i t is not a valid interagency agreement under A. R. S. Section 911- 952."
Current Organization Under
A DE Improves Control
The responsibility for JAG was returned to ADE effective July 1, 1987, which has
streamlined program oversight. ADE is now the only entity responsible for JAG
operations. NAU no longer serves as fiscal agent for the program and the Authority
Board now serves only in an advisory capacity. Thus, the fragmented responsibility
which contributed to the problems described in this Finding has been eliminated.
RECOMMENDATIONS
1. Intergovernmental agreements should be used by JAG and ADE when required
by law.
2. NAU should either return all JAG property to ADE, including furniture, or
obtain a legal agreement for its proper disposition.
C. DIANE BISHOP
Superintendent
~ rpnrtrnent of pbucation
1535 WEST JEFFERSON
PHOENIX. ARIZONA 85007
( 602) 255- 4361
January 28, 1988
Mr. Douglas R. Norton, Auditor General
State of Arizona
2700 North Central Avenue, Suite 700
Phoenix, AZ 85004
Dear Mr. Norton:
The revised final draft of the performance audit of the Jobs for Arizona Graduates
Program has been received and reviewed by myself and appropriate staff members.
In my correspondence to you on January 15, 1988, I indicated the five areas which
needed further clarification in the final report. During the exit interview with your
staff, those items were discussed at length. Upon review of the final draft report, I
note you addressed some of our concerns. However, further clarification is needed.
In just a few months, our Department has implemented an expanded initiative for
JAG which, for the first time since the program's inception, includes a major
financial ( buy- in) commitment for local educational agencies. Professional staff
development for Job Specialists, new curriculum thrusts, expanded partnerships with
business and industry, and an improved student assessment system is now in place.
Recommendation - In Arizona, there are a growing number of students who need a
school- to- work transition initiative and that initiative needs to incorporate the
positive program thrusts of programs such as Jobs for Arizona Graduates, i. e., on-the-
job training activities, business and industry linkages, instruction designed to
meet the needs of " at- risk" students, a youth organization, and parents as key
partners in the career selection process.
The Department's efforts need to be highlighted. A program which emphasizes
school- to- work transition warrants a'new time frame in order to demonstrate i t s
viability.
Mr. Douglas R. Norton
January 28, 1988
Page 2
I request this correspondence be included in the text of the published report. The
following five major issues are highlighted:
1. " At- Risk" students
2. Program Costs
3. Curriculum
4. Part- Time Employment
5. Job Placement
1. Definition of " At- Risk" Students - The report should have focused on a broader set of
criteria and references in defining the " at- risk" students that Jobs for Arizona
Graduates ( JAG) served 1985- 1986. As stated on page one, Jobs for America's
Graduates was started in 1979 as a means t o use research and demonstration
activities as a national strategy to reduce youth unemployment. As a n arm of Jobs
for America's Graduates, the mission for the Jobs for Arizona Graduates program has
been to reduce youth unemployment in Arizona. The question then is: Who are the
unemployed youth of Arizona? The resource used to answer this question is the
report: Labor Market Information, Youth in Arizona, being published by the Arizona
Department of Economic Security Research Administration in cooperation with
Arizona Department of Education, Vocational Education Division, June 1986.
Minorities - " The minority unemployment problem is not only evident at the national
level, but at the state level as well. Unemployment rates among minorities in
Arizona are very high - black teenagers, 25 percent; native Americans, 26 percent;
and Hispanics, 18 percent. This high unemployment rate is regarded as a consequence
of financial difficulties and lack of educational competencies." ( D. E. S., 1986)
High School Dropouts - " The group with the highest overall unemployment rate in
Arizona among youth is by far the teenage- dropout group at 26 percent.
Furthermore, nearly 80 percent of the youths who were employed in Arizona during
1984 were high school graduates or had some college education. Since a higher level
of education- attained has a positive effect on employment, the outlook for minorities
from this perspective is not encouraging. In 1984, the highest dropout rate, 17.6
percent, was registered by Native American enrollees followed by Hispanics with 15
percent, and Blacks with 14.6 percent. In Arizona, dropout rates among minorities
are several percentage points higher than t h e s t a t e overall rate. There is a direct
Mr. Douglas R. Norton
January 28, 1988
Page 3
relationship between groups with high unemployment rates and those with high school
dropout rates." ( D. E. S., 1986)
Living in Poverty - " Living in poverty is an important factor affecting youth
employability. The location of their residence and accessibility to a competitive edu-cation
play major roles in the effective preparation of youth for entry into the labor
market. Nearly 50,000 youths lived in poverty during 1985 in Arizona, but the propor-tions
of minorities under the poverty level was much higher than their representative
proportions in the population." ( D. E. S., 1986)
Combination of Factors - " Youth is a heterogeneous group in our society and their
high rate of unemployment cannot be attributed to a single factor, but to a
combination of factors such as lack of communications skills, inappropriate attitudes
and behaviors, lack of labor market information, early parenthood, being
economically/ educationally disadvantaged, or part of a minority group." ( D. E. S.,
1986)
Demographic Report of 1985- 86 - Jobs for America's Graduates prepared a report
entitled: Characteristics of Participants in Jobs for Arizona Graduates, Class of
1985- 86. It was prepared by the Center for Labor Market Studies, Northeastern
University, Boston, Massachusetts. This report was available to the audit staff.
The Demographic Statistics for the JAG 1985- 86 Class Are as Follows: - Racelethnic
composition: 49 percent Hispanic, 7.9 percent American Indian, 7.9 percent Black for
a total of 59.8 percent minority students served.
Proportion of JAG participants living in poor or near- poor families: 48.3 percent.
High school curriculum of JAG 1985- 86 students: general academic, 84.3 percent
Junior year grade point average 1985- 86 students: C to D and below, 84.8 percent.
Probation/ suspension/ expulsion status of junior year: 26.8 percent of JAG 1985- 86
students.
Days absent during junior year: 6 to 21 or more days: 67.5 percent of JAG 1985- 86
students.
Senior year employment and fall 1986 plans: work only, 30.7 percent;
Work plus school: 47.2 percent ( JAG 1985- 86 students)
The audit staff ( page 7 - Report 88- 11 refers to Jobs for America's Graduates
literature regarding the target group ~ f students to be served without considering
Mr. Douglas R. Norton
January 28, 1988
Page 4
constant reference to demographics of Arizona JAG ( 1986 Annual Report, 1986 Jobs
for America's Graduates, Accreditation Report, and Five Year Research Report).
These reports were available to the audit staff.
The audit staff in choosing an " a t - r i ~ ks~ tu~ d ent criteria did not include in their
definition significant demographic factors as minority, poverty level, academic
performance, absenteeism, and nowhere evaluated " a combination of ' at- risk1
factors." For example, a JAG student who is Black with a D academic average and a
high absentee rate is certainly " at- risk" of dropping out of school. I t is clearly stated
by the Department of Economic Security reports and statistics that a student who
drops out of school is llat- riskll of employment. On page 11, in fact, the Auditor
General report states that, " During 1985- 86 approximately 31 percent of the total
1985- 86 JAG student enrollment was not available for placement, and most of these
unavailable students had dropped out of school during the year." It is very, very
difficult to understand how the audit staff can s t a t e on page 8 of the report that using
their criteria 11 percent of the 1985- 86 JAG students were actually " at- risk" for
unemployment. The report states ( page 7): " In order to operationalize this definition
of ' at- risk,' we chose individual 1985- 86 student participants who had not taken
vocational education classes, had no special employability skills, and had not been
previously employed. For our analysis, students with these characteristics are defined
as " at- risk." This limited definition of ' at- risk1 accounts for the 11 percent figure and
for the tremendous discrepancy between the Jobs for America's Graduates 1986
Accreditation Report and Report 88- 1.
Jobs for Arizona Graduates ( JAMG) - The evaluation team of Jobs for America's
Graduates Accreditation team spent three days of interviewing ( both in person and by
telephone) and reviewing the Jobs for Arizona Graduates files and records while they
were present in Arizona.
The evaluation team was able to talk to ten of the fourteen principals of the high
school where the program operated. The report states that: " without exception the
principals were very favorable to the JAG program."
A physical count was made of the placement files a t the main office and no excep-tions
were found to indicate concern when the numbers were compared with the
September and October, 1986, month- end reports. Additionally, the evaluation team
Mr. Douglas R. Norton
January 28, 1988
Page 5
telephoned 28 program cornpleters or graduates of the Jobs for Arizona Graduates
program and the graduates were very favorable in their remarks.
The Arizona audit team conducted their review after the DrocEram was no longer
functioning, after job specialists had been terminated, and after files were of little
concern to the schools as they viewed the program as being terminated.
The JAMG 1986 accreditation report states: " Jobs for Arizona Graduates is operating
in full compliance with the standards a s s e t forth in the JAG school- to- work
transition model. " The student selection Drocess in Arizona amears to be in close
accord with major features of the JAG model." The report then goes on to detail at
length: the demographics of the students stressing academic achievement which was
very low, stressing the composition of the minority group, stressing the poverty level
of the students involved.
Concluding Remarks - Definition of " at- risk" - Using a narrow definition of " at- risk"
for the report failed to reflect accurate cost per student ratios and diminished the
impact the JAG program was having on " at- risk" students.
Program Cost Comparisons - The audit report compares JAG program costs
( pages 14- 15, Report 88- 1) to the cost of operating Mesa Jobs for Youth and
Vocational Education.
According to Jobs for America's Graduates Five- Year Research Report: " Findings
over the five- year period, 1979- 1984, are t h a t t h e cost of one student placement
( including nine months of follow- up) averages less than $ 1,400 ( half of the JTPA
average). The correct comparison for JAG is JTPA program costs as JTPA serves
youth of similar demographics; i. e., poverty level, minorities, etc. JAG program costs
could well be compared to the cost of the ramifications of being unemployed; i. e.,
being institutionalized, suicide, substance abuse, or incarceration. JAG is an
extremely wise use of dollars spent when making those comparisons. The audit team
comparisons to Mesa Jobs for Youth program includes cost factor ratios based on
" full- or part- time, permanent or temporary positions" ( page 15 - Report 38- 1). A
question: How many of the " temporary" jobs were babysitting placements? Who
fo:: ows up on student placements to see how many months the students stay on the
job? Getting a temporary, part- time, or full- time job is one thing, but a youth staying
on the job is another matter. Generally, an unsupervised ( no school program) youth
( 16- 19 years old) quits a job within three months.
Mr. Douglas R. Norton
January 28, 1988
Page 6
Vocational education program completers are youth who have made a " career
decision" and that makes the difference. According to the reams of literature on
" career undecided youth," they are in fact the target population of JAG. In large
measure the career undecided youth is the general academic track student, the low
academic achiever, the poverty, minority youth combination. This could also be an
answer to the discussion of program effectiveness and low pre- and posttest gains. If
students are " career undecided," they are not motivated to learn curriculum content,
to score high on tests, etc. Here, is where the JAG program could possibly be
improved. I t is the intent of the JAG Education Program Specialist with the Arizona
Department of Education to conduct experimental research during the spring of 1988
with the present JAG students. They will be administered Osipow's Career Decision
Scale and given a treatment to include the completion of a career pathing document
to improve their ability to become career decided. This in turn will improve the
number of program completers.
Concluding Remarks - Program Costs: The program cost comparisons used are
incorrect due to unlike student populations and services.
3. JAG Curriculum - The description of the JAG curriculum ( page 3, Report 88- 1) is
accurate. Vocational cooperative education and work experience programs in Arizona
as well as other job placement programs have a similar list of curriculum com-petencies
as JAG. Additionally, all the literature regarding " at- risk" youth details a
similar course of study. If a new school- to- work transition program is started as the
Auditor General suggests, dollars would once again be spent developing the same cur-riculum
or one extremely similar and that would be a waste of dollars and time.
If the JAG curriculum is a competency based one as stated in the JAG curriculum
guide, then other performance criteria need to be used to evaluate its effectiveness;
i. e., completed application blanks, videotaped interviews, etc. Making a judgment on
- one paper and pencil test is just that, one indication of curriculum effectiveness.
What needs to happen to improve the effectiveness of the JAG classroom instruction
is to administer a learning styles inventory such as the DUNN and DUNN to determine
a learning styles profile of JAG students.
, -
Based upon that learning styles profile, the instructional style that is most effective
for that learning styles profile needs to be implemented by JAG Job Specialists when
delivering curriculum competencies.
Mr. Douglas R. Norton
January 28, 1988
Page 7
All the literature on youth unemployment suggests that another problem for this
group of youth is " feeling unconnected from society. They are lacking in feelings of
potency, self- worth, connection with others, and a concern for the common good."*
The Auditor General's report does not address the Career Association that is part of
every JAG program. The Career Association activities are designed to connect the
students to the larger society; i. e., adopting a needy family at Christmas time. Your
audit staff could have identified this strength by interviewing former JAG graduates
as the Jobs for America's Graduates accreditation team in writing their 1986
accreditation report.
Concluding Remarks - Curriculum: I t is our opinion t h a t t h e measure of the
effectiveness of the JAG curriculum was too narrow.
4. Part Time Jobs - Many part- time jobs were combined with continued schooling.
According to the Jobs for America's Graduates 1986 Accreditation Report, 15.3
percent of the 1985 JAG nongraduates were combining part- time work and schooling,
6.8 percent of the 1985 JAG nongraduates were just employed part- time. In many
instances, part- time employment must be the case for JAG program participants.
This particular group of students is the low academic achieving group and many are
forced into fifth year schooling or GED preparation. When the JAG program
completers do go on to further schooling, it is at the community college institution as
opposed to a four- year college. This can be verified by reading JAG Job Specialists'
JAG 8- 9 reports.
The files that your staff used for their reporting have not been available to the
present JAG Education Program Specialist to determine details regarding part- time
employment for the 1985- 1986 JAG graduates. However, the Jobs for America's
Graduates Accreditation Team did survey 1985- 86 JAG students as to their plans
after high school and 47.2 percent of the students stated that they would be
combining part- time work and part- time schooling.
Concluding Remarks - Part- time Jobs: I t is our opinion t h a t t h e characteristics of
the student population served by JAG is the most prominent predictor of job
placement, not the program operafion.
* Wircenski, Jr. L. Ed. Handbook of Special Vocational Needs Education
Mr. Douglas R. Norton
January 28, 1988
Page 8
5. Job Placements - The mission of Jobs for Arizona Graduates as stated previously is
to solve youth unemployment; i. e., g e t t h e JAG student a full- time job. Previous
employment as stated in the Auditor General's Report 88- 1 is a strong foundation for
future employment. Students with low academic skills are less likely to be hired for
jobs above minimum wage ( D. E. S., 1986).
Concluding Remarks - Job Placements: We believe encouraging the JAG program
completer to continue part- time schooling is a solution t o higher future wages but
may not see fruition for several years after a school- to- work program.
Recommendation - In Arizona, there are a growing number of students who need a school-to-
work transition initiative and that initiative needs to incorporate the positive program
thrusts of programs such as Jobs for Arizona Graduates, i. e., on- the- job training activities,
business and industry linkages, instruction designed to meet the needs of " at- risk"
students, a youth organization, and parents as key partners in the career selection process.
In closing, I wish to thank your staff for their efforts in carrying out the mandate of
" Chapter 334, Subdivision 77 of the 1987 Session Laws which directed you to evaluate the
program and prepare recommendations.
Sincerely,
C. Diane Bishop
Superintendent of Public Instruct ion
OFFICE OF THE PRESIDENT
January 26, 1988
Mr. Douglas R. Norton
Auditor General
2700 North Central Avenue
Suite 700
Phoenix, AZ 85004
Dear Mr. Norton:
We received your performance audit of the Jobs for Arizona Graduates
program on January 23, 1988. Since that time we have reviewed the final
draft copy made available to us and we have developed responses by Northern
Arizona University for and on behalf of the Arizona Board of Regents.
Our responses are attached to t h i s l e t t e r .
Sincerely,
% ePre? sid. enet
EMH: pv
Enclosures
NAU Box 4092 Flagstaff, AZ 860 11 ( 602) 523- 3232
NORTHERN ARIZONA UNIVERSITY
RESPONSE TO
A PERFORMANCE AUDIT
OF
THE JOBS FOR ARIZONA GRADUATES PROGRAM
January 25, 1988
The responses are developed and referenced t o each page of the
report in r e l a t i o n t o the f i n a l written d r a f t on The Performance Audit ' of the Jobs f o r Arizona Graduates Program signed January 22, 1988 by
Douglas Norton, Auditor General.
1.) Appendix list of t a b l e s and t a b l e 1, page 4 includes the word
unaudited. The word unaudited is in c o n f l i c t with the statement
on page 6, second t o the l a s t sentence which s t a t e s " t h i s audit
was conducted in accordance w i t h generally accepted governmental
a u d i t i n g s t a n d a r d s . " A s was discussed in the meeting on Wednesday,
the word should be e i t h e r : ( A ) removed, or ( 3) changed t o read
audited, or ( C ) l e f t t o read as unaudited but footnoted with an
explanation as follows: The word unaudited is used because t h i s
p r o j e c t d i d not include a f u l l audit as defined by the Auditor
General S t a f f . This project was subject t o a u d i t s and was included
in each Annual Financial Report during the periods t h a t t h i s project
was a c t i v e . A limited audit was conducted by the Auditor General
Staff recently t o examine the accounts which included a review of
vouchers and v e r i f i c a t i o n of some c o s t s .
2 . ) Summary s e c t i o n , second paragraph from the end of t h a t s e c t i o n ,
second sentence from the last the word compensation is i n c o r r e c t .
I, The c o r r e c t word is reimbursement.
3 . ) F i r s t paragraph on page 17 and on page 18, f i r s t , second and t h i r d
paragraph and any o t h e r a r e a i n the report where the word agreement
is used, it should be proceeded by the word intergovernmental.
The reader of t h i s report should not be misled t h a t we did not have
an agreement. We did in f a c t have a written agreement. We did not
execute an intergovernmental agreement. Consequently, on page 17,
the f i r s t paragraph and any other place i n t h i s report where the wording
s t a t e s " following l e g a l l y required procedures" should be deleted and
changed t o read " did not execute an intergovernmental agreement".
4 . ) The last paragraph on page 17, second sentence from the l a s t , needs
t o be deleted. This sentence s t a t e s " NAU Administrative Staff saw
t h e i r r o l e a s t h a t of the " banker" f o r the program, though t h i ~
role was never defined by a formal agreement." This sentence is in
c o n f l i c t with the sentence preceeding and is i n c o r r e c t . We were
delegated and assumed the r e s p o n s i b i l i t y of f i s c a l agent. We did
provide f i n a n c i a l record keeping, personnel administrative s e r v i c e s ,
the use of our purchasing department, property control department,
accounts payable department, c a s h i e r s , computer s e r v i c e , p a y r o l l
department, grant and contract administration, sponsored project6
accounting department, NAU s t a f f attendance a t Authority Board
meetings, and a l l other r e s p o n s i b i l i t i e s f o r carrying out our
f i s c a l requirements. Further discussion of the f i s c a l agent
agreements are discussed in the next response i d e n t i f i e d as f i v e .
5 . ) In paragraph 1, page 18, second sentence from the l a s t , s t a t e s as
a r e s u l t more than $ 367,000 in i n d i r e c t and administrative cost
reimbursement was paid t o NAU in an unauthorized manner f o r
f u l f i l l i n g the f i s c a l agent r o l e . ( A ) This should be changed not
only in paragraph 1, but 2 and 3 aleo t o read during N A U ' s tenure
as f i s c a l agent it charged $ 367,202. ( B) The word unauthorized
used in paragraph 1 and 2 needs t o be deleted and the f i r ~ ste ntence
in paragraph 3 needs t o be changed t o a written agreement and t h i s
project was conducted on a continuing basis in cooperation with ADE,
NAU and Authority Board. The reason f o r deletion and change in
wording can be supported by written agreements. Attached are the
four written agreements which are i d e n t i f i e d as e x h i b i t s 1- 4, f o r
the period July 1, 1980 t o t h e ending date of June 30, 1986. Each
one of these agreements have an authorized agent signature f o r NAU
and Carolyn Warner f o r ADE as authorized signature f o r the statement
t h a t s t a t e s : " Signature s i g n i f i e s t h a t agreement and assurance& are
on f i l e a t the ADE and t h a t the Applicant Agency agrees t o a l l
conditions s t i p u l a t e d in t h i s Application and/ or Amendments, t h e r e t o . "
In addition, we are providing f u r t h e r proof of N A U ' s agreement as
f i s c a l agent, see attached copies of l e t t e r s from Carolyn Warner
dated July 1, 1980 ( Exhibit 5) and May 19, 1982 ( Exhibit 6 ) . Attached
is a copy from President Hughes dated July 19, 1980 acknowledging
r e c e i p t of letter and accepting r e s p o n s i b i l i t y ( Exhibit 7 ) .
6.) In paragraph 3, page 18, f i r s t sentence, and throughout the r e p o r t ,
I, d e l e t e " with JAG", t h a t is the program t i t l e . In l i e u of the word
" JAG", use Authority Board.
7 . ) In paragraph 3, page 18, t h i r d sentence change the word " budget" t o
a read " program expenditures". The 5% i n d i r e c t c o s t was charged against
a c t u a l not budget amounts.
8 . ) In paragraph 3, page 18, as w e agreed in the meeting on Wednesday, the
word " intangible" should be removed. We agreed t h a t we had previously
used t h i s word i n c o r r e c t l y .
9 . ) Sentence 2, recommendation, page 20. NAU response w i l l be as follows:
&
NAU does recognize the need f o r reviewing t h e need f o r an
intergovernmental agreement t o provide f o r the d i s p o s i t i o n of property.
In conclusion, we would l i k e t o thank the Auditor General and h i s ~ t a f f
f o r allowing us t o present f a c t s , changes and responses t o inform the
reader of t h i s report the r o l e of Northern Arizona University as f i s c a l
agent f o r the Jobs f o r Arizona ~ r a d u a t e s Program.
E x h i b i t 2
- - -
ATTACH TO APPLICATION DUE DATE: AS NEEDED
- . - ."- -
- -
1. - p r o j e c t S t a r t i n g Date: J u l v 1. 1980 t o Ending Date: Septerber 30, 1084
-
2. APPLICATION [ 1 AMEXDME3T [ XI
3. Progran ~~~ d source- S t a t e Vocational Educa- tion-
( Examples : ESEA T i t l e I, Consurcer iiomemking , t Y)
4. P r o j e c t T i t l s Jobs f o r Arizona Graduates
5. Applicant Agency Jobs f o r Arizona Graduates .- I
( 602) 255- 1750
D a t P , / / < s ignat- re- Authorized Agent /
I I
-- t,. -- r- r_ ol ecr. -
7. - udddau~& i g ~ ~ ~ ~/~ p-/ AtA+ t /~$ tA hoeonrt i& z ebd d DatP,/// / ' / FfJ
8. D r . Frank B e s n e t t s , Vice F r e s . f o r Adziin. & Finance, Northern Arizona Univ.
I
Typed Sam and T i t l e :
S i g n a t u r e s i g n i f i e s t h a t a g r e e E n t and assurances a r e on f i l e at t h e ADE
an< : ha; the Applicant Agegcy agrees t o a l l c s z d i t i o n s s t i p u l a i a d i n c h i s
A p p l i c a t i o n and/ or Amendmnts, t h e r e t o .
STATE EDUCXTIOX USE OtLY
9. P r o j e c t ? Ic~ krr 81- 105- 08- XX --
10. lfaintenance of F i s c a l E f f o r t 1
Second Preceding Year $
F i r s t Preceding Year $
I
T0T. V. & fOUNT APPROVED $ 49947* 707.11
~ g r o g Aru~ th. o ri. z. . e$ , AgenL, Date of S i g n a t u r e L I . . l>" I S
xz- tk--:;. . I . : 8 ,. .* 3: zv~ cf, azd state;
Signature :,,- 1- , - , - .
Signature:
ADE 40- 100, REV. 9/ 77
*. e-,.
t / . 1 761 ' / I 2 < - - "
E x h i b i t 3 . . .1 . - *-
- ---- LC&-- bd -- . -:-. --.. , -
.. _ _ _ - - PROJECTA PPLICATION! A~~ ENDMENT !. ^ _ _ _--- _ - . - . 1:.
- - _ _ . - _ - _ COVE?, SHEET -- a - -- -.- -- . - -- - - ,, -.- " r = - . i -- - .- . - I
-
,< .----: -- - - I G - .*: . . -.. 1 -
i....-. . - C_ 14 ' DUE DATZ: AS NEEDE;
. , . 2 ~ A T~ O ~ AP~ PL~ ICA TION % fJ 5:- *- -_
- . -. . - . ^_ _- - - I Z . . -- . .-.-_ - *,> G .. .- __- - - - - - - - - - - - - - r" - -- ID& . - , -_ _ _ > . . . - . . - - - -- --- -- .-- ---.-.--* : 1 . - Project Starting Date: Ju l v 1 . ? 0 e ' to znding Date: . Tlln P ? 7 0 Q
-!& a -<. 2- z.-."$-%" - > . .- -*
-.% . . - -- - - 7 . - . ,. A_. - - . - -- -- . % - - __%. i - ... _ +- -." -- . *--, --- - &.. 2. AppLlCATlON [ ] . AhqiNDMEST [ 11 -- -'
.- -- . - -, > -. - - -
a- *-- - 3. ProgramFundSource s t p t s V C = ; : ~ . G = E L- - Z C. Z Z Z Z ~ " "
~ ~ ; i n 3 3 i e sE: C I; i-.:=: 1r, vso catlonai Educatloni
5. Appiicanr Agency Jabs f 07 - 5- z; 2 z z z C ~ Z ~ Z Z:: ?
6. ProjectDirector Dr. J z t e I:. Oc? zCa.' IJy. D TeiephoneC50?) 7 5 5 - 1 ? 5 O
r 4
tpldE% Date /// JLL-/~ Y
Signarure- Au;-; onx- A ~ ? n t
/
& Dr. Frank Besnette, Vie. ?:+- a i2ezt f 3 r- i f - i n . & F i z z n c e I i v x a E ! a m e c n r 1bZ3 y o r t h e r n A: iroena Universi
!
$ i
Signature signifiefihat agrrsrncn: ~ n c i; 3 j r z n c r r 2 ;: 31 f i e at the ADE and ; hat the Apl: lic:-
I Agency agrees to- all conditions s ~ i o u i ~ r ein= ; his A p p I i ~ z i 2 na ndfor Amendments, thereto. . .
-
STATE EC! UCXTIOI\! US5 3\ NLY
9. Project Number s I - 7 n 5 - n =-:;- I - .-
i 10. Maintenance of Fiscal Effort. . --.
Second Pr-- rz: ng ' fear S
L -' . . F i t s ~ ? r = , c = -? Year S . .
I -
11. T" TA1 AMOUNT APPROVED S 5 . : ) 7 . 7 . > 7 . 1 \ ! b.. . - . .- . - -
, .. . -. - : . .
a. I I , , -, , . . ... , -..- I 2. Signature: - A- - ~ a t ief S ignature
pmc-<= A& og* a;;* 2%. --- zzc -
-----% I t i
\ \ \ \,. 5 -
Dare oi Signature
- -
14. Signature:' ], I9 ,/ p Date of Signarure
. - - - - - ,. .. + - - - . - - . ' ) \
L -- - 1:; . ' . - - -. ,-.
- - C...- \- .. i > ; .. *
-\\:. \ 3:- 54*- ADE 40- 100, Rev. 1 I82 -- -
.. . - 4 -- 5~ 7-
Page 1 of
- . ... . . . ... .- I ; .:..: . . ...: . . - . ... -. - . - - - d' ' ..;. . - ... . a . ~. . ~. .. - - . - . . -- . . . .-- --- -. ..: - . . . . - 4 , '- 5s.-<; y+ F 2 i ;; L YJ ;:...,..-.._;... I- i- - . . . -;- . . . . . . . . . . . . . . . . . : . -.. - __._._ _ .-_ _. ..-_ _.-. - . . . -. -. . . . . - . . - . r-..,__. _ .... - .
. ,-> e ..-,;. . . . . . . - _ _ .... . , . , . . PaOJECT APPL\ CATION/ L? AEND~ V~ EP~ T ' .... ? - - .: ' T : ma,-
A -. . ; . ,. - :. '- -_ _- .- -_ - / - -- * , J ;;. J -:,:.--?: a
. . . .~. . . . . . . . . . . . . . - . __ .-,. *- -.> .+ *. . a,& >. A*. , ..-. --... - .-... . . . . -;?: ..,-. - - ..;-.. ; rr-;.:+':. I. . . . . .. ' , . . COVER SIirET . - .-
, _ . -:.. - - ? = - -.-..-. . -. 3 . , : . - . : -
.._ A . . . - . . - . - ? . . - - . ! . . . . . . . . . . . - Exhibit 4 . 7::; , ,; ;-~:;;~;-: 1:- ~.'-- :~-/;' - : . - - . . . . . . < - . -.. . . . . . . . . . . . - . . .- -.--, .- .,------ - . ,' ..: , - - ..- .-.. -- - . * . . . ~---- : .- .:--..,.. ---- r, c7-~.. ,... - -.. .- . --, .___-^ I.-= . . .. .. .. . .- ... .. .. . . - . ,=.- *&+,, CHTOA~ p~' lCATION7 -- 7-" 1-.:?. 5.- 1 ---..;- -&?.'-'::; 7:.-- 5.. z.......'
. ..- -- a:- T DUG 9ATE: AS N- E- E DED -
... ._ . .- ..--.- -.-.-- Y -. . . . . . . . .
.......... , $ . . , - . -- , . .? ., . j>.;-. L<. . . . . . . . . . . . % , , ,,,-.,, .'* ...- .'-- - . '. ...... ;.-"'. .......... . . . . -. . .- - . - -..----.- - ... ---.-* . .... ........ ---. ...--.. .. .-. - .- .,-- .- .-. .. . . ...... . -. -- . . - - - . . - - -. .-.-. ... .-.-- . ...-.-. .- .-.... --- -.?-?-+ 7 "-- . zL, 7- p-.:,.*. :,-=:.. ,&*. --.----;: .,*; ..--, . i. . . . . . . . . . ? 1 . .... . . . . . . . . . . . . . . ... ~. . ;. - Xe?---. -=:-...- I - -- .... - .=...+-&- , x. :'- ... ...... - -. -. . :"' July 1,
.. -
'. .~ k" r .; 1985 -:--'-.'..:'? -"---- t&~ r, djnD~ a te: - June 33, 1936 . -( I -- .. ---. ..! :. -~ .- I ;; Projec, t Starting Date: "--
z . . . . - .- . . . . . . .
..>--.. c<,. A .*---- 5 -- w.. > . .,.* .--.. * .. ..- : .. - : ?, : .. . . . . . . . . . . -- , - 7-
I. : . . -- . . : .. ..& --> . .-.-'*. ., - - - + =- .- .-.:. . di %_._ - - .. .- . . . .. ... . .
."..._"..- m_ L---.-=-.
- - .,_-._.'. _- . . - .,_ . -.- . ... ... .
...'..*+ ~.~. .,, 2+-++ p? LiCATIO:;[ N-.. ] G-- ._ T. -: VSX. . -.: . AM E ND , ~ ENT . .. [ X ] ...........- 2 - ._.... : f__>._ . - $__ ~_~ : .:... ~:. ...~..-. .~ .;:;. .,.~. .. k. .... ,...: - ... .-. . _?_ _ . . - -. ;>~. x~:> 5y~~..-.. --.-- .-- -< k*.:..; -. .. +-: --. e .-- .. .-., ---.. . . . . . . . . . - . . - . ..... q- .... --- ... , - _. . , . < -;-- r . :- . .
*......-:...- . I .. *." -* - - , , .. ? . . . .. .. ............. 2 .... 1 . . ....... 2 - - . . . : . . . . -. .... .. i- --:. - . . - ..
;;~ a:+- s~::; 3. : p rogram ~~~ d squrceAS t- 2 te Vocational Education - . - -..---. I.--::.. . . . . . . .
.:.: :- s---- - .-.- -- .- -. -._-. -_. _.._- . .._- --. .- __. . . . . .-. ... . . . .-. ir. r -----,.... -.- . . . . . ..* . . . .., . - - ( Exarr. r; Les: EClA Ckapter 1, Vocztional Educatlcn)
. L _ l _..__&.__ - . ~
a
. . . -. ............. ..... - -- ..- .-- . - . -.. 4. proiec;~ itle J O ~ foSr Arizona GraGuates \ ~
I . - .
-- - -- - - 5. kp; iiccnt A~ enc. f Jsbs f cr Jrlzoc- Gradzzz= s ' - --.. - - . .
- -
---. - 6. -- Prajecr Director D r . J'Ae K. Sckade, Ph- D. i. \
Telephone 255- 1 750
--.- -
a
-- --- ----. --- qU# ....-.. --------- -* - - 7 -- - - Signa? iire- Au~ hcrized% ~ fit
- - - Charles W. 3alGwi. r I - -
8. , -. his. jociateV ice P r e s i d e g t for Admin. & Finace
Typed blame an0 Title them k i z o n a u n i v e r s i t y . I
- - .- i
. Si~ narures ignifies that a q r e m e n t and rvurances sre on file at the ADE and that the Applicant
Ayficy agraes to ail conditions stipulated in this Applicztisn md/ orAmendmen: s, thereto.
4
STATE EDUCATION USE ONLY
9. Proiect Rumber 81- 105-~ 8- 1~<' . . .. .
10. Maintenace of Fiscal Effofl II
Second Preceding Year S
Fir:: Prcccding Y e : S
s- .,
I I. TOTAL AMOUI'IT APPROVED S 5 , 7 4 7 , 7 0 7 . 1 1
- - -. , f . />>- 4
12. S; gr: ture: /.=- . I,
- -. - -.- - , . ~ i p 3 3 r r & ~ k ~ ~ ~ I rbL' ~ 8C~ 3~ t, 1 3 - I- - - . -... Dar o i S; sniirure
- ~ azh,, z2tical~ c o r r g c t ; z d s tzta=: -\ 1.; Siyarure: ~ e n ct;' ~ ss- x-=- lccsi s c. a 1113-
- - .- - -- a
-------- - . 1*- 1a. .- Sign2tur5: - s- . - - - - .
D ~ : o l o i S: a_ narure .... -- - -... -- - .*.- - - - -. - -- -. ...- _-. _.. _. . - - -. . . . ^. . - I - - ) --.. . -+= -.. - -- LZkTiI\; G2? i- i ~ F C NRE CEIFT 06 -- ;:\. .-- 7: :-:- - I _-. _ _ - 1 _-_ _., . - .. - .-- . " - . *; . . . - - . -- - - r * - - * - - - - . -- --/-,--. :--- - - , ' ..- -. . . . - . .- -. - EaEi;,\ L Eu;;.; cs - - . --. _ . . . ' % -- .-- < . . f i . - - _ - " -- -< - *.-
'- 1;. --*_- I-.-_. ' . . . - ;, . ; 5- .--- - - '. .- * . , - - - . - .- - -*.. - - - - - . .- - \ 3--- 2-- < '. . A .- - - -- -- _-< .- . -- - -.. - - . . - .- - - - -- - -. -* - - - . .-,- - - < - - .- . 13- I::?, E ~ v .. j 2 . : . -- - -. -- - - - - -- -- 9252 1 of - ..... - -
- -
- - - . . -
- -----. - a
-.-- - - . . . . -- - - . .* *--- - - -------- --- "-
- - -- --
-. .
I I
- 6 -
July I, 1980
. .
( . aepnrintmi of FSlrcaiinn
1535 WEST JEFFERSON
D r . Eugene M. Hughes
- Pn - or; eths? iml e n tA rizona U n i v e r s i t y
? l a g s t + f r , AX 86001
Dear D r . Hughes:
Ple a p p r e c i a t e your senins a s t ! e f i s c a l agent f o r t i e 205s
for Arizona Graduates pro9ram.. . I would r e q u e s t that D r .
Fobelrt K e r ~ o o do f your sta. 52 . b e a s s i ~ n e da s p r i n c i ; ? a l i n v e s t i g a t o r .
The progran has r e c e i v e d i n i t i a l p o s i t i v e r e c e p t i o n and
should prove h e l p f u l 1. to Arizona.
I a p p r e c i a t e y o u r h e l p .
L
S i n c e r e l y ,
Late Supexintendent
vh
. -
-. .
1535 WEST J E F E R S N
PHOENIX. ARIZONA a5cc7
2 7 1 - 4 3 8 1
D r . b g e n e H. Bugies, P r e s i d e r t
h'azther3 _ 4_' 1zo32 Eniversf ty
Boz 4092
F i a g s ~ ~ P. f2 , 86011
E x h i b i t 6 -. f
Dezr D r . Eugies: I
I
' i" 7Fs i s to i r i o m you t h a t the contract b e r ~ a o 3t he
L
k i z o n 2 S i z ~ eB oard of Educztion az? the G n i ~ z d
s r r i h j D e j z z e n ; of Labor for t h e i r p ? a e a ; s t i o n \
of Jobs f o r P ~ i z o n a Graduztes p r o g r a hzs been ex-toz6eci
t:". r3ugh Se? ce= jer 30, 1S82, v i t h ar: a t i d i t i o c z l
. .
e?? rs? r: e:; on of S10C, G00.00.
I vould 31-: e t o request t h z t S o r t h e m Arizona Ud-v
e r s i v contlnue aczing i n the r o l e of f i s c a l agent
f o r t h i s ~ z o g a .
i
Wrl ara c m t i n u i n g t o seek other sources of funding i n
t h e a t i c i ~ a t i o no f continuing t o operate t h e progrzn
a t its present l e v e l . The cooperative arrangenent
~ 5 t Nkc r t - i - Arlzo~ aU niversity has been n o s t helpful
t o Jobs f o r Xrizoaa Graduates.' 1 look f o r z r d t o con-t-
1" bg tizt vor! dng r e l a t i o ~ s ' h i p .
B
S i n c er el?
cc: Dr. Bob Kerwood . . .
- . _. . .
. - . . . . -. . . ,._ . . . - . - . . . . . ... .. . . .-:.. . .
. . . . . . . - - . , - . ~ . > . . . . . .:... . . ; . .
. . . - - . , . .. A - . . =,... ... .. .. . . . . .
. . . . .. . -. . . - - . .- . .. ,. .. . . . . . . . . .. .. . . . . . L. . . . . . -.. . . . . . :_, . .. .. . . . . . . .-...-.. . . - . - . .. -
- . . . . . . . . . . . ..., . * ..
OFFICE OF THE PRESIDENT
July 19, 1980
Mrs. Carolyn Warner, Superintendent
Arizona Department of Education
1535 West Jefferson
Phoenix, Arizona 85007
Dear Mrs. Warner:
I would like to acknowledge receipt of your letter of July Ist,
concerning the Jobs for Arizona Graduates program.
By means of this letter, I would like to indicate that I am
assigning Dr. Robert Kerwocd, Chairperson of the Department of
Vocational and Industrial Education, as principal investigator for this
particular program.
I was pleased to learn that the program has received initial
positive reception and I do know that it will prove to be of benefit to
the State of Arizona and its young people.
Sincerely yours,
President
7 . Robert Kerrood .
Dr. John Glenn .
November 4, 1987
T 0: Douglas 3. Norton, Auditor General
FROM: Arizona Legislative Council
RE: Request f o r Research and Statutory interpretation ( 3- 87 - 9)
This , memo is sect in response to a request made Qn your behalf by ' tVilliam
Thomson in a memo dated October 8, 1987.
FACT SITUATION:
The jobs for Arizona graduates program ( JAG) was established in Arizona in 1980.
During the f i r s t year of the program, the c e n t e r f o r excellence in education at Northern
Arizona University ( NAU) administered the program with grant rnonies provided to the
university by Arizona's department of education ( DOE) and the U. S. department of labor.
,417 informal agreement between YAU and DOE established that the university would only
be responsible f o r c e r t a i n administrative functions, primarily bookkeeping and personnel.
JAG had its own officers who controlled the budget expenditures. NALl only monitored
and tracked the program finances. In addition, JAG developed its own personnel policies.
Laws 1981, chapter 196 c r e a t e d an authority board to operate JAG.
.4dministrative responsijility for the program was established in the authority board n a d e
up of prominent people from business, industry, government and education. The authority
board hired an executive director and director of operations who were responsibie f o r t h e
day- to- day operations of the program and to implement the program throughout high
schools in the state. These operations included hiring and training job specialists to teach
the students at the participating high schools. In addition, the authority board retained
WAlj as fiscal agent for the program. PJAU maintained this responsibility until fiscal year
1937- 1933.
Lzws 1936, chapter 286 transferred all monies appropriated to DOE for JAG and
personnel from DOE to the c e n t e r f o r education excellence ( CEE) a t NAU. Because of
: he transfer, N, 4U was fully responsible for cversight and operation ol the JAG program.
NAU had this responsibility for only one year. The program was transferred to DOE in
fiscal year 1987 - 1958.
In fiscal years ! 98 1 - 1982 through 1986- 1987, monies were allocated for indirect
costs to reimburse YAIJ for those costs incurred while carrying out its role as fiscai
agent. This amount was informally agreed on by NAU financial staff and he JAG
controller and usualiy amounted t o approximately f i v e percent of the ? rc; grarn1s total
operating costs. This amounr was extracted from federal, s t a t e and privately donated
revenue. No formal agreement established a specific indirect cost pert entage or
identified the specific Y .4 U administrative costs to be reimbursed. According to ' VAU
officials, the five percent amount was used t o cover the basic " intangible" administrative
costs incurred 5y the school in providing fiscal oversight.
CEE, through agreement among NAU, DOE and the authority board, was designated
to provide administrative ( financial and personnel) support f o r t h e program. CEE in turn
passed this responsibility on to the administrative staff of the Arizona center for
vocational education ( A CVE).
During fiscal years 1983- 1984 through 1986- 1987, ACVE charged JAG an additional
amount in d i r e c t c o s t s as its reimbursement for administrative costs incurred. This
amount was intended to reimburse the saiaries of one accountant and one half- time
secretary used to oversee JAG operations. According to the auditor general's financiai
audit s t a f f , personnel costs are usually not reflected as indirect costs in a program
budget.
According to joint legislative budget comrnittee ( JLBC) s t a f f , s t a t e programs
usually do not charge indirect costs to line item appropriations unless the budget line item
lists the indirect cost appropriation separately. The JAG prograrn appears as a line- item
appropriation in the DOE budget from fiscal years 198 1- 1982 through 1986- 1957. Indirect
costs are not specified separately in any of those years.
The auditor general's analysis of budget and financial records shows that this
indirect cost reimbursement was not always exactly in cornpliance with the informally
agreed on percentages. In some cases, these percentages were much greater. For
example, during one fiscal year NAU charged an indirect c o s t o f approximately twelve
percent.
QUESTIONS PRESENTED:
I. g o ( Arizona s t a t u t e s allow an agency to assess a program an indirect cost charge
if no formal agreement has been made?
2. Do the statutes prescribe whether it is proper to extract indirect and
administrative costs from a line- item appropriation if these costs are not specifically
identified in the line- item?
3. May an agency e x t r a c t indirect costs from privately donated revenue?
A NS WE RS:
1. No.
2. See discussion.
3. Yes, if other requirements for payment of the indirect cost are met.
DISCUSSION:
, A. 1. JAG has been under the jurisdiction of three entities. These changes a f f e c t
the validity of any agreement f o r the payment of indirect charges. t a w s 1981, chapter
196 p l ~ c e dt he program under the authority of the authority board. The legislature also
transferred to the authority board one million one hundred thousand dollars original!^
appropriated to DOE. This act expired on June 30, 1982, however.
For fiscal years 1982- 1983 through 1985- 1986 the legislature appropriated monies
to DOE to provide vocational education assistance through JAG. During these years ? AC;
was under the jurisdiction of DOE. This conclusion is supported by the f a c t that effective
Clay 5, 1986 the legislature transferred both the 1986- 1987 fiscal year appropriation and
twenty full- time equivalent positions for JAG from DOE to NAU. Had the program been
under NAU jurisdiction the transfer of personnel would not have been necessary.
From the inception of the authority board to May 5, 1986, the indirect costs have
been part of an agreement between one s t a t e entity, NAU, and another, first the
authority board then DOE. Interagency agreements are governed by title 1 1 , chapter 7,
a r t i c l e 3, Arizona Revised Statutes ( A. R. S.). The agreements which gave rise to the
indirect cost charges qualify as interagency agreements and they must meet the
requirements 01 .4. R. S. section 11- 952.
The interagency agreement provisions apply to public agencies which include
departments, agencies and boards of this state. A.?. S. section 11- 951. QOE is a
department of this s t a t e and the authority board and the board of regents, the governing
board of VAU, are boards of this state. All three qualify as public agencies.
All agreern ents between agencies do not qualify as interagency agreements. The
agreements must be for the exercise of a power common to both parties. 83 Op. Att'y.
Gen. I 53- 057 ( 1983). The authority board was empowered to establish a program for
curriculum development, identification, screening, selection and training of Arizona high
school graduates for jobs in this state. The authority board was also authorized to provide
faciiities and staff to carry out these duties and to enter into interagency agreements.
Laws 1981, chapter 196, section 1. NAU, as a university in this s t a t e , may conduct
research related to vocational and technical education. A. R. S. section 15- 790.
DOE through the s t a t e board of education has a variety of duties regarding the
development of vocational education programs. See A. R. S. section 15 - 20 3. In regard TO
JAG, DOE was not acting merely as a conduit f o r t h e distribution of money. JAG became
a function of DOE a f t e r t h e demise of the authority board.
Secause the powers of the three agencies encompassed the duties of JAG, the
agreements involved the excercise of common powers and were interagency agreern ents
subject to A. R. S. section 11 - 952.
Interagency agreements must be in writing and specify the duration and purpose of
t h e agreement, the manner of financing the project, the methods used to accomplish the
goals of the project and any other necessary matters. The agreement must be submitted
to the a t t o r n e y f o r the agency and be filed with the secretary of state or the county
recorder depending on the scope of the agreement. A. R. S. section 11 - 952.
The agreements concerning NAU's role as fiscal agent for JAG were not in writing,
were not submitted to the attorney general, the attorney for the advisory board and DOE,
and were not filed with the secretary of state. Because these requirements were not met
no monies, including the indirect costs, could be paid under the contracts. A. X. S. section
1 1- 952, subsection J.
2. This question applies to the time when NAU had total control of JAG beginning
May 5, 1986. Thf issue is whether the " intangible" adminisrrative costs fall within the
purposes of JAG. It is not clear what these intangible costs involve so a definite answer
cannot be given about the validity of the payment of these costs from the JAG
appropriation.
Appropriated monies may only be spent for the purposes specified by the
appropriation. Webb v. Frohm iller, 52 Ariz. 128, 79 P. 2d 5 13 ( 1938). The appropriation to
YAU for fiscal year 1986- 1987 was to fund JAG. If the indirect costs are related to the
development or implementation of JAG, the costs may be paid with the monies
appropriated for the program.
3. Assuming that all other requirements are met f o r the payment of indirect costs,
privately donated monies may be used to pay indirect costs if this use is consistent with
the p u r v s e s for which the monies were given. A. R. S. sections 35- 141, 35- 142 and
35- 149.
FACT SITUATION B:
Laws 1981, chapter 196, section 1 provides in part:
Staff shall be exempt from the provisions of title 41, chapter I r , articles 5
and 6, Arizona Revised Ttatutes, and shall serve at the pleasure of the
board.
In May 1985 the authority board approved an employee incentive prograrn for JAG
employees t o t a k e e f f e c t in June, 1985. This incentive program consisted of cash awards
in recognition of sustained performance above that of a f uliy com petent employee, gifts
in recognition of specific accom plishm ents which represent special contribution ro the
program and service awards to be presented to employees who com plete significant
periods of employment.
The executive director of JAG as well as the authority board implemented the
incentive program to recognize superior effort by JAG employees. The plan also provided
that all cash awards be paid out of privately donated monies.
This question does not involve the issue of the transfer of appropriations within a budget
unit. A. R. S. section 35- 173 whicn deals with this issue specifically excludes universities
from its provisions. ( 9. R. S. section 35 - 173, subsection G.
However, this entire incentive program, particularly the granting of cash incentive
awards, was viewed with skepticism by NAU adrninistrators in ACVE who were responsible
for administering the program. According to NAU, JAG officials and JLBC staff,
although JAG was run by the authority board, i t was understood by all involved parties
that all personnel associated with JAG were employees of NAU. The incentive prograrn
was contrary t o regular NAU employee personnel policy promulgated by the board of
regents. For t h i s reason approval was sought from NAU personnel staff and NAU legal
counsel. In addition, pursuant to a board of regents policy, any change in the personnel
code at one university required approval from the personnel managers at the other two
universities ( A rizona state university and the university of A rizona). Approval was never
received from Arizona s t a t e university.
Approval f o r t h e program was eventually given in August, 1985 by NAU
administration, based on the power vested in the authority board in Laws 1981, chapter
196 even though t h i s chapter expired June M, 1982.
In fiscal year 1985- 1956, cash awards were granted t o twelve JAG employees
totaling $ 4,350. According to NAU payroll records, these cash awards originated from the
JAG private donations account and were distributed as part of the employee's regular
NA U payroll check.
QUESTIONS PRESENTED:
1. Under the provisions of Laws 1981, chapter 196 could JAG employees Se
considered officially employees of N, 4U and covered by NAU and board of regents
personnel policy? Under what circumstances?
2. Based on the power vested in the authority board in Laws 1981, chapter 195, did
the authority board have the power to approve this program?
3. If so, did this power supersede the board of regents personnel policy?
4. is the granting of cash incentive awards a violation of a r t i c l e IX, section 7,
Constitution of Arizona, which disallows the state, county, c i t y , town, municipality or
other subdivision of the s t a t e to gift pubiic monies?
ANSWERS:
1, 2 and 3. See discussion.
4. No, if the incentives a r e for future service and are implemented to achieve the
agency's public purpose.
DISCUSS ION :
B. 1, 2 and 3. Under Laws 1951, chapter 196, section 1, the authority board was
empowered to hire staff for JB, G. Staff members were specifically exempted from the
s t a t e personnel system. During its year of existence the authority board was authorized
to establish the ccmpensation for its staff.
After the expiration of t h e authority board, JAG and its staff were transferred to
DOE. Some people providing fiscal services may have been NAU employees, but, by 1986,
twenty full- time equivalent positions were under DOE. It was not until hlay 5, 1986, with
the enactment of Laws 1986, chapter 286, that these positions were transferred to NAU.
Since the authority board expired on June 30, 1982, i t did not have the power to
establish the em ployee incentive plan in 1985. Because the twenty full- time equivalent
positions were not transferred to NAU until May 5, 1986, NAU did not have authority t o
implement the employee incentive plan for these positions until that time.
4. A s t a t e agency may not make a gift of public monies, even to a s t a t e employee.
Article IX, section 7, Constitution of Arizona. But if a s t a t e agency provides an employee
benefits in exchange for services provided, no gift arises. 81 Op. Att'y. Gen. I 51- 979
( 198 1).
If the employee incentive program was applied to prior service provided by staff
members this would be a gift because the incentive benefits would be in addition to
already agreed on compensation. 83 Op. Att'y. Cen. 183- 065 ( 1983). Employee incentives
may be included as part of the compensation package if they a r e implemented to ac5a l' eve
the agency's public purpose and are applied to service performed a f t e r the program goes
into effect. 81 Op. Att'y. Gen. I 81- 079 ( 19811, 79 Op. Att'y. Gen. I 79- 121 ( 1979).
FACT SIT lJj4TION C:
NAU and JAG program officials claim that i t i s their understanding that VAU
receives all JAG furniture and property if the program lasts more than three years.
Qfficials based this assurnption on an informal understanding that any f u r n i t u r e or
property purchased on vocational education grants and the JAG grant has its t i t l e vested
in DCE for the first three years the project is in existence. If the project l a s t s three or
rnore years, the title of the property will be relinquished to the university. The JAG
program has lasted more than three years, so Y A U is claiming ownership according to this
agreement.
.4. R. S. section 11- 952 allows the authority board to enter into contracts to
prescribe the " permissible method or methods t o be emp! oyed in accomp! isnir, g the parzia:
or cornplete termination of the agreement and for disposing of property. . . ". However,
only a l e t t e r agreement defines ownership of the furniture and what methods should be
followed to dispose of this property.
QIJESTION PRESENTED:
1. In the absence of a contract as provided for in A. R. S. section 11- 952 does YAU
have proper claim to all JAG furniture and property?
ANSWER:
1. No.
DISCUSS ION:
C. 1. As indicated in the answer to question A. l., the agreements between NAU
and DOE and JAG are subject t o the requirements of A. R. S. section 11- 952. This section
allows agencies to dispose of property pursuant to an interagency agreement. The l e t t e r
from Robert V. Kerwood to Dr. Jim Hartgraves dated December 17, 1980 does constitute
a written contract. The contract was not approved by the attorney general or filed with
the secretary of state, however, so it is not a valid interagency agreement under A. R. S.
section 11- 952. ROE and NAU may provide for the transfer of the property by meeting
the requirements of A. R. S. section 11- 952.
cc: Villiam Thomson, Director
Performance Audit Division