Annual evaluation, Healthy Families Pilot Program

              ANNUAL EVALUATION
HEALTHY FAMILIES PILOT PROGRAM
Report to the Arizona Legislature
By the Auditor General
December 1995
Report # 95- 19
STATE OF ARIZONA
DOUGLAS R. NORTON, CPA OFFICE OF T H E
AUDITOR GENERAL AUDITOR GENERAL
DEBRA K. DAVENPORT, CPA
DEPUTY n" DIT0R LENERIL
December 26,1995
Members of the Arizona Legislature
The Honorable Fife Symington, Governor
Dr. Linda Blessing, Director
Department of Economic Security
Transmitted herewith is a report of the Auditor General, an Annual Evaluation of the
Healthy Families Pilot Program. This report is in response to the provisions of Session
Laws 1994, 9th Special Session, Chapter 1, Section 9.
This is the first in a series of three reports. The second and final evaluation reports are
scheduled to be released on or before December 31, 1996, and December 31, 1997,
respectively. Our evaluation study h d s that the Department of Economic Security ( DES)
successfully brought the Healthy Families program into operation by awarding contracts in
a timely and efficient manner, with low administrative costs and by developing participant
eligibility criteria.
We also found that the Healthy Families program includes components that are needed for
quality home visitation services along with a strong quality assurance and training program
and a plan for collaboration with other programs. The DES has done this with a projected
cost per participant that is comparable to the national model.
My staff and I will be pleased to discuss or clady items in the report.
This report will be released to the public on December 27,1995.
Sincerelv,
w Douglas R. Norton
Auditor General
2 9 1 0 NORTH 44TH STREET m SUITE 4 1 0 m PHOENIX, ARIZONA 8 5 0 1 8 m ( 6 0 2 ) 5 5 3 - 0 3 3 3 m FAX ( 6 0 2 ) 5 5 3 - 0 0 5 1
SUMMARY
The Office of the Auditor General has completed the first year of a three- year evaluation
of the Healthy Families Pilot Program. This evaluation was conducted pursuant to the
provisions of Laws 1994,9th Special Session, Chapter 1, Section 9. This is the first of a series
of three annual program evaluation reports to be issued on the Healthy Families Program.
The second and final evaluation reports will be released on or before December 31,1996,
and December 31, 1997, respectively.
The Healthy Families Program aims to address the growing need for child abuse
prevention. This home visitation program targets families with newborn children to prevent
child abuse and neglect, promote child wellness and proper development, strengthen family
relations and promote family unity, and reduce substance dependency. By connecting
families with proper services, the Program also aims at reducing welfare dependency among
participating families. The Division of Children and Family Services in the Department of
Economic Security ( DES) administers the Healthy Families Program through its Child Abuse
Prevention Fund. The DES awarded 5 contracts to serve 13 sites across Arizona. The
Legislature appropriated $ 1.7 million in fiscal year 1994- 95 and $ 3 million in fiscal year 1995-
96 for the Pilot Program.
DES Efficiently Implemented
the Healthy Families Program
( See pages 7 through 12)
DES successfully brought the Healthy Families Program into operation. In the process, it
awarded contracts in a timely and efficient manner, developed participant eligibility criteria
as mandated by the Legislature, and operated with low administrative costs.
DES set up an efficient selection process that resulted in timely contract award.
Consequently, Healthy Families was the first of the four early childhood programs
( Healthy Families, Health Start, Family Literacy, and At- Risk Preschool) that were
enacted at the same time to become operational.
DES developed participant eligibility criteria in accordance with legislative mandate. It
also established elaborate contractor eligibility criteria that aimed at serving the areas
and communities most in need of the Program.
DES required very few administrative personnel at the agency level to run the Healthy
Families Program. Additionally, it provided the agency level administration at no cost
to the Program.
While DES brought the Program into operation in an efficient and timely manner, it needs
to improve its contract management practice. During the first year of implementation, DES
amended some of the contracts, rather than enforcing the performance incentive clauses
within them.
Healthy Families Program
Design Appears Sound
( See pages 13 through 19)
The Healthy Families Program planning incorporates several components needed for quality
home visitation services. A strong quality assurance and staff training program, combined
with plans for legislatively mandated service delivery and collaboration with complemen~ ry
programs, are key to this plan.
Compliance with the legislative mandates in terms of service delivery and the program
model includes an intake procedure that is sensitive to participant rights and privacy,
offering services specified by the Legislature, and a plan for moving families from one
stage to the next
The Healthy Families Program has a well- defined quality assurance function and a
nationally recognized, comprehensive staff training program that helps ensure program
integrity.
Healthy Families collaboration efforts promise to both improve and streamline Arizona's
home visitation services. While the Healthy Families Program has made good progress
toward setting up collaborative efforts at various levels to make the services more
comprehensive, more effort is needed in this area. We recommend that DES continue its
collaboration efforts with the Department of Health Services' home visitation programs.
Further, DES should also consider developing partnerships with At- Risk Preschool, Family
Literacy, Even Start, and Head Start programs.
Projected Costs Are
Comparable to National Model
( See pages 21 through 24)
The projected costs for the Healthy Families Program appear comparable to the national
model program in Hawaii. The State spent an average of $ 3,269 per participant during the
first year of implementation. Taking into account the startup costs and the projected
enrollment figures for a full- year implementation of the Program, it appears that the cost
per participant will be comparable to Hawaii's Healthy Start Program by the third year of
operation.
Table of Contents
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Finding 1: DES Efficiently Implemented the Healthy Families Program . . . . . . . . . . . . . . . . . . . . . .
DES Awarded Contracts
in a Timely and Efficient Manner . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
DES Developed Family Eligibility
Criteria in Accordance with
Legislative Mandate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
DES Has Kept Administrative
Program Costs Low . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
DES Should Enforce Contract
Provisions Regarding
Performance Incentives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Recommendation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Finding II: Healthy Families Program
Design Appears Sound . . . . . . . . . . . . . . . . . . . . . . . . . . .
Program Complies with Legislative
Mandates for Home Visitation Services . . . . . . . . . . . . . . . . . . . . . . . . . . .
Quality Assurance and Comprehensive
Training Helps Ensure Program Integrity . . . . . . . . . . . . . . . . . . . . . . . . .
Collaboration Efforts Help Improve
Services and Should Be Expanded . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Recommendation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Paqe
1
Table of Contents ( conwt)
Paqe
Finding Ill: Projected Costs Are
Comparable to National Model .....................
Costs per Family . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Projected Cost per Family
Compares Well with the
National Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Statutory Annual
Evaluation Components . . . . . . . . . . . . . . . . . . . . . . . . . .
Appendix I: Definition and
Types of Child Abuse . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Appendix 11: Outcome Evaluation
Instruments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Agency Response
a- i
a- ii
Table 1
Table 2
Table 3
Table 4
Table 5
Figure 1
Figure 2
Tables and Figures
Page
Comparison of Implementation Time for
Four Early Childhood Programs . . . . . . . . . . . . . . . . . . . 8
Cost per Family
Fiscal Year 1994- 95 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
Projected Enrollment Numbers
and the State Cost per Family . . . . . . . . . . . . . . . . . . . 24
Distribution of Program
Sites and Funding . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
Fiscal Year 1994- 95 Revenues and Expenditures
Fiscal Year 1995- 96 Allocations
( Unaudited) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
Distribution of Monies
among Program Sites ( Unaudited) . . . . . . . . . . . . . . . . 9
Demographic Information on
Healthy Families Participants . . . . . . . . . . . . . . . . . . . . 27
INTRODUCTION
The Office of the Auditor General has completed the first year of a three- year evaluation
of the Healthy Families Pilot Program. This evaluation was conducted pursuant to the
provisions of Laws 1994, 9th Special Session, Chapter 1, Section 9. This is the first of a
series of three annual evaluation reports to be issued on the Healthy Families Program.
The second and final evaluation reports will be released on or before December 31,1996,
and December 31, 1997, respectively.
Child Abuse Is a Growing
Problem Nationwide and
in Arizona
The Legislature established the Healthy Families Pilot Program through the Family
Stability Act of 1994 to address the growing need for child abuse prevention. 1 ( See
Appendix I for the definition and type of child abuse.) Child maltreatment has become
an increasingly serious problem in the United States. Between 1976 and 1994, the
reported number of child abuse and neglect cases increased more than four times, from
669,000 to 3.14 million. In 1976, about 10 out of every 1,000 American children were
reported to have been abused or neglected. The same figure in 1994 stood at 47,
indicating almost a five- fold increase. A December 1995 Gallup Organization study
reports that the incidence of child abuse may be as much as 16 times the reported rate.
Between 1984 and 1994 the rate increased in Arizona as well, except in 1993- 94, when
the number of reported cases dipped 7 percent from the previous year. It is not clear
from the available data whether the drop indicates a new trend or an exception.
However, from the nationwide data, it is safe to assume that the upward trend in
Arizona continues. Additionally, the number of cases involving child abuse and neglect-related
fatalities in Arizona almost doubled between 1991 and 1994.
Recognizing the seriousness of the child abuse problem, a 1992 U. S. General Accounting
Office ( GAO) study identified three basic types of services that are needed to deal with
it. These services can be viewed as a continuum of care that starts with preventative
efforts. Treatment for the children and their families is an intermediate step, and if all
else fails, removal of the child from the abusive environment and placement in foster
care constitutes the final step.
The same act established two other pilot programs - Health Start and Family Literacy. Of these,
Health Start addresses prenatal and early childhood health care needs, and Family Literacy provides
learning opportunities to parents and children.
The GAO study also recognized home visitation programs as promising models for
prevention efforts. These programs aid in improving family health, parental education,
and increasing employment opportunities for the parents. In a more recent study, the
National Committee to Prevent Child Abuse ( NCPCA) argued that developing a
statewide intensive home visitation system allows states to directly address multiple
factors that can lead to abusive behavior and confront the symptoms of child
maltreatment before a crisis occurs.
Program Goals and Services
The Legislature specified the following goals for the Healthy Families Program:
Reduce child abuse and neglect
Promote child wellness and proper development
Strengthen family relations
Promote family unity
Reduce dependency on drugs and alcohol.
In order to achieve these goals, the Legislature determined that the following services
need to be provided to the participating families:
Informal counseling or emotional support services
Assistance in developing parenting and coping skills
Education on the importance of good nutritional habits to improve the overall
health of their children
Education on developmental assessments so that early identification of any learning
disabilities, physical handicaps, or behavioral health needs are determined
Education on the importance of preventative health care and the need for screening
exams such as hearing and vision
Assistance and encouragement to provide age- appropriate immunizations
Assistance and encouragement to access comprehensive private and public preschool
and other school readiness programs
H Assistance in applying for private and public financial assistance including
employment services
H Assistance in accessing other applicable community and public services including
employment services.
Prior Program Efforts
Prior to the legislatively mandated Pilot Program, DES had administered a Healthy
Families Program at three sites in Arizona through its Child Abuse Prevention ( CAP)
Fund.' The Program started in October 1991 at two sites in Tucson and Prescott. A
third site in Casa Grande was funded in November 1992. An additional site in Yuma
is funded by the Regional Behavioral Health Authority. With the new funding, DES has
expanded the Program to 13 additional sites around the State.
Program Model
Arizona's Heathy Families Program, like similar programs around the country, uses a
honre visitation model fashioned directly on Hawaii's Healthy Start Program, which is
considered a national model by various child abuse prevention experts. Apart from the
home visitation concept, Arizona's program closely follows Hawaii's in terms of its
goals, mandated services, and the type of personnel it employs to deliver these services.
Like Hawaii's Healthy Start, Healthy Families Arizona ( HFA) aims at improving family
functioning, promoting optimal child development, positive parenting skills, and
positive parent- child interactions as steps to prevent child abuse. It also incorporates
the following major components from Hawaii's program:
Systematic hospital- based screening to identify high- risk families from a specific
geographic area
Community- based home visiting family support services
An individualized plan stating the intensity of service based on the family's need
and level of risk
Linkage to a medical facility
Coordination of a range of health and social services.
The Child Abuse Prevention ( CAP) Fund was established by the Legislature in 1982 to promote child
abuse prevention and to provide grants to community- based agencies for this purpose. Between 1982
and 1995, the CAP Fund has provided $ 5.5 million to 50 different children and family service
programs.
While Hawaii's Healthy Start serves families from the prenatal period whenever
possible, Healthy Families Arizona can serve only from the time the target child is born.
The staffing pattern in Arizona's Healthy Families Program is also similar to that of
Hawaii's Healthy Start. At the site level, the Program employs three types of staff -
program supervisors, early identification workers, and family support workers. Each site
typically employs one professional supervisor who supervises a team of paraprofessional
family support workers. The family support workers are responsible for ongoing home
visits for up to five years. Each worker is responsible for about 15 cases the first year,
18 the second year, and 21 the third, as services to some families become less intensive.
The early identification workers are responsible for conducting the initial risk
assessment. Initially, staff receive four days of intensive training and subsequently attend
numerous in- service training programs.
Appropriations
The Legislature appropriated $ 1.7 million for fiscal year 1994- 95 and $ 3 million for fiscal
year 1995- 96 from the state general fund to the Department of Economic Security to
implement the Healthy Families Pilot Program.
Scope and Methodology
Pursuant to Laws 1994, 9th Special Session, Chapter 1, Section 9, this annual program
evaluation looks at the Program's effectiveness, its organizational structure and efficiency,
the level and scope of services included within it, the type and level of criteria used to
establish eligibility, and the number and demographic characteristics of the persons who
receive services from the Program.
For the first- year evaluation, the following broad areas were reviewed:
Program implementation by the Department of Economic Security including contract
awarding and management
Healthy Families Program design including staff training, service delivery, and quality
assurance
Program costs including cost per family and cost comparison with similar programs
in the country.
Methods used in this evaluation include interviews with agency and program staff,
analysis of program revenues and expenditures, analysis of participant enrollment and
I characteristics from the Program's participant database, program document and file
reviews, literature reviews, and direct observation of existing sites and staff training.
I The Auditor General and staff express appreciation to the Director, Healthy Families
Coordinator, and staff of the Arizona Department of Economic Security's Division of
Children and Family Services, as well as the Healthy Families Pilot Program staff for their
I cooperation and assistance during the first year of the Healthy Families Pilot Program
Evaluation.
FINDING I
DES EFFICIENTLY IMPLEMENTED
THE HEALTHY FAMILIES PROGRAM
The Department of Economic Security, the agency responsible for administering the
Healthy Families Program, successfully brought the Program into operation. Since the
key program activities such as service delivery, staff training, quality assurance, and data
management are contracted out, the Agency's performance hinged on the awarding and
managing of these contracts. DES acted in a timely and efficient manner in awarding
contracts. DES also established contractor and participant eligibility criteria pursuant to
legislative mandate, and completed all necessary administrative tasks with low
administrative overhead. In terms of contract management, however, DES needs to abide
by its original contracts as much as possible, and enforce the performance incentives
contained in the contracts.
DES Awarded Contracts
in a Timely and Efficient Manner
Of all four early childhood programs that were enacted at the same time, Healthy Families
was the first to be implemented. Several factors contributed to this early implementation
by DES. Foremost, DES awarded the contracts in a timely fashion. It developed a
comprehensive Request For Proposal ( RFP), followed by an efficient selection process that
resulted in distribution of program funds across 13 sites. The experience of previously
administering a similar program helped DES in understanding local needs and developing
program guidelines.
Ait eflicieizt selection process reslilted in tiiizely coiztract awards - An efficient contracting
process enabled DES to award contracts in a timely manner. The timely development of
a comprehensive RFP, a clear definition of eligibility and selection criteria, and early
establishment of an independent proposal selection committee ensured an efficient
selection process. Consequently, of all four early childhood intervention programs that
were mandated at the same time - Healthy Families, Health Start, Family Literacy, and
At- Risk Preschool Program, Healthy Families started operating first. Table 1 ( see page
8) compares the starting time for these programs. The early start was made possible by
DES' prompt action following the enactment of the law. DES issued the RFP within one
month from the passage of the Family Stability Act bill in June 1994. As a result, it was
able to select service providers and finalize the contracts by November 1994, and service
delivery began in January 1995.
Program
Healthy Families
Health Start
Family Literacy
At- Risk Preschool
Table 1
Comparison of Implementation Time for
Four Early Childhood Proqrams
Agency
DES
DHS
ADE
ADE
Contract
Award
Date
Program
Start
Date
a Some Health Start contracts were not signed until May 1995.
Most Family Literacy sites did not start operating until August 1995 or later.
Approximately one- third of the contracts were awarded at this time.
Source: Auditor General staff analysis of program contracts and interviews with agency staff and
contractors.
Higlz yercerztage of tlze nyproyriated a~ lzorrrznt llocnted to seuvice delivenj - DES allocated
94 percent ($ 1.7 million) of the fiscal year 1994- 95 appropriation for service delivery. Of
the remaining amount, 5 to 6 percent was spent on providing program oversight through
one of the contractors who had experience in running a previous Healthy Families site,
Data management, involving the development and updating of a program database,
accounted for less than 1 percent of the total appropriation. The total appropriation for
fiscal year 1995- 96 is $ 3 million. DES did not award any new contracts for fiscal year 1995-
96 but used the additional $ 1.3 million to expand the current contracts and serve more
participants. From the 1995- 96 appropriation, $ 49,476 remains unallocated. The Child
Abuse Prevention Fund ( CAP) coordinator indicated that the unallocated amount resulted
from conservative budgeting and would be used to expand some of the smaller programs.
DES awarded 5 cosztvacts to sene 13 sites - DES awarded 5 contracts to serve 13 sites
in 6 counties. Figure 1 ( see page 9) provides an overview of these contracts. One of the
contractors, Tucson Association for Child Care ( TACC), received two separate contracts
for urban and rural sites, respectively. In both the TACC contracts, TACC acts as an
umbrella organization that brings together a number of not- for- profit organizations as
service providers, Their urban contract includes the cost of providing statewide program
oversight through a quality assurance coordinator.
DES contracted out the database management function to a data management firm. Our
Office determined that while the Program collected sufficient data, it lacked a proper
database to comply with the legislative mandates. Accordingly, DES extended the existing
data management contract to meet the legislative as well as the long- term program needs.
The increase in data collection costs is reflected in the fiscal year 1995- 96 budget.
Figure 1
Distribution of Monies amona Proclram Sites
( Unaudited)
Cochise
County Maricopa Pirna Santa Cruz Yavapai Coconino NIA
Contractor Southwest Tucson Assoc. Tucson Assoc. Marcus Coconino
Human for Child for Child J. Lawrence County Dept. Data
Development Care ( Urban) Care ( Rural) Medical Ctr. of Public Health Collections
Number of
Sites
Fiscal Year ' 95
Budget
Fiscal Year ' 96
Budget 0
4 3 3 1 2 NIA
$ 500,000 $ 558,894 $ 369,304 $ 1 06,252 $ 160,000 $ 5,550
$ 983,000 $ 799,174 $ 650,523 $ 136,403 $ 320,096 $ 61,328
Source: Auditor General staff analysis of Healthy Families Program contracts.
DES Developed Family Eligibility
Criteria in Accordance with
Legislative Mandate
As mandated by the legislation, DES developed eligibility criteria individual families must
meet to participate in the Program. DES followed a nationally recognized set of criteria
for participant eligibility. Additionally, contractor eligibility criteria were defined with
a view to serve communities and geographic areas that most need the Program.
Participant eligibilitj miteria - DES specified eligibility criteria for individual families
following Hawaii's Healthy Start model. In order to receive Healthy Families services,
a family must:
Have a newborn baby
Reside in one of the program target areas where the contractors operate
Screen positive on two separate screening tools that identify families under stress and
therefore at risk for child abuse.
Contractor eligibility criteria - DES developed the contractor eligibility criteria with
a view to serve communities and geographic areas that most need the Healthy Families
Program. In determining which contractors were eligible to apply for funding, DES used
different criteria for rural and urban applicants. The eligibility criteria for rural program
sites are based on geographic area distinguished by zip codes. These rural zip code areas
must have:
At least 350 live births annually
At least 75 Child Protective Services ( CPS) reports involving children 0- 5 years of age
A low rate of prenatal care
Underutilization of health care services.
Some rural area towns, whose citizens may need Healthy Families Programs, do not meet
the eligibility criteria because of their small population. Consequently, contractor
applicants are encouraged to pursue creative staffing options through positioning some
family support workers in neighboring high- risk towns.
For urban areas in Maricopa and Pima counties, DES identified certain zip code areas
in need of services based on CPS statistics and available services. Applicants wishing to
operate in other zip code areas of these two counties had to demonstrate the need for
a program in those areas.
DES Has Kept Administrative
Program Costs Low
DES maintained very low administrative overhead costs at the agency level to run the
Healthy Families Program. For example, it did not expend any program dollars for
agency- level administration. As a result, a high percentage of the legislatively
appropriated funds were spent on actual service delivery. While DES has hired additional
personnel to cope with the increased funding and resulting program expansion for fiscal
year 1995- 96, DES managed to absorb the increased cost without taking money from the
program budget.
The Division of Children and Family Services ( DCFS) administers the Healthy Families
Program through its Child Abuse Prevention ( CAP) Fund. The CAP Fund administrator
was given this responsibility because of her previous experience administering the existing
Healthy Families Program. In fiscal year 1994- 95, the CAP Fund had only one full- time
employee - a program specialist, who coordinated the Fund's programs. According to
her own estimate, the CAP Fund Coordinator spent approximately 70 percent of her time
administering the new Healthy Families Program, 15 percent on the existing Healthy
Families Program, and 15 percent on other CAP Fund activities. Thus, the Healthy
Families Program essentially required only 0.7 FTE in fiscal year 1994- 95.
Beginning fiscal year 1995- 96, a full- time program specialist position has been added to
assist the CAP Fund Coordinator with the additional administrative responsibilities
associated with the increased appropriation and the resulting expansion of the existing
contracts.' As with the CAP Fund Coordinator position, DES is shouldering the personnel
costs associated with the program specialist position.
DES Should Enforce Contract
Provisions Regarding
Performance Incentives
While DES performed well in terms of timeliness and efficiency in bringing the Program
into operation, it needs to improve the contract management process. During the first
year of implementation of the Healthy Families Program, DES amended some of the
contracts rather than enforcing their performance incentive clauses.
Program appropriations increased from $ 1.7 million in fiscal year 1994- 95 to $ 3 million in fiscal year
1995- 96. DES hopes to serve 917 families in 1996- 97, which is almost double the figure served in 1995- 96.
The number of online staff also increases from 42 to 55 at the same time.
DES typically employs a negotiated rate in reimbursing the contractor for service delivery.
Generally, a negotiated rate is obtained by simply dividing the total budget dollars by
the number of months in the budget period. The contractors receive this monthly payment
automatically if they fulfill the obligations outlined in the contract.
For Healthy Families contracts, DES computed the negotiated rate differently, not only
to ensure that the contractors enroll the number of participants as specified in the contract,
but also to make sure contractors are not overly penalized for failing to meet specified
enrollment rates. DES recognized that some factors, such as the birth rate and program
refusal rate, are out of the contractor's control. Accordingly, 80 percent of each contractor's
approved budget amount was awarded at the monthly negotiated rate. The remaining
20 percent for each contractor was awarded on a per- participant basis to encourage
contractor performance in terms of participant enrollment.
However, DES failed to enforce the contracts' performance incentive clauses. Instead,
when two of the contractors failed to meet the target numbers by significant margins,
DES chose to amend their contracts to reimburse the full amount anyway. According to
DES, mitigating factors contributed to these contract amendments; namely, unrealistic
target numbers for some areas, and the need for startup time. While the total reimburse-ment
ceiling remained unaffected, this action defeated the original intent of the 80- 20 split.
This resulted in a higher cost per participant than originally projected since the same
amount of money was spent on fewer participants. Moreover, such actions set a bad
precedent and treated unfairly those contractors who actually met or exceeded the target
enrollment figures. The 80- 20 split serves the dual purposes of providing contractors with
sufficient monthly cost reimbursement yet retaining a portion based on performance. DES
should abide by this control and continue this incentive,
RECOMMENDATION
DES should enforce the provisions of the Healthy Families' contract performance
incentives.
FINDING II
HEALTHY FAMILIES PROGRAM
DESIGN APPEARS SOUND
The Healthy Families Program design includes several components needed for quality
home visitation service delivery. Home visitation services vary both in type and intensity
for each participant; therefore, the Program must ensure staff deliver services that are
appropriate for each participating family. To accomplish this task, the statewide program
design attempts to ensure program quality in the following ways. First, home visitation
practices comply with legislative intent. Second, a strong quality assurance component
and extensive staff training work to ensure staff follow standard and approved practices
for home visitation. Finally, the Program seeks collaboration with other Arizona agencies
and programs to provide better coordinated and more comprehensive home visitation
services. These collaboration efforts should be further strengthened and diversified.
Program Complies with
Legislative Mandates for
Home Visitation Services
The Healthy Families Program incorporates legislated home visitation requirements. For
example, existing intake procedures and service components ensure families are
appropriately enrolled, offered a broad array of services, and progress toward individual
and program goals and objectives.
Intake proced~ ire protects pnrticipasit riglits asid privacy - Because of the sensitive and
personal nature of Healthy Families service delivery, program administrators developed
a comprehensive set of rules for the intake process. These rules also address the legislated
intake requirements. The procedures require that initial contact and assessment respect
the needs and rights of the potential participants and ensure that:
When possible, the initial contact and assessment is conducted in the hospital.
No contact during the intake process occurs at the potential participant's home without
prior consent.
Parents are aware of their rights and responsibilities.
Parents are allowed sufficient time to make enrollment decisions.
An informed consent has been obtained before service delivery begins.
Healthy Families staff are trained to appropriately perform the intake procedures. Their
adherence to established procedures is further monitored through the quality assurance
function. In our opinion, these steps adequately protect the potential partisipants' privacy
and allow them to make informed enrollment decisions.
Model ineovporates mandated sevvices - The Healthy Families model incorporates the
specific services required in the program legislation. DES requires program contractors
to offer the following services at every site:
Crisis intervention
Emotional support to parents
Teaching and modeling of parenting, home management, nutrition, child development,
preventive health education, and life coping skills
Education on child development and early identification of learning disabilities,
physical handicaps, or behavioral health needs
rn Bonding and attachment activities and utilization of an in- home curriculum
Linkage to medical home for comprehensive preventive health care
Assistance and encouragement to fully immunize program children
Information about school readiness programs
w Use of formal and informal community resources to include referrals to job training
and employment services
Training and instruction in child care, behavioral management, and physical and
emotional development
rn Case management services
Information and referral
Transportation by Healthy Families Program staff.
Staff conducting home visits are required to document services as they are delivered to
the families, This documentation, along with ~ therre quired forms, provides a way to
track compliance with legislated service components. If the family needs a particular
service, the site supervisor or the quality assurance coordinator can make sure the
appropriate service was provided. Compliance checks are also completed by the quality
assurance coordinator through site visits and home visitation observations.
The Program distributes informational materials to participating families to comply with
mandated service components. While many of these informational materials are
recommended by the National Committee to Prevent Child Abuse, the Program incorporates
locally available materials whenever appropriate. As required by the legislation, the Program
also distributes the Arizona Family Resourm Guide' developed by the Department of Health
Services. While the informational materials distributed to the participants are meaningful
and understandable, the program staff receive more specialized materials as part of their
Healthy Families training. The majority of these materials were developed by the Hawaii
Family Stress Center after years of research. We reviewed selected materials for both
participants and staff, and found them to adequately address program needs.
Home visitation model includes plan for transitioning families - Healthy Families has
developed specific criteria for moving participants along and transitioning them to a less
program- dependent situation. Families who exhibit growth toward individual and
program goals move from initially intensive services to services that are less frequent and
necessary. As families effectively transition through the Program, they move closer to
the long- term legislated goals of reducing illiteracy, reducing dependency on welfare,
increasing employment, and achieving self- sufficiency.
The Healthy Families model includes four levels of service that depend on the intensity of
the participants' needs. The first and most intensive level requires that the family is visited
at least once a week by a Family Support Specialist ( FSS). Participants in the second level
require one home visit every two weeks, and the third level requires only one visit a month.
The fourth level signifies " graduation" from the Program. The FSS maintains minimum
contact with the participant, usually once every three months, until the child enrolls in a
preschool program.
Staff move families from one level to the next based on criteria specified in the program
model. Decisions to move families depend primarily on improvements in home environments,
parental coping skills, parent- child interactions, identification of a support network, and
meeting the child's medical and immunization needs. For instance, no occurrence of crisis
situations for a month, or appropriate response to a crisis situation for a family with high
child abuse potential, is considered in making the transition decision.
Although there is no fixed time for level- to- level transitions, interviews with program
personnel suggest that the level of a participant's at- risk behaviors upon enrollment is a strong
predictor of the program duration necessary for that participant On average, program
personnel estimate that it takes between 6 and 12 months for level 1 participants to move
to level 2. They also believe that the transition from level 1 to level 2 is the hardest and most
critical one for participants to achieve. This transition signifies overcoming a great
motivational barrier. Once this hurdle is passed, the subsequent transitions should be easier.
Arizona Family Resource Guide contains a comprehensive list of agencies who provide emergency and
other services for children and families.
15
However, the new program personnel do not yet have any direct experience with the
subsequent transitions because the program is less than one year old. Based on the quality
assurance coordinator's estimates, the total program length will vary between two and three
years for the participants.
Quality Assurance and
Comprehensive Training
Helps Ensure Program
Integrity
The Healthy Families Program contains strong quality assurance and training components
to ensure staff deliver services as they were intended. Because the Program has a lengthy
list of procedures and extensive documentation requirements, it must provide consistent
monitoring and training to maintain quality services. DES administrators and the quality
assurance coordinator preserve program integrity through visits to all Healthy Families sites,
communication with program staff, on- and off- site problem solving, and ongoing staff
development training.
Stro1zgqrualit. rl~ z ssurn~ zcec mzpo~ mt- t Healthy Families Program integrity, compliance with
DES and legislative requirements, and meeting nationally accepted standards are assured
through a statewide quality assurance coordinator. This is a unique position that DES defined
and contracted out for, along with the staff training responsibilities, to the Tucson Association
for Child Care ( T'ACC). Typically, this function would be handled at the state agency level.
However, TACC can provide this service at a lower cost than DES.
The quality assurance coordinator is certified as a national trainer for Healthy Families
Program staff by the National Committee to Prevent Child Abuse. She is responsible for
providing adequate staff training, monitoring program data collection and recordkeeping,
service delivery, and general compliance with program requirements. The coordinator
has been instrumental in developing and adopting most of the program materials in
Arizona. Family support specialists in Pima and Maricopa Counties consistently
articulated the program goals, their responsibilities, and participants' needs. We attribute
this to the Program's quality assurance function.
Staff traitzfilg - DES requires each worker to receive at least 30 hours of training on the
Healthy Famllies Program concept and implementation. Healthy Families provides
compulsory initial training for all new staff, and then supplements training with a staff retreat
and in- service training.
Initial Staff Training - The initial training addresses topics recommended by the
National Committee to Prevent Child Abuse - the leading child abuse prevention
advocacy and research organization in the country. The training is intended to provide
staff with the basic knowledge and expertise needed to start home visitation. This four-
16
day training is mandatory for all new Healthy Families workers and supervisors. A nine
day training is mandatory for all supervisors.
Statewide Staff Refreat - The statewide staff retreat serves two functions: a) to provide
specialized and more in- depth training that staff need on certain topics; b) to allow
interaction among program staff from different sites. The staff retreats are mandatory,
and give employees an opportunity to interact with other staff throughout the State, and
to receive speciahd training and feedback.
In- Service Training - Individual contracting agencies are responsible for in- service
training of their staff. Healthy Families Arizona provides a general list of topics for these
in- house training sessions, but recommends that the actual topics be selected based on
local needs. This is the only training that could potentially differ substantially from site
to site depending on the contractors' resources, access to qualified trainers, and staff
development programs.
Interviews with supervisors and family support specialists indicate the training provided
by Healthy Families Arizona seems to adequately address the training needs of the program
staff. As the program evolves, we plan to monitor the training to determine if it continues
to meet the training needs of the supervisors and family support specialists.
Collaboration Efforts Help
Improve Services and
Should Be Expanded
Healthy Families collaboration efforts promise to both improve service quality and to
streamline Arizona's home visitation services. Collaboration and partnerships have several
potential benefits including more comprehensive services for participants, sharing of expertise,
less duplication of services, reduced cost, and in general, better utilization of available outside
resources. We have identified three areas of collaboration where the Healthy Families
Program has already made sigruficant progress, and some others where further efforts are
needed.
The Healthy Families Program has already made substantial progress in establishing some
of the needed partnerships that allow sharing of resources and expertise among program
personnel and the community- at- large. These partnerships help improve service quality.
Within- Program Collaboration - Healthy Families fosters within- program collaboration
through a Policy and Procedure Council that consists of program staff from all levels and
from different sites. Collaboration within the Healthy Families Program is crucial to
maintain program standards, enhance staff morale, and share expertise available within
the Program. The council provides a forum for discussion on service delivery issues and
recommends policies and procedures to overcome existing and potential problems.
Community Partnerships - The Program has made considerable progress in setting up
community partnerships that require the involvement of a broad range of entities,
including people with active interests in child abuse prevention, hospitals, local
businesses, and other community organizations. These partnerships provide additional
resources and may occur at the contractor, regional, or state level. At the contractor level,
for instance, partnerships with the local hospitals help facilitate the initial screening.
Additionally, much of the in- kind contributions that are part of the individual contractors'
budget have come from private donations or through local businesses. For example, the
contractor in Maricopa County provided almost a quarter of the total service cost in the
County through in- kind contributions. These partnerships underscore the communities'
commitment to the Program as well as make the programs more cost- efficient
State- Level Advisory Council - At the state level, a number of persons from both
governmental and nongovernmental agencies who are committed to preventing child
abuse have formed a Healthy Families advisory council. This council brings together
individuals with varying expertise and is actively involved in long- term, strategic planning
for the Program.
While the Program has made progress in establishing some of the needed partnerships, more
effort is needed to foster others that will help streamline services for children in Arizona as
well as make them more comprehensive for the families in need. These partnerships involve
collaborating with other govenunent agencies or programs with similar or complementary
goals and services.
Collaboration with the Department of Health Services ( DHS) Programs - Healthy
Families needs to collaborate and coordinate with other community programs to
effectively cater to the differing needs of its families and to streamline home visitation
services. Currently, program administrators are in the process of setting up coordinated
efforts with several programs run by the Department of Health Services. These programs
include another Family Stability Act program - Health Start, plus the Newborn Intensive
Care Program, Community Health Nursing, and Teen Prenatal Express. This will allow
Healthy Families to provide a more comprehensive array of services to families with
newborn children. These partnerships will be particularly relevant for the level 1
participants who have the greatest need for early childhood intervention programs.
Recommended collaboration with preschool programs - As more participating families
move to levels 3 and 4, Healthy Families should collaborate with programs catering to
preschool children's needs. Coordination with programs such as At- Risk Preschool, Head
Start, Even Start, and Family Literacy may help fulfill this need. Since transition to levels
3 and 4 will occur as early as 1995- 96 for some of the participants, the Program should
pursue developing partnerships with these four designated programs as quickly as
possible.
RECOMMENDATION
The Healthy Families Program should continue to enhance its collaborative efforts.
Specifically,
It should continue collaborating with DHS, and set up at least one model site where
it can work with select DHS programs during the Program's pilot phase.
Healthy Families should also collaborate with programs catering to preschool
children's needs. Particularly, the Program should consider developing partnerships
with At- Risk Preschool, Family Literacy, Even Start, and Head Start Programs.
FINDING Ill
PROJECTED COSTS ARE COMPARABLE
TO NATIONAL MODEL
The projected costs for the Healthy Families Program appear comparable to the available
costs for the national model program in Hawaii. Using cost per family as an indicator
of program costs, the first- year. costs seem somewhat higher than the national model.
However, when startup costs and the projected enrollment figures for a full- year
implementation of the Program are taken into account, it appears that costs will become
comparable to Hawaii's Healthy Start Program by the third year of operation.
The total program cost in fiscal year 1994- 95 was $ 1.92 million. Of this total, $ 1.7 million
came from state appropriations and the remaining $ 220,138 came from in- kind
contributions. Ninety- four percent of the total state appropriation went to actual service
delivery, while only 5 percent was spent on quality assurance and less than 1 percent
was spent on data management. The Department of Economic Security provided the state-level
administrative services at no cost to the Program.
Costs per Family
The State spent an average of $ 3,269 per family during the first year of program
implementation. State costs per family were lower than what the Program actually spent
for each family. This occurred because of in- kind contributions from the contractors.
Although the RFP did not specifically call for in- kind contributions, these voluntary
contributions, as proposed by the contractors, were written into ihe contracts once the
budgets were finalized. Once the in- kind contributions are taken into account, the cost
per family becomes $ 3,693. On average, the contractors contributed more than 10 percent
of per- family costs. Table 2 ( see page 22) provides details of program costs, contractor
contributions, and the costs per family.
Projected Cost per
Family Compares Well
with the National Model
The projected cost per family in Arizona's Healthy Families Program is comparable to
Hawaii's Healthy Start Program. While the first- year costs were somewhat higher, after
taking into account certain factors associated with the first year of implementation as well
as the projected costs, the Arizona Healthy Families Program cost appears reasonable.
In addition, the costs per family should decrease once the Program is better established
and has a mix of participants from different levels.
1 Table 2
~ Cost per Family - Fiscal Year 1994- 95
Families
enrolled Total Cost State Cost/
Total In- Kind as of Family Family
Service State Other June 30, ( Actual ( Actual
Contractor County Cost Cost Sources 1995 Enrollment) Enrollment)
SWHDa Maricspa $ 666,151 $ 500,000 $ 166,151 182 $ 3,666 $ 2,747
TACC ( Urban) Pima 481,049 463,003 18,046 143 3,364 3,238
TACC ( Rural) Cochise and
Santa Cruz 386,677 369,304 17,373 117 3,305 3,156
,!? CCDPH~ Coconino 160,000 160,000 0 57 2,807 2,807
MJLMC Yavapai 117,772 106,252 11,520 21 5,608 5,660
TACC~
( Quality Assurance) 102,939 95,891 7,048 N/ A N/ A N/ A
Data Management
Contractor 5,550 5,550 0 N/ A N/ A N/ A
Total $ 1,920,138 $ 1,700,000 $ 220,138 -- 520
a SWHD - Southwest Human Development
CCDPH - Coconino County Department of Public Health
MJLMC - Marcus J. Lawrence Medical Center
TACC - Tucson Association for Child Care
Source: Auditor General staff analysis of Healthy Families Program contracts.
Cost comparison with other similar programs remains a difficult task as evaluators have,
to a large extent, traditionally ignored cost- effectiveness issues in evaluating the efficacy
of social programs. In the case of Healthy Families programs, only Hawaii's Healthy Start
Program provides any detailed cost calculations. Additionally, the National Committee to
Prevent Child Abuse ( NCPCA) has recently published its own estimate of how much a
program like Healthy Families should cost per family in the first year of implementation. .
The first- year cost per family is sornewlsat high - The State spent an average of $ 3,269
per Healthy Families participant in Arizona, which is 16.7 percent higher than Hawaii and
9 percent higher than the NCPCA estimate. The total cost per family in the Hawaii program
stands at $ 2,801, of which 86 percent is spent on case management and 5 percent is spent
on screening and assessment. The rest is split among staff training, statewide retreat, and
monitoring- and evaluation- related activities. The NCPCA estimate of cost per family, while
theoretical, nevertheless provides a more realistic comparison figure since it explicitly
predicts the first- year costs. According to NCPCA, the cost per family in the first year of
implementation should be approximately $ 3,000.
Factors associated witls first- year ilnplensentation slsould be taken into account - While
comparisons with Hawaii and the NCPCA estimate are helpful in giving an idea of how
expensive the Arizona Healthy Families program is when compared to similar programs
in the country, it is somewhat unfair to the Arizona Program. The figures available for the
Arizona program come from the first year of program implementation, which includes the
following mitigating factors.
The Program was implemented for only part of the fiscal year, which prevented some
sites from fulfilling their target number of families while paying for the personnel costs.
This contributed to higher costs per family because costs were allocated among fewer
families.
Nearly all the participating families in the first year needed the most intensive level
of services, limiting the number that could be served by each worker. In subsequent
years, an increasing mix of levels should enable each worker to serve more families.
We used a conservative estimate of cost per family in Arizona by not including those
who enrolled and later dropped out before the end of the year. Currently, we do not
have good criteria for determining how long a family should remain in the Program
before being considered full participants. However, if any allowance is made for part-year
participation in the Program, the estimated cost- per- family will decrease.
Projected yrogra~ gzg rowtls will reduce costs - Once the Program is better established with
full- year implementation and a mix of participants from different levels, the cost per family
in Arizona should decrease and become more comparable to the cost in Hawaii as early
as fiscal year 1996- 97. Table 3 ( see page 24) shows the growth in the number of
families and the consequent reduction in cost per family. These projections are based on
the assumptions that target enrollment numbers will be reached in subsequent years, the
family support specialists ( FSS) will be able to handle higher caseloads, the predicted live
birth and rehsal rates will hold constant, and state appropriations between fiscal years
1996 and 1997 remain constant.
Fiscal
Year
Table 3
Projected Enrollment Numbers
and the State Cost per Family
Allocation No. of
in Millions FSS
Projected
Enrollment
a Fiscal year 1995 numbers are all actual.
Excludes $ 49,476 from state appropriation that DES is yet to allocate.
Not yet appropriated, projected only.
Source: Auditor General staff analysis of DES Healthy Families projection.
Caseloadi State Cost/
FSS Family
RECOMMENDATlONS
This finding provides information only; therefore, no recommendations are presented.
STATUTORY ANNUAL
EVALUATION COMPONENTS
Pursuant to Laws 1994, gth Special Session, Chapter 1, Section 9, our Office is required to
include the following information in the annual program evaluation.
1. Information on the number and characteristics of the program participants.
Of the 607 families that enrolled in the program, this report contains demographic
information on families for whom data were available. Demographic information
includes the families that left the program before June 30, 1995. In general, more
complete information was available on the mother of the child than on the father. Figure
2 ( see page 27), portrays various participant demographic information.
Participants by County - At the end of program year 1994- 95, the Healthy Families
Program was serving 520 families. Over 60 percent of these families came from the
urban areas of Maricopa and Pima Counties. The rest came from four predominantly
rural counties, Coconino and Yavapai in the north and Cochise and Santa Cruz in
the south.
Participants by Age, Employment, and Education - The information available on
these three characteristics varies in reliability and completeness. The median age
of the participating mother was 22 years. The median age of the father was 24 years.
Thirty- one percent or almost one- third of the mothers were teenagers, with the
youngest being just 13 years old. In contrast, less than 15 percent of the fathers were
teenagers, although the youngest among them was only 15 years old.
Information on participants' employment status and education is much less complete.
In fact, no information is reported on the father's employment status or educational
background due to the unreliability of the data. Most mothers in the Program ( 86.7
percent) reported being unemployed. In addition, of the 61 percent of the mothers
for whom education information is available, 45 percent reported having graduated
from high school or possessing a GED while 55 percent reported possessing neither.
Other Demographic Information - As illustrated in Figure 2, the marital status
and living situations of mothers participating in the Program varies widely. Almost
62 percent ( 61.7) of the participating mothers were single while another eight percent
were either divorced or separated. However, only 15 percent actually lived alone
while a majority lived with their parents, husbands, other relatives, or nonrelatives.
While one- quarter ( 25.6 percent) of all participating mothers lived with their parents,
the percentage rose to almost half ( 49 percent) for the teenage participants. Most
participants 30 years or older, on the other hand, tended to live alone or with
persons other than their parents.
Most program participants belonged to impoverished households. Among the 309
households whose income including wages and assistance, could be determined,
almost three- quarters ( 72.8 percent) reported annual household income, including
government assistance, below $ 10,000 translating to a median income of only $ 6,656
per family. Only 11 families reported income higher than $ 20,000. A large number
of families depended on one or more welfare benefits, most commonly AFDC, food
stamps, and WIC programs. Only 13.6 percent reported having private or other
insurance while over 80 percent were on AHCCCS ( see Figure 2, page 27).
The ethnicity of the program participants varies; however, most are of Hispanic
origin. Over 56 percent ( 56.8 percent) of the mothers and 57.9 percent of the fathers
were of Hispanic origin. Non- Hispanic White participants accounted for 29.4 percent
of the mothers and about 25 percent of the fathers. African- Americans accounted
for less than 10 percent of the participants.
2. Information on contractors and program service providers.
DES awarded 5 contracts to serve 13 sites in 6 counties. Table 4 ( see page 28) provides
an overview of these contracts. One of the contractors, Tucson Association for Child
Care ( TACC) received two separate contracts for urban and rural sites, respectively.
In both the TACC contracts, TACC acts as an umbrella organization that brings together
a number of not- for- profit organizations as service providers. Their urban contract
includes the cost of providing statewide program oversight though a quality assurance
coordinator.
DES contracted out the database management function to a data management firm, Our
Office determined that while the Program collected sufficient data, it lacked a proper
database to comply with the legislative mandates. Accordingly, DES extended the
existing contract with the data management contractor, to meet the legislative as well
as the long- term program needs. The increase in data collection costs is reflected in the
fiscal year 1995- 96 budget.
Figure 2
Demographic Information on
Healthy Families Participants
Employment
Status
oEmplo yed
Unemployed
Status Situation
Married
Single
B23 Divorced
0 Separated
Cohabitating
Alone
0 Parents
Husband
@ Other Relative
NopRelative
Source: Auditor General staff analysis of Heallhy Families parlicipant information provided by DES Data
Mal~ agernenlC. ontractor.
Contractor
Table 4
Distribution of Proaram Sites and Fundinq
Number Fiscal Year Fiscal Year
County of Sites 1995 Budnet 1996 Budget
Southwest Human
Development Maricopa 4 $ 500,000 $ 983,000
Tucson Association for
Child Care ( Urban) Pima 3 558,894 799,174
Tucson Association for Cochise
Child Care ( Rural) Santa Cruz 3 369,344 650,523
Marcus J. Lawrence
Medical Center Yavapai 1 106,252 136,403
Coconino County Dept.
of Public Health Coconino 2 160,000 320,096
Data Management
Contractor N/ A N/ A 5,550 61,328
Total - - 6 - 13 $ 1,700,000 $ 2,950,524
Source: Auditor General staff analysis of Healthy Families program.
3. Information on program revenues and expenditures.
The State appropriated $ 1.7 million in fiscal year 1994- 95 and $ 3 million in fiscal year
1995- 96 for the Healthy Families Pilot Programs. Additionally, the Program received
another $ 220,138 through in- kind contributions from the contractors in fiscal year 1994-
95, raising the total program revenue to $ 1.92 million. For fiscal year 1995- 96 the
contractors have committed a total of $ 442,759 in in- kind contributions which will
potentially raise the revenue to $ 3.44 million.
Of the fiscal year 1994- 95 state appropriation of $ 1.7' million, DES spent 94 percent
for service delivery and took no agency administrative cut at all. Of the remaining
amount, 5.6 percent was spent on statewide quality assurance of the Program. Data
management cost less than 1 percent of the total appropriation. The fiscal year 1995- 96
state appropriation is $ 3 million. DES plans to spend over 91 percent on service
delivery, 5.3 percent on quality assurance, 2 percent on data management, and 1.6
percent remains unallocated. The CAP Fund coordinator indicated that the unallocated
amount resulted from conservative budgeting and could be used to expand some of
the smaller programs. She does not foresee this amount remaining unspent at the end
of the fiscal year. Revenue, expenditure, and allocation figures for fiscal year 1994- 95
and 1995- 96 are presented in Table 5.
Table 5
Fiscal Year 1994- 95 Revenues and Expenditures
Fiscal Year 1995- 96 Allocations
( Unaudited)
Fiscal Year 1994- 95 Fiscal Year 1995- 96
Contractor Contractor
State Contribu- Total State Commit- Projected
Expenditures tions Expenditures Allocations ments Expenditures
Service
Delivery $ 1,598,559 $ 213,090 $ 1,811,649 $ 2,730,332 $ 440,759 $ 3,171,091
Quality
Assurance 95,891 7,048 102,938 158,864 2,000 160,864
Data Man-agement
5,550 N/ A 5,550 61,328 N/ A 61,328
Total $ 1,700,000 $ 220,138 $ 1,920,138 $ 2,950,524 " $ 442,759 $ 3,393,283
" $ 49,476 remains unallocated from the state appropriation of $ 3,000,000.
Source: Auditor General Staff analysis of Healthy Families Program contracts.
4. Information on the number and characteristics of enrollment and disenrollment.
At the end of fiscal year 1994- 95, Healthy Families Arizona was serving 520
participants out of the state- appropriated fund against an original target number of
585. The Program enrolled a total of 607 families. On average, 14 percent attrition
occurred statewide. There was no significant difference in this rate across the State.
However, the sites in Sierra Vista, Page, and Tuba City reported less than 7 percent
attrition in fiscal year 1994- 95. As of August 31, 1995, the latest date for which
information was available, the Program was serving a total of 606 families.
Of the 87 families that terminated from the Program, 31 percent were reported to have
moved and another 20 percent could no longer be contacted.
5. Information on the average cost for each participant in the program.
In fiscal year 1995- 96, Healthy Families programs spent an average of $ 3,693 per
family. Per- family costs by contractor range from $ 2,800 in Coconino County to $ 5,600
in Yavapai County. The high per- family costs in Yavapai County resulted from
significantly lower than expected enrollment figures. While the CAP Fund coordinator
attributes this to a lower than expected birth rate, we were unable to determine the
exact causes at this time. At all other sites, the cost per family was lower than $ 4,000.
These figures were obtained by dividing the total program funds by the number of
enrolled participants. Because the total program fund also included contractor in- kind
contributions, these figures are higher than what the State actually spent per family.
The State spent only $ 3,269 per family, a reduction of about $ 400 from the actual cost
per family. Contractor contributions made the Program more cost- efficient from the
State's perspective. Finding I11 ( see pages 21 through 24) presents detailed cost figures
for the program.
6. lnformation concerning progress of program participants in achieving goals and
objectives.
This report does not address the progress of participants in achieving program goals
and objectives. During the part- year implementation of the Program in fiscal year 1995,
most participants remained in the most intensive service delivery level. The rate of
progress will become clearer during the next fiscal year. Information demonstrating
progress toward reduced child abuse and neglect, promotion of child wellness and
proper development, strengthened family relations and family unity, and reduced
substance dependency for the participating families will be available in 1996, after
programs deliver services for one full program year.
Instruments that are being used to measure program participants' progress in
achieving goals and objectives are described in Appendix II.
7. Recommendations regarding program administration.
Overall, the program administration was efficient and the administrative tasks at the
DES level were completed in a timely fashion. At this time, we have two recommenda-tions
to make regarding program administration.
a. DES should not amend its Healthy Families contracts too frequently. Doing so may
create inequity among different contractors and contribute to higher per- family
costs in certain areas.
b. DES should enhance its collaborative efforts with other agencies such as DHS and
ADE to provide more comprehensive services to the participants. We recognize
that at this time the Department has already made significant progress toward
collaboration with DHS and encourage such efforts at all levels.
8. Recommendations regarding informational materials distributed through the
programs.
The Healthy Families Program distributes informational materials in accordance with
the state- mandated services. Our office reviewed selected materials related to child
development, parent- child attachment, and bonding issues and found them to
adequately address the program needs ( see page 15). No recommendation is deemed
necessary regarding informational materials distributed through the Program at this
time.
9. Recommendations pertaining to program expansion.
Recommendations regarding program expansion can only be made after the programs
have operated for at least the mandated period of three years and some outcome
information is available.
Agency Response
ARIZONA DEPARTMENT OF ECONOMIC SECURITY
171 7 W. Jefferson - P. O. Box 61 23 - Phoenix, AZ 85005
Fife Symington Linda J. Blessing, DPA
I Governor Director
December 20,1995
Mr. Douglas R. Norton, CPA
Auditor General
Office of the Auditor General
2910 North 44th Street, Suite 410
Phoenix, Arizona 85004
Dear Mr. Norton:
Thank you for the opportunity to review the preliminary draft report of the first Annual
Evaluation of the Healthy Families Pilot Program.
I am pleased that this report captures the tone and direction of the initial steps the
department has taken to establish a Healthy Families Program in Arizona. As you are
aware, the reform of Arizona's child welfare services is the department's top priority.
Toward that end, the establishment of a successful Healthy Families Program is central to
our prevention efforts.
Please know that the department will thoroughly and enthusiastically analyze and
implement the recommendations for improvement contained in the report. Finally, the
department wishes to express its appreciation to you and your staff for the time and effort
invested in this critically important audit.
Sincerely,
I<&
inda J. Blessing
Appendix I
Definition and Types of Child Abuse
Definition and Types
of Child Abuse
Child abuse occurs when a parent or caretaker inflicts or allows the infliction of physical,
sexual, or emotional abuse, neglect, or abandonment. The Child Protective Services ( CPS)
classifies child abuse cases into the following broad categories:
Physical Abuse - Physical abuse is any non- accidental injury to a child under the
age of 18 by a parent or caretaker. Non- accidental injuries may include beatings,
shaking, burns, human bites, strangulation or immersion in scalding water, with
resulting bruises and welts, broken bones, scars, or internal injuries.
Sexual Abuse - Child sexual abuse is the exploitation of a child or adolescent for
the sexual gratification of another person. It includes behaviors such as intercourse,
sodomy, oral- genital stimulation, verbal stimulation, exhibitionism, voyeurism,
fondling, and involving a child in prostitution or the production of pornography.
Incest is sexual abuse that occurs within a family. The abuser may be a parent, step-parent,
grandparent, sibling, cousin, or other family member.
Neglect - Neglect is the chronic failure of a parent or caretaker to provide a child
under 18 with adequate food, clothing, medical care, protection and supervision.
Emotional Abuse - Emotional abuse is a chronic pattern of behaviors, such as
belittling, humiliating, and ridiculing a child. Related to this, emotional neglect is the
consistent failure of a parent or caretaker to provide a child with appropriate support,
attention, and affection.
Causes of Child Abuse
Child abuse may occur in various situations within and outside the family. Experts have
identified four levels of factors that may contribute to child maltreatment. These are 1)
individual, 2) family, 3) community, and 4) culture. The Healthy Families Program aims
at alleviating mainly the individual and family level factors. Some of these factors are
marital conflict, decreased family cohesion, lack of intellectual and recreational activities,
deficient parenting skills, and in general, an above- average negative adult- child interaction
pattern. Studies have also identified these factors to be intergenerational in nature. For
instance, parents with a childhood history of abuse report more family conflicts than those
without.
a- i
Appendix I1
Outcome Evaluation Instruments
Pursuant to the legislative mandate the Office of the Auditor General is required to report
on the Program's impact on family unity, child abuse, child development, and welfare
dependence in the final outcome evaluation due in December of 1997. Accordingly, in
collaboration with program personnel, we have selected the following instruments which
capture most of these constructs. For impact on welfare dependence, we are negotiating
with the Department of Economic Security for instituting a tracking mechanism across
some major databases that maintain welfare recipient data. For the other factors, the
following instruments have been selected for use with the Program.
Instruments
Fnltzily Unity
Family Adaptability and Cohesion Evaluation Scale II ( FACES II*)
Authors: David Olson, U. of Minnesota; Volker Thonzas, Purdue University
FACES I1 is a self- report instrument designed to measure both perceived and ideal family
function. This is accomplished by measuring two key constructs - family cohesion and family
adaptability.
There are two separate sub- scales for the two constructs. Each subscale consists of 10
items. The cohesion subscale ( alpha = .77) is more reliable than the adaptability subscale
( alpha = .62). Items follow a 5- point Likert scale ranging from almost never to almost
always.
FACES I1 is usually administered to both parents and adolescents in a family. In the
context of Healthy Families, the instrument could be administered to the parents only.
This instrument will be fast and easy to use and will not require extensive training for
the home visitors.
Note: FACES 111, the latest version that we reviewed, was not commercially available at
the time the instrument was selected. The authors recommended using FACES I1 which,
they assured us, will function the same way as FACES 111.
Honze E~ zviro~ znzent
Home Observation fm the Measurement of the Environment ( HOME)
Authors: Bettye M. Caldwell G.' Robert H. Bradley, University of Arkansas
a- ii
HOME is a widely used observation and interview instrument that measures some key
elements in a child's home environment. Different forms are used for three different age
groups - 0- 3 years ( infants & toddlers), 3- 6 years ( preschoolers), 6- 10 years ( elementary
school age). Healthy Families Arizona uses only the first one meant for infants and
toddlers. The 0- 3 years form consists of six subscales measuring the following six factors
of home environment - 1) Emotional and verbal responsivity of parent, 2) acceptance of
child's behavior, 3) organization of physical and temporal environment, 4) provision of
appropriate play materials, 5) parent involvement with child, and 6) opportunities for variety
in daily stimulation.
Apart from the measurement needs, this instrument allows the assessment workers to
identify areas of home environment that need strengthening or improvement. As a
measurement instrument it allows comparison with the FACES 11, which is a self- report
instrument.
Clrild Abzcse Potential
Child Abuse Potential ( CAP) Inventory
Author: Joel Milner, Northern Illinois University
CAP Inventory ( Form VI) is a self- report physical child abuse screening device. The
inventory contains a total of 160 items in a dichotomous, agree/ disagree format. Of these,
77 items form a physical child abuse scale. The scale has a third- grade readability level.
The physical abuse scale contains six descriptive factors: distress, rigidity, unhappiness,
problems with child and self, problems with family, and problems from others. The CAP
Inventory also contains three validity scales: a lie scale, a random response scale, and an
inconsistency scale. The validity scales are used in various combinations to form three
response distortion indices: the faking- good index, the faking- bad index, and the random
response index.
The author reports that the inventory has been successfully employed to evaluate child
abuse prevention programs. In one of the larger studies involving several sites, the
National Council on Prevention of Child Abuse used this instrument and found significant
decreases in abuse scores from the pre- intervention period to the post- intervention period.
Clrild Develoylnent
Ages and Stages Questionnaire ( ASQ)
Authors: Jane Squires, LaWanda Potter, Diane Bricker, University of Oregon
ASQ is a parent- completed child monitoring system. The questionnaire can be
administered 11 times, until the child turns 4. However, depending on program needs
it can be administered fewer times. After assessing the Program and evaluation needs
of Healthy Families Arizona, we decided to use the questionnaire 8 times.
a- iii
Each administration time has a corresponding set of developmentally appropriate
questions. The questionnaire addresses the following five areas of child development:
1) Communication, 2) Gross Motor, 3) Fine Motor, 4) Problem Solving, 5) Personal- Social.
In addition, an overall section asks about general parental concerns.
ASQ is not a diagnostic tool. It can be used to identify developmental delays, but once
detected, the child needs to be assessed with a diagnostic instrument by a trained
professional. However, research shows the ASQ to have strong correlations with
established diagnostic instruments.
The fact that the instrument can be used only to identify problems but not to diagnose
them, serves the Healthy Families Program's needs well. The family support specialist
can refer the identified child to a professional for further assessment.
a- iv