Annual evaluation, Healthy Families Program

              State of Arizona
Office
of the
Auditor General
PERFORMANCE AUDIT
Report to the Arizona Legislature
By Debra K. Davenport
Auditor General
HEALTHY FAMILIES
PROGRAM
February 2001
Report No. 01-02
The Auditor General is appointed by the Joint Legislative Audit Committee, a bipartisan committee
composed of five senators and five representatives. Her mission is to provide independent and impar-tial
information and specific recommendations to improve the operations of state and local government
entities. To this end, she provides financial audits and accounting services to the state and political
subdivisions and performance audits of state agencies and the programs they administer.
Audit Staff
Carol Cullen—Manager
and Contact Person (602) 553-0333
Joanne Dukeshire—Team Leader
Laurie Cohen—Team Member
Copies of the Auditor General’s reports are free.
You may request them by contacting us at:
Office of the Auditor General
2910 N. 44th Street, Suite 410
Phoenix, AZ 85018
(602) 553-0333
Additionally, many of our reports can be found in electronic format at:
www.auditorgen.state.az.us
2910 NORTH 44th STREET • SUITE 410 • PHOENIX, ARIZONA 85018 • (602) 553-0333 • FAX (602) 553-0051
DEBRA K. DAVENPORT, CPA
AUDITOR GENERAL
STATE OF ARIZONA
OFFICE OF THE
AUDITOR GENERAL
February 15, 2001
Members of the Arizona Legislature
The Honorable Jane Dee Hull, Governor
Mr. John Clayton, Director
Department of Economic Security
Transmitted herewith is a report of the Auditor General, an evaluation of the Healthy families
Program. This evaluation was conducted pursuant to A.R.S. §41-1279.08. I am also
transmitting with this report a copy of the Report Highlights for this evaluation to provide a
quick summary for your convenience.
As outlined in its response, the Department agrees with all of the findings and
recommendations.
My staff and I will be pleased to discuss or clarify items in the report.
This report will be released to the public on February 16, 2001.
Sincerely,
Debbie Davenport
Auditor General
Enclosure
i
OFFICE OF THE AUDITOR GENERAL
SUMMARY
The Office of the Auditor General has completed an evaluation
of the Healthy Families Program. The Office of the Auditor Gen-eral
is required pursuant to Arizona Revised Statutes (A.R.S.)
§41-1279.08 to evaluate the Healthy Families Program annually.
This report contains information related to program procedures
and effectiveness.
Healthy Families—Arizona is a child abuse prevention and child
wellness and development program administered by the De-partment
of Economic Security (DES). The Healthy Families
Program has five goals: (1) reduce child abuse and neglect; (2)
promote child wellness and proper development; (3) strengthen
family relations; (4) promote family unity; and (5) reduce de-pendency
on drugs and alcohol.
The program coordinates with hospitals to identify mothers
giving birth in specific geographic areas whose family character-istics
may place them at risk for committing child abuse or ne-glect,
and/or whose babies may face increased health risk.
Healthy Families provides services through various contractors,
which include medical centers, local health departments, and
social service agencies. These contractors provide services at 23
sites in 10 counties.
The program is based on a home-visitation model. Home visitors
regularly visit families to offer support, educational materials,
and referrals to needed resources. Home visitors also help fami-lies
develop family support plans and encourage positive parent-child
interaction to strengthen bonds and promote development.
Participation in the program is voluntary and may continue for
up to five years. In fiscal year 2000, 1,254 families were enrolled
in the program.
Summary
ii
OFFICE OF THE AUDITOR GENERAL
Healthy Families Participant Child
Abuse Rates Do Not Differ Significantly
from the Comparison Group
(See pages 13 through 19)
For the second year in a row, comparison of child abuse rates for
Healthy Families participants and a comparison group of fami-lies
who left the program shows that abuse rates were not sig-nificantly
different. Specifically, 1.7 percent of participant fami-lies
received a substantiated report of abuse and 1.8 percent of
families who left the program before receiving four home visits
received a substantiated report. These rates are comparable to
the rates reported in the previous Office of the Auditor General
evaluation of the program (see Report No. 00-1), in which 1.6
percent of participants and 1.4 percent of the comparison group
received substantiated reports of abuse.
While one possible explanation for these findings might be that
the program does not have a significant effect on child abuse,
other explanations may also account for the similarity in out-comes
between the two groups. First, it is possible that the effects
of the program on preventing child abuse cannot be adequately
measured by the current evaluation design because of changes in
DES procedures for substantiating child abuse reports. In par-ticular,
the occurrence of substantiated abuse reports has de-creased
since the implementation of an appeals process. Second,
preliminary analysis of when abuse occurs for both program
participants and the comparison group suggests that abuse is
likely to occur a year or more after enrollment, and in most cases,
after the family has left the program. Thus, participants may not
be enrolled in the program at the time when the risk of child
abuse increases.
Healthy Families Program Provides
Health Referrals, Encourages
On-Time Immunizations, and
Promotes Safety
(See pages 21 through 26)
The Healthy Families Program has been successful in providing
health referrals, encouraging parents to get their children immu-nized,
and in promoting home safety. On average, the majority
Summary
iii
OFFICE OF THE AUDITOR GENERAL
of children in the program are fully immunized and most chil-dren
have a primary health care provider. Additionally, Healthy
Families staff regularly conduct assessments of the physical and
social development of children enrolled in the program. Parents
improved their compliance with home safety measures such as
covering electrical outlets and keeping poisons and choking
hazards out of reach.
Parenting Stress Measure
Shows Improved
Family Relations
(See pages 27 through 29)
Healthy Families parents showed significant decreases in indica-tors
of parental stress after one year in the program, suggesting
that program services may improve parent-child relations. In
addition, parents with higher stress levels related to areas such as
social isolation and depression also received referrals to mental
health services at a higher rate than parents with lower stress. To
assist families in improving parent-child interaction, home visi-tors
regularly provide emotional support and information to
help parents understand their child’s behavior.
Other Pertinent Information
(See pages 31 through 34)
The previous report issued by the Office of the Auditor General
(see Report No. 00-1) offered several recommendations to im-prove
the measurement of the program’s effect on family func-tioning
and reduction of substance abuse. The previous evalua-tion
also recommends tracking referrals for children with devel-opmental
delays. To address these recommendations, the pro-gram
discontinued using the Home Observation for Measure-ment
of the Environment (HOME) questionnaire and will in-stead
use the Parenting Stress Index (PSI) to measure the pro-gram’s
effect on family relations. The program also replaced the
CAGE substance abuse screen with a new questionnaire that can
measure the program’s effect on reducing drug/alcohol depend-ency
over time. Finally, the program added questions to its fam-ily
update form, which will enable staff to track whether children
with developmental delays are offered the appropriate referrals.
Summary
iv
OFFICE OF THE AUDITOR GENERAL
Statutory Annual Evaluation Components
(See pages 35 through 44)
This report also contains information that addresses various
components of the Healthy Families evaluation statute. These
components include information regarding participant demo-graphics
such as mother’s age, marital status, income, ethnicity,
and household size. Other areas covered in the statutory compo-nents
section contain information about enrollment and disen-rollment
and employment.
v
OFFICE OF THE AUDITOR GENERAL
TABLE OF CONTENTS
Page
Introduction and Background.......................... 1
Finding I: Healthy Families Participant
Child Abuse Rates Do Not
Differ Significantly from
the Comparison Group................................ 13
Child Abuse Rates for
Healthy Families Participants and
Comparison Group Similar............................................ 13
Possible Explanations
of Results......................................................................... 16
Finding II: Healthy Families Program
Provides Health Referrals, Encourages
On-Time Immunizations, and Promotes
Safety ............................................................ 21
Background..................................................................... 21
Participants Receive
Developmental Screenings,
Referrals, and On-Time
Immunizations................................................................ 22
Participants Show Significant
Improvements in Home Safety...................................... 24
Recommendations .......................................................... 26
Finding III: Parenting Stress Measure
Shows Improved Family Relations............. 27
Background..................................................................... 27
Participants’ Stress Reduced
After 12 Months.............................................................. 27
Table of Contents
vi
OFFICE OF THE AUDITOR GENERAL
TABLE OF CONTENTS (Cont’d)
Page
Finding III: (Concl’d)
Parents with Higher Stress
More Likely to Receive Mental
Health Referrals .............................................................. 29
Home Visitors Provide
Information on
Parent-Child Bonding..................................................... 29
Other Pertinent Information ............................. 31
Statutory Annual Evaluation Components...... 35
Appendix ........................................................... a-i
Agency Response
Figures
Figure 1 Healthy Families Program
Contract Provider Locations
Year Ended June 30, 2000............................. 8
Figure 2 Healthy Families Program
Number of Participants and Comparison
Group Members by County
January 1994 through May 2000.................. 37
Figure 3 Healthy Families Program
Reasons for Exit
January 1994 through May 2000.................. 40
Figure 4 Healthy Families Program
Employment Rates of Mothers and
Fathers in Program
January 1994 through May 2000................. 44
Table of Contents
vii
OFFICE OF THE AUDITOR GENERAL
TABLE OF CONTENTS (Cont’d)
Page
Tables
Table 1 Healthy Families Program
Schedule of Distributions by Funding
Source and Contractor, and Average
State Contribution per Family
Year Ended June 30, 2000
(Unaudited)................................................... 7
Table 2 Healthy Families Program
Percentage of Participants and Comparison
Group Members with Substantiated and
Unsubstantiated Reports of Abuse and
Referrals to Family Builders
Reports Received Between
January 1, 1998 and May 31, 2000................ 15
Table 3 Healthy Families Program
Percentage of Participants and Comparison
Group Members with Substantiated
Abuse Reports
Office of the Auditor General
Evaluations.................................................... 16
Table 4 Healthy Families Program
Ages and Stages Questionnaire
Percentage of Children Developing at
Age-Appropriate Levels
January 1994 to May 2000............................. 24
Table 5 Healthy Families Program
Percentage of Safety Measures Implemented
by Age of Child
July 1998 to May 2000................................... 25
Table 6 Healthy Families Program
Average Parenting Stress Scores
After 3 Weeks and 12 Months
in the Program
July 1998 to May 2000................................... 28
Table of Contents
viii
OFFICE OF THE AUDITOR GENERAL
TABLE OF CONTENTS (Concl’d)
Page
Tables (Concl’d)
Table 7 Healthy Families Program
Participant and Comparison Group
Member Profile
January 1994 through May 2000.................. 36
Table 8 Healthy Families Program
Participants and Comparison Group Members
Profile of Family Characteristics
January 1994 through May 2000.................. 38
Table 9 Healthy Families Program
Months of Participation for Families at Exit
January 1994 through May 2000.................. 40
1
OFFICE OF THE AUDITOR GENERAL
INTRODUCTION AND BACKGROUND
The Office of the Auditor General has completed an evaluation
of the Healthy Families Program. According to Arizona Revised
Statutes (A.R.S.) §41-1279.08, the Office of the Auditor General is
required to evaluate the Healthy Families Program annually.
This report contains information regarding the program's proce-dures
and outcomes.
Healthy Families Offers
a Preventative Approach to the
Problem of Child Abuse
Healthy Families—Arizona is a child abuse prevention and child
wellness and development program administered by the De-partment
of Economic Security (DES). Preventing child abuse is a
serious concern in Arizona, as in other states. During fiscal year
1999, Arizona's Department of Economic Security, Division of
Child Protective Services received 32,631 reports of maltreatment
toward children. Twenty-five percent of these reports were sub-sequently
substantiated.1
The State Legislature established Healthy Families—Arizona as a
pilot program in 1994, and gave it permanent program status in
1998. Healthy Families attempts to identify families at risk for
committing child abuse or neglect and that may have poor
health outcomes. The program provides participants with sup-port,
education about child development and referrals for nutri-tion,
medical care, counseling, and education and employment
programs. Arizona's program is part of a national initiative pro-moted
by Prevent Child Abuse America. It is currently offered in
39 states, the District of Columbia, and Canada and serves over
400 communities. Program enrollment and participation are
1 Department of Economic Security. Child Welfare Reporting Requirements:
Annual Report for the Period of July 1, 1998 through June 30, 1999. Substanti-ated
reports are those reports DES determines met the standard of prob-able
cause to conclude that abuse occurred. Note: Of the total reports for
fiscal year 1999, 153 fell within the jurisdiction of military or tribal gov-ernments.
Healthy Families is a child
abuse prevention and child
wellness and development
program.
Introduction and Background
2
OFFICE OF THE AUDITOR GENERAL
voluntary. However, nationally, 90 percent of families offered
Healthy Families services enroll in the program.
The Healthy Families Program is based on home visitation.
Home visitors regularly visit families to offer support, educa-tional
materials, and referrals to various medical, mental health,
and social services. (For information regarding home visitor
training see the Training, Quality Assurance, and Credentialing
section on page 5.) During the home visit, home visitors help
families develop family support plans in which parents identify
goals for themselves and their children and encourage positive
parent-child interaction to strengthen bonds and promote devel-opment.
Home visitors also determine which services a family
might need, and provide educational material on child health,
developmental milestones, safety, discipline, and nutrition. In
addition to home visits, the program holds parent and play
group meetings to provide participants with the opportunity to
meet other families in their community. Families may participate
in the program for five years.
Program Stresses Early
Screening and Intervention
Healthy Families establishes its family visitation within the first
three months after a child is born. By doing so, the program at-tempts
to help families establish positive parent-child relation-ships
and positive child development outcomes early on. To
identify families most at risk, Healthy Families coordinates with
hospitals to screen mothers giving birth within the program's
service area (see Figure 1, footnote 1, page 8). To be eligible for
program enrollment, participants cannot have any substantiated
reports of child abuse on file with Child Protective Services (CPS)
and must reside in a Healthy Families service area. The intake
process is completed in two stages. Specifically,
n The first stage includes a screening in which parents are iden-tified
as possibly at risk for child abuse based on a combina-tion
of factors such as inadequate income, unstable housing,
lack of a high school diploma, inconsistent or late prenatal
care, and being unmarried or separated. The screening proc-ess
is usually completed while the mother is still in the hospi-tal.
The program refers to those screenings that identify at-
The program provides services
through home visitation.
Participants are enrolled
within the first three months
after their child is born.
Introduction and Background
3
OFFICE OF THE AUDITOR GENERAL
risk parents as positive screens. During fiscal year 2000,
Healthy Families screened 17,005 families. Of this total, 49
percent (8,382) screened positive.
n During the second stage, parents who receive a positive
screen are contacted by a Healthy Families worker about
completing an assessment. If the parent consents to complet-ing
the assessment, he or she is interviewed about topics in-cluding
family history, history of substance abuse or criminal
activity, stress, self-esteem issues, expectations for the child,
and plans regarding discipline. If the parent receives a posi-tive
assessment, indicating a potential risk for committing
child abuse, he or she is invited to enroll in the program
pending a CPS check. Assessments are completed a few days
after the screening process either in the hospital, in the par-ent's
home, or over the telephone.
Families who received a positive screen and were not offered
the assessment include those who were ineligible for the pro-gram
because they have a substantiated abuse report on file
with CPS, could not be reached, or lived in an area in which
the Healthy Families Program was full. During fiscal year
2000, of those families contacted and offered the assessment,
1,069 completed the assessment and 533 refused the assess-ment.
Nine-hundred thirty-one families assessed positive for
child abuse risk and 854 of these families enrolled in the pro-gram.
Once enrolled in the program, Healthy Families participants are
required by statute to perform community service in exchange
for program services. The program has established that families
must complete 12 hours of community service per year of par-ticipation.
Home visitors at each site are responsible for serving 12 to 25
families. The frequency of home visits varies according to the
family's level of participation. Visits are more frequent at the
beginning of the family's participation and less frequent as the
family becomes more self-sufficient. Increased self-sufficiency is
determined by a variety of factors such as a stable home envi-ronment,
ability to access resources independently, and utiliza-tion
of support networks.
Introduction and Background
4
OFFICE OF THE AUDITOR GENERAL
Program Goals
Five program goals are outlined in A.R.S. §8-701. Healthy Fami-lies
goals and the procedures used by the program to address
these goals are listed below:
n Goal 1: Reduce child abuse and neglect—To reduce child
abuse and neglect, home visitors assist families with enhanc-ing
parent-child interaction, attachment, and bonding. They
provide information about child safety, discipline, and meth-ods
for anger management, and assess parent stress levels
and home safety. As appropriate, home visitors may provide
referrals for counseling and treatment services to address
substance abuse and domestic violence.
n Goal 2: Promote child wellness and proper develop-ment—
Home visitors provide information on child health
and nutrition and encourage families to have a primary care
physician, to receive well-baby check-ups, and to have their
children immunized. They also conduct periodic develop-mental
assessments and provide referrals to medical provid-ers
and the Arizona Early Intervention Program (AzEIP) for
children with potential delays.
n Goals 3 and 4: Strengthen family relations and promote
family unity—Healthy Families uses an approach that fo-cuses
services on the family and building on family strengths.
They encourage positive parent-child interaction and discuss
family relationship issues with participants. When appropri-ate,
they refer participants to counselors to discuss family re-lationship
issues. Although most primary caregivers who en-roll
in the program are mothers, the program encourages fa-thers
and other family members to participate in home visits
and group activities.
n Goal 5: Reduce dependency on drugs and alcohol—To
address issues of substance abuse, Healthy Families began
administering a substance abuse assessment instrument in
July 1998. When alcohol or drug abuse problems are identi-fied,
home visitors are trained to discuss the issue with the
participant and/or provide a referral for treatment.
Introduction and Background
5
OFFICE OF THE AUDITOR GENERAL
Training, Quality Assurance,
and Credentialing
Home visitors are trained home visiting specialists. They must
have at least a high school diploma to be employed by the pro-gram,
but many also have college degrees. Home visitors receive
a minimum of 30 hours of training a year in subjects such as
child development, substance abuse, and identifying and report-ing
child abuse and neglect. New home visitors are required to
complete four and one-half days of initial training. New staff also
complete self-guided training modules under the supervision of
site managers. Initial training includes methods for identifying
child abuse and neglect, infant growth and development, and
methods for interacting with families and encouraging positive
parent-child interaction. Finally, new staff must also complete
on-the-job training in which they observe home visits conducted
by more experienced staff.
To help ensure that Healthy Families’ sites are in compliance
with program policies and procedures, DES contracts with an
evaluation firm to provide quality assurance and database man-agement.
Specifically, quality assurance coordinators from this
firm visit each site at least twice a year to review participant files
for accuracy. This firm also collects and maintains all program
data from each Healthy Families site and is responsible for test-ing
this data to ensure that it is reliable and accurate.
In April 2000, Healthy Families—Arizona became the first pro-gram
in the nation to receive a statewide credential from Prevent
Child Abuse America and the Council on Accreditation. During
the credentialing process, the program had to demonstrate ad-herence
to national standards by submitting information on
training, technical assistance, policies, quality assurance, evalua-tion,
and administration. Each site completed a self-assessment
of its performance according to 12 critical elements and 138 stan-dards
of best practice. Seven sites received formal visits from
national credentialing committee representatives. During the site
visits, committee representatives evaluated the quality of service
provided by each site.
The credential attests that Healthy Families—Arizona meets
nationally established research-based standards for quality ser-
Healthy Families—Arizona
was first in the nation to re-ceive
a statewide credential.
Introduction and Background
6
OFFICE OF THE AUDITOR GENERAL
vice delivery, and best practice standards for management and
administration.
Contractors
and Appropriations
Healthy Families provides services through various contractors,
which include medical centers, local health departments, and
social service agencies. In fiscal year 2000, DES had 11 contracts
with 10 separate contractors. These contractors provide services
at 23 sites in 10 counties (see Figure 1, page 8). For fiscal year
2000, two new sites were added in Maricopa County. In addi-tion,
the Pascua Yaqui Tribe site offered services in previous
years through a federal grant but was awarded program funding
for the first time this year. Each site serves selected areas within a
40-mile radius of the contractor's office. These areas, identified by
their zip codes, are chosen based on the number of live births per
year, the number of CPS reports for children ages 0 to 5 years,
and other factors, including low income and under-utilization of
health care services. Additionally, Healthy Families contracts
with an evaluation firm for database management, evaluation,
quality assurance, and training.
For fiscal year 2000, program funding for Healthy Families to-taled
about $5.6 million, $4 million of which was from the State’s
General Fund appropriation. In fiscal years 1997, 1998, and 1999
the General Fund appropriation totaled about $3 million for each
fiscal year. Other funding sources for the program include DES'
Child Abuse Prevention Fund, federal grants, contractor contri-butions,
and private donations (see Table 1, page 7).
Scope and Methodology
A.R.S. §41-1279.08 mandates that the Office of the Auditor Gen-eral
evaluate the Healthy Families Program. The statute specifies
that the evaluation will include an assessment of the program's
effectiveness in achieving its goals. According to the statute,
additional evaluation requirements include providing informa-tion
about the level and scope of program services, enrollment
eligibility, participant demographic characteristics, long-term
Healthy Families provides
services through local contrac-tors.
Introduction and Background
7
OFFICE OF THE AUDITOR GENERAL
Table 1
Healthy Families Program
Schedule of Distributions by Funding Source and Contractor, and
Average State Contribution per Family 1
Year Ended June 30, 2000
(Unaudited)
Funding Source
Contractor:
State
General
Fund
Child Abuse
Prevention
Fund
Tobacco
Taxes 2
Federal
Grants 3 Total
Southwest Human Development $1,098,787 $179,892 $132,580 $195,837 $1,607,096
Child and Family Resources:
Urban 769,980 106,530 213,923 1,090,433
Rural 539,699 184,819 183,569 908,087
Coconino County Department of Pub-lic
Health 296,761 33,506 31,586 1,325 363,178
Yavapai Regional Medical Center 215,587 22,773 25,534 263,894
Pinal County Department of Public
Health 171,605 45,512 24,735 241,852
Lake Havasu Social Services Inter-agency
Council 190,772 20,470 20,470 231,712
Yuma County EXCEL Group 108,903 40,743 20,008 169,654
Verde Valley Medical Center 88,875 30,415 16,590 135,880
Pascua Yaqui Tribe 8,254 38,730 27,792 74,776
Parents Anonymous of Arizona (Tuba
City) 32,940 20,790 10,530   64,260
LeCroy & Milligan (provides evalua-tion,
quality assurance, and training
statewide) 445,043     36,832
   
  481,875
Total monies distributed $3,967,206 $ 724,180 $ 716,357 $224,954 $5,632,697
Average state contribution per family 4 $ 3,164 $ 577 $ 571 $ 4,312
1 Department-distributed monies to contractor, rather than contractor expenditures, are presented. Contractors do not
report expenditures in a timely manner and funding sources appear to be approximate expenditures of Department-distributed
monies. In addition, the schedule excludes the required contractor matches of at least 10 percent, because con-tractors
do not report the value of their noncash resources, such as office space and personnel, on a consistent basis.
2 The program received monies from the Parent’s Commission on Drug Education and Prevention beginning in January
2000. The Commission’s funding comes from tobacco taxes; therefore, this schedule presents the source of funding as to-bacco
taxes.
3 Consists of approximately $164,400 and $60,600 received from the Community Based Family Resource and Support and
the Safe and Stable Families Act federal grants, respectively.
4 Calculation based on the total number of families served (1,254) during the fiscal year, including families who have
disenrolled. Calculation excludes the federal grants and contractor contributions to arrive at the State’s contribution per
family.
Source: Auditor General staff analysis of financial information provided by the Department of Economic Security.
Introduction and Background
8
OFFICE OF THE AUDITOR GENERAL
Figure 1
Healthy Families Program
Contract Provider Locations1
Year Ended June 30, 2000
u
Coconino
County
Health
Serving Page and
Flagstaff areas
u
Verde Valley
Medical Center
Serving Verde Valley area
u
Yavapai Regional
Medical Center
Serving Prescott area
u
Inter-Agency
Council
Serving Lake Havasu
and Parker areas
u
Southwest Human
Development
Serving 7 sites in
Maricopa County
u Excel Group
Serving Yuma area
u
Pinal County
Health
Serving Casa Grande
and Coolidge areas
Child & Familyuu
Resources
Serving 4 sites in
Metro Tucson
u
u
Child & Family
Resources
Serving Nogales
Sierra Vista, and Douglas/Bisbee
areas
u
1 Each site serves selected areas within a 40-mile radius of the contractor’s office. These areas, identified by their zip codes,
are chosen based on the number of live births per year, the number of CPS reports for children ages 0 to 5 years, and
other factors including low income and under-utilization of health care services.
Source: Auditor General staff analysis of Department of Economic Security Healthy Families contracts for the
year ended June 30, 2000.
Pascua
Yaqui
Tribe
Parents
Anonymous
Serving
Tuba City
u
u
¿
Introduction and Background
9
OFFICE OF THE AUDITOR GENERAL
savings associated with the program and rates of enrollment and
disenrollment.
During this annual evaluation, Auditor General staff visited 11 of
23 sites. Site visits included:
n An interview with site manager(s);
n A group interview with home visitors and assessment work-ers;
n Accompanying home visitors on 1 to 2 home visits. During
home visits, evaluators interviewed 13 program participants
and conducted 13 structured observations of the visit; and
n Reviews of 110 participant files to check the accuracy of
Healthy Families databases. Specifically, information ob-tained
from the file reviews was compared to information in
the program's database. This comparison showed that data
was reliable.
Evaluators also analyzed data from the following assessment
tools:
n The Parenting Stress Index, which provides a measure of
parental stress;
n The Child Safety Checklist, which measures the safety of the
home environment;
n The Ages and Stages Questionnaire, which assesses children
for potential developmental delays.
A description of the assessment tools is included in the Appen-dix
(see pages a-i through a-ii).
In addition to the assessment tools, evaluators collected and
analyzed:
n Approximately 83,000 records of substantiated and unsub-stantiated
CPS reports to determine the number of reports re-ceived
by program participants and members of the com-parison
group;
Introduction and Background
10
OFFICE OF THE AUDITOR GENERAL
n Immunization rates of children in the program;
n Healthy Families participant usage of two types of public
assistance: Temporary Assistance for Needy Families (TANF)
and food stamps.
In conducting the analyses of child abuse rates, evaluators com-pared
substantiated and unsubstantiated report rates for Healthy
Families participants with those of a comparison group. Healthy
Families participants included in the analysis of abuse rates are
those participants who enrolled in the program from July 1, 1997
to November 30, 1999, received at least four home visits, and
participated in the program for at least six months. The compari-son
group comprised individuals who enrolled in Healthy Fami-lies
between July 1, 1997 and November 30, 1999, but left the
program before receiving four home visits. Evaluators also ana-lyzed
CPS data to determine how many participants and com-parison
group members were referred to Family Builders, which
is a program that provides services to families who have been
reported to CPS for low and potential risk of child abuse.
The evaluation includes findings and recommendations in the
following areas:
n Child abuse rates for Healthy Families participants and the
comparison group are not significantly different;
n The Healthy Families program succeeds in providing health
referrals, encouraging on-time immunizations, and promo-ting
safety.
n Parenting stress measure shows improved family relations.
In addition to these finding areas, the evaluation also contains an
Other Pertinent Information section (see pages 31 through 34),
regarding the implementation of new methods for measuring
program outcomes. These changes address recommendations
made in the last Auditor General Report (Report No. 00-1).
Introduction and Background
11
OFFICE OF THE AUDITOR GENERAL
Acknowledgements
The Auditor General and staff express appreciation to the Direc-tor
of the Department of Economic Security and DES staff in the
Division of Children, Youth, and Families and the Office of
Evaluation; the Manager of the Office of Prevention and Family
Support, Healthy Families Statewide Coordinator, and Program
Specialist; Healthy Families Data Management, Evaluation and
Quality Assurance staff; Healthy Families supervisors, home
visitors, and family assessment workers; and the Department of
Health Services' Bureau of Epidemiology and Disease Control
for their cooperation and assistance during this evaluation.
12
OFFICE OF THE AUDITOR GENERAL
(This Page Intentionally Left Blank)
13
OFFICE OF THE AUDITOR GENERAL
FINDING I HEALTHY FAMILIES PARTICIPANT
CHILD ABUSE RATES DO NOT
DIFFER SIGNIFICANTLY FROM
THE COMPARISON GROUP
For the second year in a row, comparison of child abuse rates for
Healthy Families participants and families who left the program
show that abuse rates were not significantly different. While one
possible explanation of these findings might be that the program
does not have a significant effect on child abuse, other explana-tions
may also account for the similarity in outcomes between
the two groups.
Child Abuse Rates for Healthy Families
Participants and Comparison Group Similar
Child abuse rates for families who participated in the Healthy
Families Program did not differ significantly from child abuse
rates for a comparison group of families who left the program
after only a short time. Additionally, substantiated reports for
many participants and control group members occurred a year
or more after program enrollment. Finally, abuse rates found in
this report are consistent with rates found in the Auditor Gen-eral’s
last evaluation (see Report No. 00-1).
Evaluation compared Healthy Families participants with others
who left the program—To assess the program’s effect on child
abuse, evaluators reviewed CPS records in the Children’s Infor-mation
Library Data Source (CHILDS).1,2 The analysis included
1 The data system contains abuse report information such as the alleged
abuser, report date, reporting source, and investigation findings.
2 Auditor General staff also reviewed the CHILDS system’s intake, investi-gation,
and security procedures to determine whether adequate controls
for ensuring the reliability and accuracy of data were in place. Staff con-cluded
that such controls were in place.
Finding I
14
OFFICE OF THE AUDITOR GENERAL
all substantiated and unsubstantiated CPS reports made at least
six months after intake in Healthy Families for participants and
comparison group members. This time period was selected to
allow participants to have sufficient program exposure before
assessing whether the program had an impact. For instance, for
families with intake dates of July 1, 1997, evaluators examined
reports made on or after January 1, 1998. Evaluators also exam-ined
the proportion of participants and comparison group mem-bers
who received referrals to Family Builders during the same
period.
The Healthy Families participant group comprised families who
received at least four home visits and participated in the pro-gram
for at least six months; members of the comparison group
left the program before receiving four home visits. The analysis
included 1,139 program participants and 512 families in the
comparison group.
No significant difference between Healthy Families and com-parison
group—The analysis revealed that Healthy Families
participants and comparison group members had similar rates of
CPS reports. For substantiated reports—those in which DES
concluded that abuse occurred—the percentage of families who
had a report on file was 1.7 for program participants, and 1.8
percent for the comparison group. The slight difference between
the two is not statistically significant. For unsubstantiated re-ports,
program participants had a slightly higher rate (6.1 percent
versus 5.5 percent for the comparison group), but again the dif-ference
was not statistically significant.
As a further point of comparison, evaluators also analyzed the
percentage of program participants and comparison group
members who were referred to Family Builders. Family Builders
is a program that provides services to families who have been
reported to CPS for low and potential abuse risk. Once referred
to Family Builders, the report is closed and CPS does not conduct
an investigation of the case.1 Results show that 1.8 percent of
program participants and 2.7 percent of the comparison group
1 To be eligible for Family Builders, the family cannot have an existing open
file with CPS. Additionally, the case cannot involve allegations of sexual
abuse, or current injuries, or involve a child who is currently in out-of-home
placement or a ward of the State.
Participant abuse rates do not
differ from comparison group.
Finding I
15
OFFICE OF THE AUDITOR GENERAL
were referred to Family Builders. However, this difference is also
not statistically significant. Results are summarized in Table 2.
Table 2
Healthy Families Program
Percentage of Participants and Comparison Group Members
with Substantiated and Unsubstantiated Reports of
Abuse and Referrals to Family Builders
Reports Received Between January 1, 1998 and May 31, 2000
Outcome Measure
Program
Participants
Comparison
Group
Substantiated reports of abuse 1 1.7% 1.8%
Unsubstantiated reports of abuse 1 6.1 5.5
Referred to Family Builders 1 1.8 2.7
1 Differences are not statistically significant.
Source: Auditor General staff analysis of data provided by Child Protective Services and the
Healthy Families Program.
Many reports received one year after intake—The majority of
program participants and comparison group families with sub-stantiated
reports of abuse received these reports at least one
year after their initial intake into the Healthy Families Program.
Over half the reports were received two years after intake.1
Moreover, the results of this analysis also show that 79 percent of
Healthy Families participants who have a substantiated report in
the CHILDS database received the report after they left the pro-gram.
Comparison with prior Auditor General reports—The rates
found in this report are consistent with those found in the Audi-tor
General’s last evaluation (see Report No. 00-1). However, an
Auditor General evaluation issued in January 1998 found higher
1 For this analysis, evaluators included all substantiated reports that oc-curred
at any point after intake—that is, they did not limit the analysis to
reports that occurred after six months of exposure to the program.
Finding I
16
OFFICE OF THE AUDITOR GENERAL
abuse rates (see Report No. 98-1). Specifically, 3.3 percent of pro-gram
participants and 8.5 percent of comparison group members
received substantiated reports of abuse (see Table 3). These find-ings
show a decrease in rates over the last two years.
Possible Explanations
of Results
There are several possible explanations for the evaluation results.
First, it is possible that the effects of the program on preventing
child abuse cannot be adequately measured by the current
evaluation design because of various factors associated with
changes in abuse reporting requirements. Second, it is possible
the program’s impact is lessened because most program services
occur before the time period when the risk of child abuse in-creases.
Third, the program may not have an effect on prevent-ing
child abuse.
Explanation 1: Program reduces child abuse risk but vari-ous
factors might have affected the results—Compared to
the 1998 evaluation, Healthy Families participants and compari-son
group members were found to have lower rates of abuse in
Table 3
Healthy Families Program
Percentage of Participants and Comparison Group Members
with Substantiated Abuse Reports
Office of the Auditor General Evaluations
Issue Date
Healthy Families
Participants
Comparison
Group
January 1998 a 3.3% 8.5%
February 2000 b 1.6 1.4
February 2001 b 1.7 1.8
a Difference between Healthy Families participants and comparison group statistically
significant at the .001 level. That is, the probability that the difference in rates occurred by
chance is less than 1 in 1,000.
b Difference between Healthy Families participants and comparison group not statistically
significant.
Source: Auditor General staff analysis of data provided by Child Protective Services and
the Healthy Families Program.
There are several possible
explanations for evaluation
results.
Finding I
17
OFFICE OF THE AUDITOR GENERAL
the current and last evaluations. The reduction in abuse rates for
both groups may reflect changes in CPS child abuse substantia-tion
standards, changes in the construction of the comparison
group, and changes in program enrollment that occurred after
the 1998 evaluation. These changes may have affected the
evaluation design, limiting the ability of evaluators to assess the
program’s impact on child abuse. Specifically,
n Overall there is a lower occurrence of substantiated
abuse, due in part to the appeals process—Effective
January 1, 1998, DES changed its procedures for substantiat-ing
reports of abuse and neglect. Under A.R.S. §8-811, the
Department created the Protective Service Review Team
(PSRT). The PSRT reviews reports of abuse and/or neglect to
determine if report allegations should be substantiated in the
Department’s Central Registry.1 Since the implementation of
the PSRT, substantiation rates have decreased. According to a
DES report, 25 percent of CPS cases were substantiated in fis-cal
year 1999, the first full fiscal year with the appeals process
in place.2 In comparison, 45 percent of cases were substanti-ated
in fiscal year 1997, the last fiscal year without the ap-peals
process.
n Comparison group different—The current comparison
group comprised families who left the program before
receiving four home visits. This comparison group differs
from the group used in the 1998 evaluation, which was con-structed
using individuals eligible for Healthy Families but
who could not enroll because the program was full. Al-though
current comparison group families did not receive
four home visits, many received one to three visits. During
these initial visits, Healthy Families staff explain the pro-gram’s
mandatory CPS reporting policy and provide infor-mation
about child safety (for example, the importance of
never shaking a baby, or leaving a baby unattended). Within
1 PSRT reviews do not include those cases in which criminal or civil charges
have been filed against the alleged abuser.
2 Department of Economic Security. Child Welfare Reporting Requirement:
Annual Reports for the Period July 1, 1996 through June 30, 1997, and the Period
of July 1, 1998 through June 30, 1999.
Finding I
18
OFFICE OF THE AUDITOR GENERAL
the first three visits these families can also receive various re-ferrals
for family assistance.
In addition, in the current analysis, there are some small but
statistically significant differences between participants and
the comparison group. Participants are slightly older, less
likely to be single, and less likely to be employed. Participant
scores on the Family Stress Checklist Risk Assessment are
also higher by an average of one point. No significant differ-ences
were found when comparing annual household in-come,
education level, or ethnicity.
n Families with abuse history excluded from the pro-gram—
Effective June 1, 1998, the Healthy Families statute
(A.R.S. §8-701) was changed, prohibiting the enrollment of
families with prior substantiated CPS abuse reports. Families
who received a substantiated report, but were already en-rolled
in the program, were also required to exit.1 Thus, fami-lies
with a demonstrated propensity for abuse are no longer
enrolled in the program.
Explanation 2: Program impact on abuse may be affected by
timing and length of participation—Evaluators’ preliminary
analysis of report dates for substantiated abuse reveals that the
majority of program participants and comparison group mem-bers
who receive reports do so one year or more after the fam-ily’s
initial exposure to the program. In most cases, the abuse
reports for program participants also occurred after the family
left the program as a majority of families participate for 12 or
fewer months. Therefore, it is possible that the risk of child abuse
may increase as the child gets older, but that participants have
already left the program before this increase occurs.
1 Comparison of initial assessment scores for parents who entered the pro-gram
before June 1, 1998, with scores of those parents who entered after
June 1, 1998, suggest current enrollees are less at-risk. Specifically, Family
Stress Checklist (FSC) scores are lower for participants who entered the
program after the enrollment criteria changed. The average FSC score is
38.6 for pre-June 1, 1998, enrollees and 36.4 for post-June 1, 1998, enrollees.
This difference is significant at the .001 level. Scores of 25 or higher indicate
a potential risk to commit abuse.
Finding I
19
OFFICE OF THE AUDITOR GENERAL
Explanation 3: Groups do not differ because program might
not reduce child abuse risk—Because last year’s and this
year’s evaluations show no difference in program participant
and comparison group abuse rates, it could be concluded that
the program does not reduce child abuse risk. However, before
making this determination, evaluators would prefer to ad-dress
the other possible explanations. This would involve
constructing additional comparison groups of “at-risk” fami-lies
who have not received any child abuse prevention ser-vices.
Additionally, because abuse often occurs a year or
more after program enrollment, other measures of the pro-gram's
long-term effect will have to be developed. However,
establishing new comparison groups and developing addi-tional
long-term measures would require extensive efforts
and the results of the further analyses would likely not be
available for several years.
20
OFFICE OF THE AUDITOR GENERAL
(This Page Intentionally Left Blank)
21
OFFICE OF THE AUDITOR GENERAL
FINDING II HEALTHY FAMILIES PROGRAM
PROVIDES HEALTH REFERRALS,
ENCOURAGES ON-TIME
IMMUNIZATIONS, AND
PROMOTES SAFETY
Although the Healthy Families Program’s long-term effect on
preventing child abuse cannot be clearly determined, it has been
successful in providing health referrals, encouraging on-time
immunizations, and promoting safety. More specifically, on
average, the majority of children in the program are fully immu-nized
and most children have a primary health care provider.
Additionally, most children receive regular developmental as-sessments.
Parents also improved their compliance with home
safety measures over time.
Background
According to program goals defined in the Healthy Families
statute (A.R.S. §8-701), the program is required to provide par-ents
with information about child development and preventative
health care. Program goals also include reducing participant
substance abuse. To determine if the program is achieving these
goals, evaluators examined several health, safety, and develop-ment
measures. The measurement tools used to collect this in-formation
are described in the Appendix (see pages a-i through
a-ii).
Although program goals include reducing participant substance
abuse, evaluators reported in the February 2000 evaluation (see
Auditor General Report No. 00-1) that the program’s impact on
reducing participants’ dependency on drugs and alcohol could
not be determined. Specifically, evaluators reported in the
Finding II
22
OFFICE OF THE AUDITOR GENERAL
February 2000 evaluation that the CAGE1 substance abuse screen
used by the program was not an adequate measure for assessing
change in drug and alcohol usage and recommended replacing
this instrument. To address this recommendation, the Healthy
Families Program began using a new substance abuse question-naire
in July 2000 (see Other Pertinent Information, pages 31
through 34). However, data from the questionnaire will not be
available for analysis until the 2001 evaluation.
Participants Receive
Developmental Screenings,
Referrals, and On-Time
Immunizations
Evaluators found that consistent with the January 1998 and Feb-ruary
2000 evaluations, the majority of children in the Healthy
Families Program received age-appropriate immunizations and
developmental screenings. Most participants also had a primary
health care provider. Results of the developmental assessments
show that most children in the program are developing nor-mally.
However, for those children identified as potentially hav-ing
delays, the program provides referrals to appropriate ser-vices
and recently began tracking those referrals in its new fam-ily
update form (see Other Pertinent Information on page 31).
Healthy Families monitors immunization rates and promotes
preventative medical care—Eighty-five percent of two- to three-month-
old children enrolled in the program are fully immu-nized.
At four months of age, 73 percent of children in the pro-gram
are fully immunized; at 6 months of age, 66 percent are
fully immunized; and at 1 year to 15 months of age, 73 percent
are fully immunized. Generally, these rates are higher than esti-mated
vaccination rates for children served by local public health
clinics in the same areas as program sites. Additionally, these
1 The acronym “CAGE” is derived from the first letter in each of four ques-tions
that are asked of the respondent: (1) Have you every felt the need to
cut down on drinking/drug use? (2) Have you ever felt annoyed by oth-ers’
criticism of your drinking/drug use? (3) Have you ever felt guilty
about your drinking/drug use? (4) Have you ever had a drink/taken
drugs first thing in the morning (eye-opener)? Two or more “yes” re-sponses
are considered indicative of a substance abuse problem.
Finding II
23
OFFICE OF THE AUDITOR GENERAL
rates are comparable to statewide rates for Arizona reported by
the Centers for Disease Control and Prevention.
In addition to encouraging parents to ensure that their children
are fully immunized, the program also emphasizes the impor-tance
of preventative and routine medical care. At 2 months in
the program, 97 percent of children had a medical primary care
provider; and at 6 and 12 months, 98 percent of children had a
medical provider.
To promote positive health outcomes, home visitors ask about
immunization schedules for the children they serve, provide
educational information about immunizations, and refer families
to health/immunization clinics. In addition, program staff em-phasize
the importance of routine and preventative medical care
and encourage participants to use referrals to obtain a primary
care provider.
Healthy Families program conducts developmental assess-ments—
Home visitors use the Ages and Stages Questionnaire
(ASQ) to conduct regular assessments of the physical and social
development of children enrolled in the program. The ASQ is
administered at the following ages: 4, 6, 12, 18, 24, 30, 36, and 48
months. In addition to conducting these assessments, home visi-tors
provide families with information about the developmental
stages to alert parents to signs of potential delays and reduce
fears associated with unrealistic expectations. Program partici-pants
cited developmental assessments as important for identify-ing
milestones, early detection of potential delays, and reducing
confusion about their child’s development.
Results of the ASQ reveal that most children in the program are
developing at levels that are appropriate for their age (see Table
4, page 24). The ASQ assesses five developmental areas—
communication, gross motor skill, fine motor skill, problem solv-ing,
and personal social development. If a child receives a score
below the minimum level for normal development, home visi-tors
are trained to explain the result to the parent and provide
referrals to a primary health care provider, or the Arizona Early
Intervention Program (AzEIP), to arrange for further assessment.
However, as with the previous Auditor General report (see Re-port
No. 00-1), evaluators were unable to directly link develop-
Program conducts develop-mental
assessments of partici-pants’
children.
Finding II
24
OFFICE OF THE AUDITOR GENERAL
mental screenings with referrals to AzEIP or a health care pro-vider.
The program has revised its family update form. The up-date
form now contains questions which enable the program to
document if children with developmental delays are offered the
appropriate referrals. This change is discussed in the Other Per-tinent
Information section of this report (see pages 31 through
34).
Table 4
Healthy Families Program
Ages and Stages Questionnaire
Percentage of Children Developing at Age-Appropriate Levels
January 1994 to May 2000
Age in Months
Type of
Development 4 6 12 18 24 30 36 48
Communication 98.4% 99.8% 98.6% 89.8% 89.4% 91.8% 92.4% 96.7%
Gross motor skills 91.7 98.5 97.5 99.2 95.6 95.9 98.2 98.9
Fine motor skills 96.5 99.0 98.8 98.9 96.0 91.2 95.5 98.9
Problem solving 96.7 99.1 96.0 98.6 93.8 93.9 89.2 81.9
Personal/Social 96.5 98.6 97.0 99.9 93.3 94.7 94.6 100.0
Number of children1 1,884 1,849 1,183 741 499 342 224 94
1 The number of children varies by scale; the number reported is the largest number assessed for that age group.
Source: Auditor General staff analysis of data provided by the Healthy Families Program.
Participants Show Significant
Improvements in Home Safety
The safety of program participants’ homes improves over the
course of their enrollment in the program. Healthy Families staff
administer the Child Safety Checklist (CSC) to identify potential
safety hazards and also discuss with parents ways to improve
home safety. On average, program participants improved their
home safety after one year by taking steps to comply with items
on the CSC.
Finding II
25
OFFICE OF THE AUDITOR GENERAL
Healthy Families participants maintain homes that meet child
safety requirements—Evaluators analyzed results from the CSC
at 2, 6, and 12 months and found that home safety improves after
a year of enrollment. On average, the percentage of child safety
measures implemented improves from 87 percent to 94 percent
between 2 to 12 months (see Table 5). Analyses of individual
items on the checklist show significant improvement in key
safety measures such as covering outlets and keeping poisons
and small items that are potential choking hazards out of reach
(see Appendix, page a-ii, for a description of the checklist).
Nearly all of the program participants indicated initial (2
months) and subsequent (6 and 12 months) compliance with
other important safety measures such as using car seats, making
sure their child is never alone in the house or car, and keeping
dangerous objects such as scissors and matches out of their
child’s reach.
Table 5
Healthy Families Program
Percentage of Safety Measures Implemented
by Age of Child
July 1998 to May 2000
Age of Child
Percentage
Implemented 1
2 months 87%
6 months 91
12 months 94
1 Increases in percentages of safety measures implemented are all statisti-cally
significant at the .001 level. That is, the probability that the average
gain in compliance occurred by chance is less than 1 in 1,000.
Source: Auditor General staff analysis of data provided by the Healthy Fami-lies
Program.
Participants improve home
safety.
Finding II
26
OFFICE OF THE AUDITOR GENERAL
Recommendations
1. To ensure compliance with its statutorily defined goal to
reduce substance abuse, the Healthy Families Program
should continue to implement the new questionnaire for as-sessing
participant substance abuse problems and method
for tracking substance abuse referrals.
2. To ensure that families with children who receive scores in
the delayed range on the ASQ receive appropriate referrals,
the Healthy Families Program should continue to implement
new measures for tracking these referrals.
27
OFFICE OF THE AUDITOR GENERAL
FINDING III PARENTING STRESS MEASURE
SHOWS IMPROVED FAMILY
RELATIONS
Results from the Parenting Stress Index show that parental stress
decreases after 12 months of participation. Evaluators also found
that parents with higher scores on the restricted role, isolation,
and depression subscales were more likely to receive referrals to
mental health services compared with those whose scores indi-cated
lower risk. To assist families in improving parent-child
interaction, home visitors provide emotional support and infor-mation
to help parents understand their child’s behavior.
Background
To address recommendations in the last Auditor General evalua-tion,
the Healthy Families program discontinued using the
Home Observation for Measurement of Environment (HOME)
to assess family functioning (see “Other Pertinent Information,”
pages 31 through 34). In place of the HOME, the program is
using the Parenting Stress Index (PSI) to measure family rela-tions.
The PSI is first administered to program participants after
3 weeks of enrollment and then again at 6 and 12 months. Cur-rently,
there is no comparison group for this measure.
Participants’ Stress
Reduced After
12 Months
Although scores for most Healthy Families participants are
within normal ranges, after 12 months in the program partici-pants
show statistically significant decreases in parental stress on
6 of the PSI’s subscales. As shown in Table 6, stress related to
attachment, role restriction, competence, social isolation, depres-sion,
and mood is lower at 12 months than it is at 3 weeks.
Parents have reduced stress
after 12 months in the pro-gram.
Finding III
28
OFFICE OF THE AUDITOR GENERAL
Research has shown that high scores on parenting stress are
related to the potential to abuse one’s child. Conversely, low to
normal scores are related to positive parent-child relationships.
Thus, reductions in parental stress suggest that program services
may improve family relations.
Table 6
Healthy Families Program
Average Parenting Stress Scores
After 3 Weeks and 12 Months in the Program
July 1998 to May 2000
Participant Scores
Scale1
Normal Stress
Range 3 Weeks 12 Months
Competence2 23-34 31.9 29.9
Attachment2 10-15 13.1 12.0
Role Restriction2 14-23 20.0 18.5
Depression 2 16-24 20.2 19.1
Isolation 2 10-16 14.1 13.2
Distractibility 3 20-28 25.3 25.0
Mood 2 7-11 10.9 9.4
1 Higher numbers indicate higher stress levels. See Other Pertinent Information (pages 31
through 34) for detailed information on these scales.
2 Differences from 3 weeks to 12 months are statistically significant at the .05 level or bet-ter.
That is, the chance that the average reduction in parenting stress occurred by chance
is less than 5 in 100. Evaluators also analyzed participant PSI scores at 6 months. For the
competence, role restriction, and mood subscales evaluators found significant differences
between scores at 3 weeks and 6 months. Scores for the depression and attachment sub-scales
at 6 months were significantly different from scores at 12 months.
3 Differences not statistically significant.
Source: Auditor General staff analysis of data provided by the Healthy Families
Program.
Finding III
29
OFFICE OF THE AUDITOR GENERAL
Parents With Higher
Stress More Likely
to Receive Mental
Health Referrals
Additionally, participants who scored in the higher risk ranges
for depression, isolation, and role restriction were more likely
than families scoring at lower risk levels to receive referrals for
mental health services. This shows that the PSI may also be a
useful tool for identifying families who need additional services.
Home Visitors Provide
Information on
Parent-Child Bonding
To assist families with parent-child interaction, home visitors
provide emotional support and information to help parents un-derstand
their child’s behavior. Information provided can in-clude
methods for alleviating stress, activities to improve parent-child
interaction and bonding, and referrals to counseling ser-vices
to assist with family relationship problems.
30
OFFICE OF THE AUDITOR GENERAL
(This Page Intentionally Left Blank)
31
OFFICE OF THE AUDITOR GENERAL
OTHER PERTINENT INFORMATION
The previous report issued by the Office of the Auditor General
(see Report No. 00-1) offered several recommendations to im-prove
the measurement of the Healthy Families Program’s effect
on family functioning and reduction of substance abuse. It is
necessary for the program to continue collecting data on the
program’s effect on family relations and reducing dependency
on drugs and alcohol because these are defined in statute (A.R.S.
§8-701) as two of the program’s five goals. The previous evalua-tion
report also recommends tracking referrals for children with
developmental delays. To address these recommendations, the
program has implemented several changes.
Healthy Families stopped using the HOME—During last year’s
evaluation, evaluators were not able to make definitive conclu-sions
about the program’s impact on family functioning because
the analysis was limited by reliability problems with the Home
Observation for Measurement of the Environment (HOME)
assessment tool. Specifically, statistical analyses of variety, accep-tance,
and organization—three of six areas assessed by the
HOME—revealed that questions within each area were not
strongly related to each other. This suggests that some items on
the HOME do not measure the concepts they were intended to
assess.
Following the Auditor General’s recommendation, the program
stopped using the HOME. However, because improving family
relations and family unity are defined in statute as program
goals, it is necessary for the program to collect information re-lated
to these goals. Therefore, in place of the HOME, the pro-gram
will use the Parenting Stress Index (PSI) to assess the pro-gram’s
impact on family relations. Past research has shown that
low scores on the PSI are related to positive parent-child relation-ships.
In particular, the PSI, designed to measure “stressful par-ent-
child systems,” can identify areas in which improvements
are needed in parent-child relations. Below are descriptions of
what high shores on each subscale indicate, according to
Program stopped using the
HOME assessment.
Other Pertinent Information
32
OFFICE OF THE AUDITOR GENERAL
Richard Abidin, PSI’s creator1. The first five scales measuring
competence, isolation, attachment, restricted role, and depression
focus on parent characteristics. The last two scales, measure child
distractibility and child mood, focus on child characteristics.
n Competence: High scores on the competence scale can be
the result of the lack of child development knowledge, lim-ited
child management skills, or a feeling that parenting is
less reinforcing than expected. High scores may also be re-lated
to “lack of acceptance and criticism from the child’s
other parent.”
n Isolation: Parents who score high on the social isolation
scale tend to be “isolated from their peers, relatives, and other
emotional support systems.” Additionally, “in many in-stances,
their relationships with their spouses are distant and
lacking in support for their efforts as parents.”
n Attachment: High scores on attachment may indicate that
parents do not feel closeness with their child or feel unable to
understand the child’s feelings and needs.
n Restricted Role: High scores indicate that the parent sees
the child as restricting their freedom and may feel controlled
by their child’s demands or needs. As a result, resentment
and anger may build toward one’s child and/or spouse.
n Depression: High scores on the depression scale can be
indicative of clinical depression. Depressed parents may lack
the energy needed to fulfill parental responsibilities.
n Child Distractibility: High scores on this subscale indicate
that either the child has Attention Deficit Disorder with hy-peractivity
or that the child is normal, but that the parent
lacks the energy to keep up with the child.
n Child Mood: High scores on child mood are associated with
children who are unhappy and depressed. When scores are
extremely high, there may be impairment in parental at-
1 Richard R. Abidin. Parenting Stress Index: Professional Manual. Odessa, FL:
Psychological Assessment Resources, Inc., 1995.
Other Pertinent Information
33
OFFICE OF THE AUDITOR GENERAL
tachment or parents may be absent or unavailable due to
drug abuse or alcoholism.
Healthy Families replaced the CAGE—Evaluators noted in the
previous report that, lacking a better instrument, Healthy Fami-lies
has used the CAGE substance abuse screening measure since
July 1, 1998. However, the CAGE, which contains questions for
identifying individuals with an alcohol and/or drug abuse prob-lem,
does not allow the program to track whether a participant’s
problem with substance abuse changes over time. To address the
Auditor General recommendation to replace the CAGE, the pro-gram
began using a new questionnaire on July 1, 2000. The new
questionnaire measures the program’s effect on reducing de-pendency
on alcohol and drugs over time. In addition, the pro-gram
has also added questions to its family update form, which
will enable staff to track referrals for substance abuse treatment.
The new substance abuse assessment was adapted from the
Health Screening Survey (Fleming and Barry, 1991) and includes
items that measure current substance usage. For instance, par-ticipants
are asked to indicate, over the past three months, the
number of days per week they drank alcohol, and the number of
drinks they typically have had during a single occasion. Unlike
the CAGE, which only captures lifetime usage, the new assess-ment
allows evaluators to assess changes in drinking patterns
over time.
The program will track referrals for children with delays—The
previous Auditor General report noted that when children re-ceive
scores in the delayed range on the Ages and Stages Ques-tionnaire
(ASQ) home visitors are trained to provide a referral to
a medical provider or the Arizona Early Intervention Program
(AzEIP). However, because most families in the program receive
medical or social services, evaluators were unable to determine
the extent to which families receive services directly related to a
developmental delay identified by the ASQ. To address the pre-vious
report recommendation that the program track referrals
for children identified as having a potential developmental de-lay,
Healthy Families has revised its family update form. The
form now contains questions, which supplement the ASQ,
Program replaced substance
abuse questionnaire.
Program will track referrals
for children with developmen-tal
delays.
Other Pertinent Information
34
OFFICE OF THE AUDITOR GENERAL
and will be used to track whether children with developmental
delays are offered appropriate referrals. The new form was im-plemented
on July 1, 2000, and data from this measurement will
be analyzed in the next evaluation.
35
OFFICE OF THE AUDITOR GENERAL
STATUTORY ANNUAL EVALUATION COMPONENTS
Pursuant to A.R.S. §41-1279.08, the Office of the Auditor General
is required to include the following information in the annual
program evaluation.
C.1. Information on the number and characteristics of
program participants.
n Number of Participants—Since 1994, 4,421 families
have participated in Healthy Families. Of these, 3,267
met program criteria for being officially engaged in
the program. That is, they received at least four home
visits from a home visitor. The remaining 1,154 re-ceived
three or fewer home visits and were included
as a comparison group when examining child abuse
rates.
n Mother’s Age, Marital Status, Income, Ethnicity,
Education, Employment Status, and Risk Score—
Information about the age, marital status, income,
ethnicity, employment status, and educational status
of participants is shown in Table 7 (see page 36). Be-cause
in nearly all cases, the child’s mother is the pri-mary
program participant, information is provided
for mothers only and is presented separately for par-ticipant
families and for comparison group families.
Statistical analyses showed that these two groups
were similar in most areas, with a few exceptions.
Comparison group members were more likely than
participants to be single and employed. On average,
participants were also slightly older (average age of
22.9) than the comparison group (average age of 22.3).
In addition, scores on the Family Stress Checklist risk
assessment tool were slightly higher for participant
families (average score of 38) than for comparison
group families (average score of 37), and the differ-ence
was statistically significant. No significant differ-ences
were found when comparing annual household
income, education level, or ethnicity.
Statutory Annual Evaluation Components
36
OFFICE OF THE AUDITOR GENERAL
Table 7
Healthy Families Program
Participant and Comparison Group Member Profile
January 1994 through May 2000
Program
Participants
N (total number of
participants) =3,267
Comparison
Group
N (total number of comparison
group members)=1,154
Mother’s age
11-15 6.0% 5.7%
16-19 34.6 37.8
20-25 34.2 36.1
26-30 13.1 11.4
31 and older 12.1 9.1
Marital status
Single 68.2 76.7
Married 20.3 13.4
Divorced 3.1 3.3
Separated 3.5 3.6
Widowed .2 .3
Other/Unknown 4.6 2.8
Annual income
$10,000 and under 61.9 62.5
$10,001-$15,000 20.4 19.2
$15,001-$20,000 9.0 8.7
$20,001-$30,000 5.8 7.9
$30,001 and higher 2.9 1.7
Mother’s ethnicity
Hispanic 54.3 54.8
Caucasian 23.8 24.0
African-American 7.9 8.7
Native American 10.0 8.4
Asian-American .5 .2
Other 3.5 4.0
Percentage employed at intake 13.2 18.8
Full-time 6 2 . 9 6 9 . 1
Part-time 37.1 30.9
Average years of education 10.3 10.4
Average risk score on Family Stress Checklist
(Range 5 to 85) 37.9 37.0
Source: Auditor General staff analysis of data provided by the Healthy Families Program.
n Regions Served—The Healthy Families Program
serves families in 10 of Arizona’s 15 counties. The
number of participants served from each county is
shown in Figure 2 (see page 37).
Statutory Annual Evaluation Components
37
OFFICE OF THE AUDITOR GENERAL
Figure 2
Healthy Families Program
Number of Participants and Comparison Group Members by County
January 1994 through May 2000
1,013
371
753
243
444
132
284
96
235
107
213
85
147
51
126
21
52 48
0
200
400
600
800
1,000
1,200
Number of Families
Participants Comparison Group Members
Source: Auditor General staff analysis of data provided by the Healthy Families Program.
Maricopa Pima Cochise Coconino Yavapai Santa Cruz Yuma La Paz/ Pinal
Mohave
County
n Number of Children, Household Size, Sex of
Child, Health Insurance, and Public Assistance
Usage—Table 8 (see page 38) presents information on
program participants’ number of children, household
size, and the proportion of female and male children
served by the program. It also provides information
on participants’ self-reported health insurance status
and use of public assistance. Statistical analyses re-vealed
that, on average, comparison group members
had a slightly larger household size (average of 4.89)
than participants (average of 4.65). No significant dif-ferences
were found for the number of children in
participant versus comparison group families or for
the proportion of male versus female children served
in each group. In addition, no group differences were
found for health insurance status or for the proportion
Statutory Annual Evaluation Components
38
OFFICE OF THE AUDITOR GENERAL
of program participants using Temporary Assis-tance
for Needy Families (TANF), food stamps, or
Women, Infants and Children (WIC).
Table 8
Healthy Families Program
Participants and Comparison Group Members
Profile of Family Characteristics
January 1994 through May 2000
Program
Participants
Comparison
Group
N (total number of partici-pants)
= 3,267
N (total number of comparison
group members) = 1,154
Number of children
1 54.0% 55.2%
2-3 34.3 35.6
4-5 9.5 7.3
6 or more 2.1 1.8
Household size
2-3 33.0 30.6
4-5 39.2 37.4
6-7 19.4 21.1
8-9 6.5 7.2
10 or more 1.8 3.6
Sex of child
Female 49.6 50.2
Male 50.4 49.8
Health insurance
AHCCCS1 80.3 80.1
Private 9.3 9.1
Other 3.3 2.9
None 6.1 6.1
Unknown 1.0 1.2
Public assistance services used as
reported at intake
TANF2 24.7 23.3
Food Stamps 35.6 35.4
WIC3 80.1 78.9
1 Arizona Health Care Cost Containment System.
2 Temporary Assistance for Needy Families.
3 Women, Infants and Children Nutrition Program.
Source: Auditor General staff analysis of data provided by the Healthy Families Program.
Statutory Annual Evaluation Components
39
OFFICE OF THE AUDITOR GENERAL
C.2. Information on contractors and program service
providers.
See Introduction and Background (page 6) for informa-tion
on contractors and program service providers.
C.3. Information on program revenues and expenditures.
See Table 1 (page 7) for information on program revenues
and expenditures.
C.4. Information on the number and characteristics of
enrollment and disenrollment and information from
program participants on the reasons for each.
As of May 31, 2000, there were 1,178 families enrolled in
Healthy Families. Since 1994, 4,421 families have enrolled.
Table 9 (see page 40) provides information about the
length of participation for families who have exited the
program. These families participated for a median of 195
days (or 6 and a-half months). Among families who were
active participants as of May 31, 2000, the median dura-tion
of participation was slightly more than 1 year (379
days).
In addition, the January 1998 Healthy Families evaluation
reported an attrition rate of 45 percent for fiscal year 1997.
An attrition rate for a given fiscal year is computed by di-viding
the number of families who exited the program
during that year by the number who participated in the
program at any point during the year. The corresponding
attrition rates for fiscal years 1998, 1999, and 2000 were 42
percent, 40 percent, and 35 percent, showing a gradual
improvement in retention. 1
Figure 3 (see page 40) shows the most common reasons
participants exit the program. More than half terminate
because they have moved or because the program could
not contact the family after several attempts. Other rea-sons
for termination include participants’ refusal of fur-
1 Thirty-five percent attrition rate for 2000 includes 11 months of data from
fiscal year 2000 (July 1, 1999 through May 31, 2000).
Statutory Annual Evaluation Components
40
OFFICE OF THE AUDITOR GENERAL
ther services, refusal of a change in home visitor, and
achievement of self-sufficiency.
Table 9
Healthy Families Program
Months of Participation for Families at Exit
January 1994 through May 2000
Number Percentage
1-6 months 1,520 46.9%
7-12 months 773 23.9
13-18 months 382 11.8
19-24 months 197 6.1
25-30 months 128 3.9
31-36 months 90 2.8
37-48 months 97 3.0
49-60 months 53 1.6
Total 3,240
Source: Auditor General staff analysis of data provided by the Healthy Families Program.
Figure 3
Healthy Families Program
Reasons for Exit
January 1994 through May 2000
n (total number of families who left the program) = 3,212
Source: Auditor General staff analysis of data provided by Healthy Families staff.
Refused home visitor
change 4%
Self-sufficiency
9%
Unable to
contact 31%
Refused services
14%
Family moved
28%
Other
14%
Statutory Annual Evaluation Components
41
OFFICE OF THE AUDITOR GENERAL
C.5. Information on the average cost for each participant
in the program.
See Table 1 (page 7) for information on the average cost
for each program participant.
C.6. Information concerning the progress of program
participants in achieving goals and objectives.
See Finding I (pages 13 through 19) for information on the
program’s progress in reducing rates of child abuse and
neglect.
See Finding II (pages 21 through 26) for information re-garding
the program’s progress in improving children’s
health and participants’ home environment. Finding II
includes information about immunization rates, devel-opmental
screenings, child safety, and parent-child inter-action.
See Section D (pages 42 through 43) for information on
participants’ progress in increasing self-sufficiency
through employment, and reducing their dependence on
public assistance.
C.7. Information on any long-term savings associated
with program services.
As discussed in Finding I (see pages 13 through 19),
evaluators did not find statistically significant differences
in the rates of substantiated child abuse reports for
Healthy Families participants and the comparison group.
Thus, evaluators cannot estimate potential savings asso-ciated
with reduction in child abuse rates, examining
such factors as costs of CPS investigations, foster care
placements, and in-home services.
C.8. Recommendations regarding program administra-tion.
1. To ensure compliance with its statutorily defined goal
to reduce substance abuse, the Healthy Families Pro-gram
should continue to implement new measures
Statutory Annual Evaluation Components
42
OFFICE OF THE AUDITOR GENERAL
for assessing participant substance abuse problems
and tracking substance abuse referrals.
2. To ensure that families with children who receive
scores in the delayed range on the ASQ receive ap-propriate
referrals, the Healthy Families Program
should continue to implement new measures for
tracking these referrals.
C.9. Recommendations regarding informational materials
distributed through the program.
Current and past evaluations of Healthy Families infor-mational
materials show that materials distributed by the
program are adequate to address program needs. Specifi-cally,
Healthy Families provides program participants
with information regarding medical care, mental health,
employment, and education. Additionally, the program
provides information to promote parent-child interaction
and to assist parents with understanding their child’s de-velopment.
D. Effect of the program on encouraging parental re-sponsibility
in employment, self-sufficiency, and child
safety. Document the income level and family size of
those receiving program services.
n Employment rates—As seen in Figure 4 (see page
44), employment rates for both mothers and fathers
enrolled in Healthy Families generally increased over
time. Healthy Families can influence these rates by
providing referrals to job services. According to pro-gram
data, 6.3 percent of mothers and 10.5 percent of
fathers received a referral to job services within two
months of enrollment, 10.2 percent of mothers and
12.8 percent of fathers received such a referral at 6
months, and 12.2 percent of mothers and 12.7 percent
of fathers received a job-related referral at 12 months.
Although employment rates show an increase, it is
important to note that due to attrition, at each time
period, there are fewer participants represented.
Therefore, it is difficult to determine the extent to
Statutory Annual Evaluation Components
43
OFFICE OF THE AUDITOR GENERAL
which increases in employment rates are due to pro-gram
participation or due to differences among peo-ple
who remain in the program compared to those
who drop out.
n Public assistance—In order to conduct an analysis
of public assistance usage, evaluators requested, data
from DES,containing all records of Temporary Assis-tance
to Needy Families (TANF) and food stamps re-ceived
by Healthy Families participants and compari-son
group members. However, after receiving the
data, evaluators learned that information about sev-eral
of the participants was missing. Therefore, it was
determined that any analysis of public assistance us-age
would provide inaccurate results. In addition, it
was not possible to make additional data requests due
to the substantial amount of time it would have taken
to make the request, receive the data, test it for reli-ability,
and analyze the results.
n Child safety—See Finding II (pages 21 through 26)
for information on child safety, including the results
of the Child Safety Checklist assessment.
n Income level and family size of those receiving
program services—See C.1. (pages 35 through 38) of
this section for income levels and family size of those
receiving program services.
Statutory Annual Evaluation Components
44
OFFICE OF THE AUDITOR GENERAL
Figure 4
Healthy Families Program
Employment Rates of Mothers and Fathers in Program
January 1994 through May 2000
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Months in Healthy Families Program
Mothers Fathers
Source: Auditor General staff analysis of data provided by the Healthy Families Program.
Intake 2 6 12 18 24 30 36 42 48 54 60
Percentage Employed
OFFICE OF THE AUDITOR GENERAL
Appendix
OFFICE OF THE AUDITOR GENERAL
(This Page Intentionally Left Blank)
a-i
OFFICE OF THE AUDITOR GENERAL
Appendix
Assessment Tools
n Ages and Stages Questionnaire (ASQ)—The ASQ is a
developmental screening tool that is completed by the parent
and is used to assess whether children are developing nor-mally,
both physically and socially. The questionnaire ad-dresses
five areas of child development: (1) Communication,
(2) Gross Motor Skills, (3) Fine Motor Skills, (4) Problem Solv-ing,
and (5) Personal Social Skills. For each area, parents are
asked to respond to six questions about whether their chil-dren
are engaging in behavior appropriate for their age. The
ASQ is administered at the following ages: 4, 6, 12, 18, 24, 30,
36, and 48 months.
Reference: Bricker, Diane, Jane Squires, Linda Mounts, La-
Wanda Potter, Bob Nickel, and Jane Farrell. The Ages and
Stages Questionnaire: A Parent-Completed, Child-Monitoring Sys-tem.
Baltimore: Paul H. Brookes Publishing Co., 1995.
n The CAGE Questionnaire—Substance Abuse Screen-ing—
The CAGE Questionnaire was designed to identify po-tential
alcohol abuse problems. It was also modified to in-clude
the abuse of drugs other than alcohol. The acronym
“CAGE” stands for the first letter in each of four questions
that are asked of respondents: (1) Have you ever felt the need
to cut down on drinking/drug use? (2) Have you ever felt
annoyed by others’ criticism of your drinking/drug use? (3)
Have you ever felt guilty about your drinking/drug use? and
(4) Have you ever had a drink/taken drugs first thing in the
morning (eye-opener)? Two or more “yes” responses are
considered indicative of a substance abuse problem. The
CAGE is administered after 6 and 12 months of program par-ticipation.
Appendix
a-ii
OFFICE OF THE AUDITOR GENERAL
Reference: Mayfield, D., G. McCleod, and P. Hall. The CAGE
Questionnaire: Validation of a New Alcoholism Screening
Questionnaire. American Journal of Psychiatry, 1974, 131, 1121-
1123.
n The Child Safety Checklist (CSC)—The Child Safety
Checklist is an instrument that assesses whether various
safety measures in the home have been implemented. The
CSC is administered by home visitors who ask parents
whether or not each safety measure on the checklist has been
taken (for example, “do you use a car seat for your baby?”).
There are two versions of the child safety checklist. The first
is administered when the child is 2, 6, 12, 18, 24, and 30
months of age. The second contains questions designed for
families with older children and is administered at the fol-lowing
ages: 36, 42, 48, 54, and 60 months.
n Parenting Stress Index (PSI)—The PSI is an instrument
designed to identify stressful situations that could potentially
put parents at risk for “dysfunctional parental behavior,” in-cluding
abuse. It includes several subscales that measure
stress related to child characteristics and parent functioning.
For the evaluation, seven subscales were used. These in-cluded
two that focused on child characteristics (child’s
mood and distractibility/hyperactivity) and five that fo-cused
on adult characteristics (depression, attachment, re-striction
of role, sense of competence, and social isolation).
The PSI is administered after 3 weeks, than at 6 and 12
months of program participation.
Reference: Abidin, Richard R. Parenting Stress Index Admini-stration
Manual, Third Edition. Odessa, FL: Psychological As-sessment
Resources, Inc., 1995.
OFFICE OF THE AUDITOR GENERAL
Agency Response
OFFICE OF THE AUDITOR GENERAL
(This Page Intentionally Left Blank)
ARIZONA DEPARTMENT OF ECONOMIC SECURITY
1717 W. Jefferson - P.O. Box 6123 - Phoenix, AZ 85005
Jane Dee Hull John L. Clayton
Governor Director
Ms. Debbie Davenport, CPA
Auditor General
2910 North 44th Street, Suite 410
Phoenix, AZ 85018
Dear Ms. Davenport:
The Department wishes to thank the Office of the Auditor General for the opportunity to
respond to the recently completed audit of the Healthy Families Arizona Program.
Report findings II and III indicate that the program benefits the families we serve. Finding I is
inconclusive and is based on an indicator (child abuse report rates) that is a relatively low
occurring event in the participant families, the comparison group, and in the general population.
It is important that we keep in mind that the reason the program exists and the reason we focus
resources on prevention is to help ensure that no child is maltreated and that all children have the
chance to be nurtured and comforted during their growing years into adulthood. Because of the
years of positive evaluation outcomes, we believe the Healthy Families Arizona Program is an
effective prevention program.
The Department welcomes the opportunity to work with you in helping to find other methods of
determining the impact of the program’s positive outcomes on the prevention of child abuse and
neglect.
The Department will continue to utilize the new forms listed in the two recommendations under
Finding II.
Finally, please accept our appreciation for the time and effort invested in this important
evaluation. We wish to specifically recognize Laurie Cohen and JoAnne Dukeshire for their
dedication during the evaluation process.
Sincerely,
John L. Clayton
DEPARTMENT OF ECONOMIC SECURITY
RESPONSE TO THE
HEALTHY FAMILIES ARIZONA PROGRAM EVALUATION
FINDING I: Healthy Families Participant Child Abuse Rates Do Not
Differ Significantly from the Comparison Group
This report has found no significant difference between Healthy Families participants and the
comparison group on the rates of substantiated CPS reports. The report states that there are
three possible explanations for these results. The true reason may never be known as, but we
do realize that using substantiated child abuse and neglect reports as a measure for program
success is fraught with difficulties.
State child abuse data often are considered to be the primary indicator of successful prevention
efforts. However, if one wishes to obtain an accurate appraisal of family functioning, using CPS
data may not be adequate. For example, a national study found that one third of child
maltreatment incidents go unreported (Sedlak & Broadhurst, 1996), so reliance on this data will
produce an incomplete picture.
Observations of interactions between a parent and child, maternal warmth, sensitivity, and
nurturing are better indicators of parent-child relationships than maltreatment reports. Healthy
Families Arizona independent evaluations have time and again over the years showed excellent
outcomes in these meaningful areas.
The Department will cooperate in any way possible in helping to find or develop a better
method of determining the impact of the program’s positive outcomes on the prevention of child
abuse and neglect.
There are no recommendations under this finding.
DEPARTMENT OF ECONOMIC SECURITY
RESPONSE TO THE
HEALTHY FAMILIES ARIZONA PROGRAM EVALUATION
FINDING II: Healthy Families Program Provides Health Referrals, Encourages
On-Time Immunizations and Promotes Safety
The Department agrees the Healthy Families Arizona Program provides health referrals,
encourages on-time immunizations and promotes safety in the program’s participant families.
The Department is proud of the many successes in these areas.
An example of one success is how the direct involvement of a Healthy Families home visitor
saved the life of a family of nine. An early morning fire swept through the two-bedroom
apartment of a participant family. The mother was alerted in time because the Healthy Families
home visitor had recently completed the Child Safety Checklist with the family and had helped
them purchase and install smoke detectors in the apartment.
The finding of the Auditor General is agreed to and the audit recommendation will be
implemented which states that to ensure compliance with its statutorily defined goal to reduce
substance abuse, the Healthy Families Program should continue to implement the new
questionnaire for assessing participant substance abuse problems and method for tracking
substance abuse referrals.
The Healthy Families Program started using the new substance abuse assessment questionnaire
on July 1, 2000 to measure the effect of the program on reducing dependency on alcohol and
drugs over time. Questions have been added to the family update form to enable staff to track
referrals for substance abuse treatment.
The finding of the Auditor General is agreed to and the audit recommendation will be
implemented which states that to ensure families with children who receive scores in the delayed
range on the Ages & Stages Questionnaire (ASQ) receive appropriate referrals, the Healthy
Families Program should continue to implement new measures for tracking these referrals.
As this report states, the Healthy Families Program already revised the family update form,
which was implemented on July 1, 2000. The form includes tracking reporting information on
referrals for children identified as having a potential developmental delay. This new form
contains questions which supplement the ASQ, and is being used to track whether children with
developmental delays are offered appropriate referrals.
DEPARTMENT OF ECONOMIC SECURITY
RESPONSE TO THE
HEALTHY FAMILIES ARIZONA PROGRAM EVALUATION
FINDING III: Parenting Stress Measure Shows Improved Family Relations
The Department agrees the Healthy Families Program has improved family relations in
participant families. The outcomes from the Parenting Stress Index show that parental stress
decreases after 12 months of participation in the program. As pointed out in this report,
research shows that high scores on parenting stress are related to the potential to abuse one’s
child, and, conversely, low to normal scores are related to positive parent-child relationships.
The program also assists participants with higher stress scores to seek mental health services,
thus providing help to parents so that they have a better opportunity to become nurturing
parents to their children.
The Department fully intends to continue independent evaluation of the Healthy Families Arizona
Program and to maintain national accreditation to ensure continued excellence in program
delivery and administration.
There are no recommendations under this finding.
Other Performance Audit Reports Issued Within
the Last 12 Months
00-4 Family Builders Pilot Program
00-5 Arizona Department of Agriculture—
Licensing Functions
00-6 Board of Medical Student Loans
00-7 Department of Public Safety—
Aviation Section
00-8 Arizona Department of Agriculture—
Animal Disease, Ownership and
Welfare Protection Program
00-9 Arizona Naturopathic Physicians
Board of Medical Examiners
00-10 Arizona Department of Agriculture—
Food Safety and Quality Assurance
Program and Non-Food Product
Quality Assurance Program
00-11 Arizona Office of Tourism
00-12 Department of Public Safety—
Scientific Analysis Bureau
00-13 Arizona Department of Agriculture
Pest Exclusion and Management
Program
00-14 Arizona Department of Agriculture
State Agricultural Laboratory
00-15 Arizona Department of Agriculture—
Commodity Development Program
00-16 Arizona Department of Agriculture—
Pesticide Compliance and Worker
Safety Program
00-17 Arizona Department of Agriculture—
Sunset Factors
00-18 Arizona State Boxing Commission
00-19 Department of Economic Security—
Division of Developmental
Disabilities
00-20 Department of Corrections—
Security Operations
00-21 Universities—Funding Study
00-22 Annual Evaluation—Arizona’s Family
Literacy Program
01-01 Department of Economic Security—
Child Support Enforcement
Future Performance Audit Reports
Department of Public Safety—Telecommunications
Department of Public Safety—Drug Abuse Resistance Education (D.A.R.E.) Program
Board of Osteopathic Examiners in Medicine and Surgery