Annual evaluation, Health Start pilot program

              ANNUAL EVALUATION
HEALTH START
PILOT PROGRAM
State o f Arizona
office
of the
Auditor General
Report to tLe Arizona Legislature
By Douglas R. Norton
Auditor General
January 1992
Report No. 91- 1
DOUGLAS R. NORTON, CPA
AVDlTDl GENERAL
STATE OF ARIZONA
OFFICE OF THE DEBRA K. DAVENPORT, CPA
DEPUTY AUDITOR GENER* L
AUDITOR GENERAL
January 17,1997
Members of the Arizona Legislature
The Honorable Fife Symington, Governor
Dr. Jack Dillenberg, Director
Department of Health Services
Transmitted herewith is a report of the Auditor General, an Annual Evaluation of the Health
Start Pilot Program. Tlus report is in response to the provisions of Session Laws 1994, Ninth
S. S., Chapter 1, 59.
This is the second in a series of three reports. The final evaluation is scheduled to be released
on or before December 31,1997. Our evaluation fmds that although there are some variations
from the model, the services provided by Health Start address the Program's goals. Also,
while Health Start clients are experiencing a low rate of low birth weight babies and are
receiving adequate prenatal care, it is not yet known to what extent the outcomes can be
attributed to the services the Program provides. The Department of Health Services has
improved the contracting and reimbursement process with its Health Start providers. Finally,
it is noted that there are some factors that may limit the outcome evaluation that is due by the
end of the year.
My staff and I will be pleased to discuss or clarify items in the report.
Tlus report will be released to the public on January 20,1997.
Sincerely,
~ o k j $ d RN. o rton
Auditor General
Enclosure
2910 NORTH 44TH STREET m SUITE 410 . PHOENIX, ARIZONA 85018 . ( 602) 553- 0333 . FAX ( 602) 553- 0051
SUMMARY
The Office of the Auditor General has completed the second year of a three- year evaluation
of the Health Start Pilot Program. The evaluation was conducted pursuant to the provisions
of Laws 1994, Ninth S. S., Chapter 1/ 99. This second- year, interim evaluation report provides
a description of and some preliminary outcomes for the Program. The final evaluation report
will focus on the Program's impact and is to be released on or before December 31,1997.
The Legislature established the Health Start Pilot Program with the legislation known as the
Arizona Chldren and Families Stability Act of 1994. The Joint Committee on Community
Program Evaluation was created by an amendment to that law, Laws 1996, Chapter 247, to
oversee program implementation and recommend criteria concerning provider contracts,
eligibility screening, and service delivery.
Administered by the Arizona Department of Health Services' Office of Women and Chldren's
Health, Health Start uses lay health workers in prenatal education outreach efforts in selected
Arizona communities. Arizona's Health Start Pilot Program attempts to provide chldren with
a healthy start in life by identtfying pregnant women in communities the Program serves and
providing clients with education, advocacy, and referrals to needed services. The percentage
of pregnant women the Program served in the targeted communities ranged from 3 to 90
percent.
The Program's speclfic goals are to increase pregnant women's access to prenatal care, reduce
the incidence of low birth weight babies, improve childhood immunization rates, reduce the
incidence of children affected by chldhood diseases, provide information about preventive
health care, and assist families in identifying programs that prepare chldren to start school.
Health Start Services
Address Program Goals but
Variations to the Model Exist
( See pages 7 through 13)
The Health Start model as a home- based, community outreach program may be threatened
by several program factors. The four- year follow- up period may be longer than needed to
accomplish the Program's primary goals, and may diminish the Program's ability to provide
prenatal services to eligible women in the communities. In addition, some Health Start
providers are associated with health clinics and almost half receive more than one- fourth of
their client referrals from the clinics, rather than recruiting direct lay health worker outreach
effort clients who are more in need of services. Some of the large providers rely on providing
services through group rather than individual encounters, which deviates from the lay health
worker model.
Health Start does, however, appear to be providing services to clients who can benefit. Many
Health Start clients face obstacles to receiving adequate prenatal care. They are predominantly
low- income, minority women who, as a group, traditionally need help in understanding and
accessing proper medical services.
In addition, the services Health Start provided target the goals mandated by the legislation.
Education and referral services are the primary means by which the Program attempts to
increase clients' understanding of health issues and prevent behaviors that can result in medical
problems. Health Start's educational and referral services generally focus on program goals.
These services are delivered in a variety of settings, including clients' homes, the program office,
and group classes. It appears that the Program addresses most of its goals through educating
clients and referrals to appropriate services.
To deliver these services, the Program recruits and trains lay health workers who are
representative of, and therefore understand, the communities and cultures they serve.
Some Factors Affecting Outcomes
are Unclear
( See pages 15 through 17)
Preluninary results show individual Health Start clients are receiving adequate prenatal care
and experiencing a low incidence of low birth weight babies. Preliminary analysis for the first
year of data suggests that Health Start clients are experiencing positive outcomes. When
compared to rates for the State, AHCCCS clients, or all women in communities that Health Start
serves, individual Health Start clients are reporting adequate prenatal care and fewer low birth
weight babies. However, it is not known to what extent the outcomes can be attributed to the
services provided by Health Start.
Analysis of outcomes for individual Health Start communities shows no consistent program
effects. The number of women receiving adequate prenatal services has increased in most
communities, whle the number of low birth weight babies has also increased. For example,
in Guadalupe, the rate of low blrth weight worsened from 6.2 to 7.8 percent between 1993 and
1995, but the percentage of women who entered prenatal care in their first trimester increased
during ths same period.
The Arizona Department of
Health Services Has Improved Its
Health Start Pilot Program Contracts
( See pages 19 through 23)
Anzona Department of Health Services ( ADHS) has improved its Health Start Pilot Program
fiscal year 1997 contracts from the original contracts used in 1995 and 1996. These contract
improvements include a better reimbursement process that equalizes rates across sites, increases
contractors' flexibihty in providing services, and removes disincentives to providing services.
The contract changes may, however, result in hgher costs per client.
Statutory Annual
Evaluation Components
( See pages 25 through 31)
This report also contains information required to be included in each annual evaluation by Laws
1994, Ninth S. S., Ch. 1, 59. As part of this information, we note that some of the outcome
evaluation due next year may be lunited because of several factors. First, there is concern that
it wdl not be possible to isolate other programs' efforts from the Health Start Program's effects.
Additionally, the relatively short length of the evaluation may make it difficult to assess some
program outcomes that are related to the age of the chldren in the Program, and some
performance outcomes may be unrealistic expectations for a lay health worker- provided, home
visitation program of limited intensity.
iii
( This Page Intentionally Left Blank)
Table of Contents
Page
Introduction and Background . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Finding I: Health Start Services
Address Program Goals but
Variations to the Model Exist . . . . . . . . . . . . . . . . . . . . . . . . .
Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Deviations from Health Start
HomeVisitModel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Health Start Clients' Poverty
and Cultural Barriers May
Deter Prenatal Care . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Health Start Provides Relevant
Information in a Variety of Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Health Start Oversees
Lay Health Worker
Recruitment and Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
Finding II: Some Factors Affecting
Outcomes Are Unclear . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Preliminary Analysis Shows
Positive Client- Level Outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Positive Outcomes
Not Appearing at
theComrnunityLeve1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Table of Contents ( con't)
Paae
Finding Ill: The Arizona Department of
Health Services Has Improved Its Health
Start Pilot Program Contracts . . . . . . . . . . . . . . . . . . . . . . . . 19
Health Start Pilot Program
Negotiates Improved
Performance Contracts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Improved Contracts Impact
Program Delivery and Costs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
Statutory Annual
Evaluation Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
Agency Response
Figures
Figure 1: Health Start Pilot Program
Client Ethxucity
March 1,1995 through February 29,1996 . . . . . . . . . . . . . . . . . . . . . 10
Figure 2: Health Start Pilot Program
Client Health Insurance Coverage
at Time of Giving Birth
March 1,1995 through February 29,1996 . . . . . . . . . . . . . . . . . . . . . 25
Figure 3: Health Start Pilot Program
Percentage of Clients Entering
Postnatal Family Follow- up or Disenrolling
March 1,1995 through February 29,1996 . . . . . . . . . . . . . . . . . . . . . 28
Table of Contents ( concl'd)
Paae
Tables
Table 1: Health Start Pilot Program
Providers, Service Areas, and Contract Amounts
Years Ended or Ending June 30,1995,1996, and 1997
( Unaudited) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Table 2: Health Start Pilot Program
Percentage of Clients Receiving
Instruction by Educational Topics
and Related Legislated Goals
March 1995 through June 1996 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Table 3: Health Start Pilot Program
Provider Reimbursements for Prenatal Clients
Years Ended or Ending June 30,1996 and 1997
( Unaudited) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
Table 4: Health Start Pilot Program
Services Contracted and Provided
Year Ended June 30,1996 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
Table 5: Health Start Pilot Program Budget
Year Ended June 30,1996
( Unaudited) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
Table 6: Health Start Pilot Program
Cost per Client and Visit
Years Ended June 30,1995 and 1996 . . . . . . . . . . . . . . . . . . . . . . . . . 29
vii
( This Page Intentionally Left Blank)
viii
INTRODUCTION AND BACKGROUND
The Office of the Auditor General has completed the second year of a three- year evaluation
of the Health Start Pilot Program. The evaluation was conducted pursuant to the provisions
of Laws 1994, Ninth S. S., Chapter 1/ 59. This second- year, interim evaluation report provides
descriptive and preliminary outcome information regarding the Program. The final evaluation
report will focus on the Program's impact and is to be released on or before December 31,1997.
Legislation and
Appropriations
The Legislature established the Health Start Pilot Program with legislation known as the
Arizona Children and Famhes Stabihty Act of 1994. Administered by the Arizona Department
of Health Services' Office of Women and Children's Health ( OWCH), Health Start uses lay
health workers in prenatal education outreach efforts in selected Arizona communities. The
Joint Committee on Community Program Evaluation was created by Laws 1996, Chapter 247,
to oversee program implementation and recommend criteria concerning provider contracts,
eligibility screening, and service delivery.
State appropriations for the 1995 fiscal year totaled $ 975,000. For each of the fiscal years 1996,
1997, and 1998, $ 1,400,000 was appropriated.
Need for the Program,
Its Goals and Services
Timely and adequate prenatal care can reduce the incidence of low brrth weight, whch in turn
can improve a child's health, reduce health care costs, and reduce infant mortality. Although
it is important for pregnant women to receive prenatal care in the first trimester, vital statistics
show that 31 percent of Arizona's pregnant women did not receive such care in 1993, the year
prior to Health Start's implementation.' In addition, the incidence of low birth weight babies
born in Anzona rose from 6.1 percent in 1983 to 6.8 percent in 1994.
Arizona's Health Start Pilot Program attempts to provide chldren with a healthy start in life
by identdying pregnant women in communities the Program serves and providing them with
education, advocacy, and referrals to needed services. The Program's specific goals are to
increase pregnant women's access to prenatal care, reduce the incidence of low birth weight
babies, improve chldhood immunization rates, reduce the incidence of children affected by
1 Vital statistics reported in the ADHS publication " Arizona Health Status and Vital Statistics 1993."
1
childhood diseases, provide information about preventive health care, and assist families in
identifying programs that prepare children for school.
Health Start lay health workers strive to meet these goals by:
w Using outreach and networlung techniques to identdy and approach potential clients;
w Educating and assisting clients with accessing appropriate prenatal, chdd, and family health
care;
w Educating clients about proper nutrition and preventive health care behaviors;
w Encouraging child immunization and enrollment in early childhood education; and
H Assisting participants in applying for applicable community and public services, including
employment services.
Program Model Has
Changed over Time
The program from whch the Health Start Pilot Program was modeled has changed sig- luficantly
since it was initiated in 1988. The predecessor to today's Health Start, Un Comienzo
Sano/ Health Start, began serving h o n a communities in 1988 through a federal Rural Health
Outreach grant administered by Anzona State University. In 1993, the Program was expanded
when ADHS began to provide monies, and it was expanded again in 1994 with the Arizona
Children and Families Stability Act Today's Health Start Pilot Program is a community
outreach program delivering health education and referral services to women and their famllies
through 12 providers in 66 communities in 11 of Arizona's 15 counties. Although the Program
targets pregnant women in specific communities, the percentage of pregnant women the
Program served in the targeted communities ranged from 3 to 90 percent. See Table 1 ( page
4), for a listing of the program providers, the areas they serve, and their contract amounts. The
following descriptions illustrate the Program's expanding focus.
1988 Model - Prenatal Only Focus
Un Comienzo Sano/ Health Start began in 1988 in Yuma County, Arizona. It focused on
prenatal education, referral for health care needs, and client advocacy for pregnant women.
Women received one post- natal visit, and little formal emphasis was placed on assisting
the rest of the family.
1993 Model - Prenatal and Immunization Focus
Health Start expanded its scope in 1993 with financial support from the National Association
for the Education of Young Chddren to include a two- year follow- up period for Health Start
infants and their siblings. The follow- up period included at least six home visits by lay
health workers in the first year of client enrollment and focused on the importance of
immunization and preventative health care education.
1994 Model - Prenatal and Family Preventive Health
The 1994 legislation retained the lay health worker as the primary source for outreach and
delivery of services to pregnant women in the Health Start Pilot Program, but expanded
the Program's scope to include:
1) Extending the family follow- up period from two to four years;
2) Educating families on the importance of early identification of developmental
abnormalities, and screening examinations for the entire family;
3) Assisting families in identifying private and public school readiness programs; and
4) Promoting client self- sufficiency, literacy, and community involvement.
1996 Model- Eligibility Criteria Inclusion
The 1996 legislation retained all of the 1994 model provisions but required ADHS to develop
eligibility criteria for individuals. Previously, all pregnant women in a contractor's service
area were eligible for the Program. As of October 1996, ADHS had developed criteria and
began using a 35- point screening tool based on behavioral, physical, and social risk factors.
Women who score above a designated level are eligible for the Program.
1995 Report and Follow- up
In the first year's report, several problems with the Health Start Pilot Program were
identified ( Auditor General Report 96- 2). In response to these concerns, the Legislature
created new requirements for the Program through Laws 1996, Second Regular Session,
Chapter 247. The ADHS has responded in the following ways to the concerns that were
identified in the first- year report and addressed by the 1996 legislation:
Procedures for selecting pilot sites need to be improved - Although problems with pilot
site selection were identified in the first- year report, ADHS did not revise criteria for site
selection since new contractors were not sought for the 1997 fiscal year. ADHS has reported
it will seek new contractors if and when the Program is expanded and will revise site
selection criteria only if this occurs.
Table 1
Health Start Pilot Program
Providers, Service Areas, and Contract Amounts
Years Ended or Ending June 30,1995, 1996, and 1997
( Unaudited)
Provider Service Area
County Health Departments
Cochise Douglas and Bisbee
Coconino Page and surrounding areas
Pima
Pinal
Tucson and rural areas
Eloy plus Casa Grande in 1997
Yavapai Various communities
Yuma Yuma and surrounding communi-ties
Area Health Education Centers
Northern Arizona ' Hopi and Navajo reservation, and
other communities in Navajo
County plus communities in La
Paz and Mohave Counties for
1996 and 1997
Western Arizona Communities in La Paz and
Mohave Counties
Community Health CenterslBehavioral Health Centers
Centra de Amistad, Inc. Guadalupe
Clinica Adelante, Inc. Migrant areas around Phoenix
Indian Community Native Americans in metropolitan
Health Service, Inc. Phoenix area
Mariposa Community
Health Center Nogales and Rio Rico areas
Mountain Park Health
Center South Phoenix
Total
Contract Amounts
1995 1996 1997
$ 30,150 $ 57,900 $ 84,000
60,800 86,400
24,210 82,000 75,300
38,080 114,050 121,000
31,650 73,500 70,300
Northern Arizona Area Health Education Center merged with the Flagstaff Community Free Clinic in 1996
to form the North Country Community Health Center and assumed responsibility for communities in La
Paz and Mohave Counties.
Source: Auditor General staff analysis of data provided by the Arizona Department of Health
Services, Health Start, proposals and contracts, Office of Women and Children's Health
summary map of Health Start providers and sites, and Health Start database.
EfSoouts needed to coordinate with related programs- In the first year, it was reported that
some sites might be over- serving women or enrolling participants who would be better
served by another program. In response to this concern, Chapter 247 established the
requirement for ADHS to conduct a Health Start Program coordination study. The
coordination study, published October 1,1996, repudiates some of the Office of the Auditor
General's concerns by failing to identify many similar programs the Health Start Program
could coordinate with or which might be providing similar services to the same population.'
However, the study's conclusions are open to interpretation because it reports that only
three programs have clear similarities to Health Start, and its criteria for " similar" are so
narrow that two of the Health Start sites do not fully meet the criteria. Regarding
coordination and consolidation, the report cites three specific activities that have been
undertaken and identified seven areas, such as training and administration, that are
appropriate for coordination and are being included in a plan to coordinate programs
administered by various agencies and offices.
Lack of individual eligibility miteria- In the first- year evaluation it was reported that lack
of eligibility criteria for the Program could result in Health Start serving families who do
not need services or who might be better served by another program. As a result of these
concerns about lack of eligibility criteria, the ADHS has identified 35 factors to assess
women's eligibility for the Program. The risk factors include health problems, such as heart
problems or high blood pressure; use of drugs, alcohol, or tobacco; being homeless or
migrant; and having a hstory of miscarriages. All newly enrolled clients will be assessed
in all of these areas. ADHS began using the eligibility screen on October 1, 1996. The
appropriateness of the criteria and the minimum eligibility score will be assessed in the
last year of the evaluation and will be discussed in the final report.
The information gathered from the eligibility assessment will also be used by the lay health
workers to identify the specific types of services and referrals each client needs. For
example, a pregnant woman who does not have enough money to meet basic needs could
be referred to private charitable or public assistance programs, or a mother who does not
speak English could be counseled to take an English class.
Evaluation Scope
and Methodology
The Arizona Children and Family Stability Act requires the Office of the Auditor General
to annually evaluate the results of the Health Start Pilot Program. The Act requires
evaluation of items such as the Program's effectiveness, its organizational structure and
efficiency, the type and level of criteria used to establish eligibility for the Program, and
the number and characteristics of the people receiving services from the Program.
The primary methods used in this evaluation include: 1) analysis of program participant
data contained in the ADHS Health Start database, 2) review of Health Start Pilot Program
' ADHS contracted with Gill and Cannon, Inc., to conduct the required assessment of the feasibility of the
comprehensive program coordination for the Health Start Pilot Program.
5
documents, 3) interviews with Health Start providers, 4) observations during 19 lay health
workers' client visits, and 5) analysis of aggregated vital statistics from the Arizona
Department of Health Services. In addition, 17 staff trainings were observed and literature
concerning the prevention of low birth weight babies and improving the effectiveness of
lay health worker programs was reviewed.
This evaluation is the second in a series of three. In addition to the issues discussed above,
the first- year evaluation report included information regarding the Program reverting a
significant amount of its appropriation due to the short amount of time it provided services
during the 1995 fiscal year, and information on the Arizona Family Resource Guide, a list
of medical services statewide that is to be given to Health Start clients and to all women
in the State released from hospitals after giving birth. This second report provides
information regarding the Program's implementation. Specifically presented are:
How Health Start appears to be providing services mandated by legislation, but some
questions about the implementation and effectiveness of the model cannot be answered;
Preliminary analysis of how outcomes show results are mixed.
Information on contract reimbursement procedures.
A report on statutory annual evaluation components including client characteristics
and program costs.
The tlurd and final report will focus on the Program's effectiveness in meeting its goals
and objectives and the impact the Program has had on program participants.
The Auditor General and staff express appreciation to the Director of the Department of
Health Services, the Chief and staff of ADHS' Office of Women and Children's Health,
and the Health Start Pilot Program staff for their cooperation and assistance during the
second year of the Health Start Pilot Program Evaluation.
FINDING I
HEALTH START SERVICES
ADDRESS PROGRAM GOALS BUT
VARIATIONS TO THE MODEL EXIST
The Health Start model as a home- based, community outreach program may be threatened
by several program factors. The lengthy four- year family follow- up period, the relationship
between providers and their parent organizations, and a reliance on group versus
individual- based services may threaten the Program's ability to provide adequate services.
However, the Program does appear to be providing services to clients who will benefit,
the services appropriately target program goals, and the Program recruits and trains lay
health workers who are representative of, and therefore understand, the communities and
cultures they serve. The ADHS should monitor the program providers to ensure the
potential benefits of the program model and the services provided are not compromised.
Background
The Health Start Pilot Program was designed to serve communities with a high incidence
of inadequate prenatal care, inadequate infant health care, low birth weight babies, or
inadequate early childhood immunization. To effect positive changes in these outcomes,
lay health workers educate families on the importance of early and adequate prenatal care,
family planning, good nutrition, child development, preventive health care, benefits of
education for all, and self- sufficiency. Lay health workers also help families access needed
social, nutritional, and medical services through referrals and one- on- one assistance.
Deviations from Health
Start Home Visit Model
The Health Start model as a home- based, community outreach program may be threatened
by several program factors. First, the Program has a lengthy four- year family follow- up
period, which may interfere with prenatal services. Second, the relationship between
providers and their parent organizations may focus services on clients with less need. Tlurd,
reliance on group versus individual- based services may threaten the Program's ability to
provide adequate services. The ADHS should monitor the program providers to ensure
the benefits of the program model are not compromised.
Four- year follow- up period may be too extensive- Many clients elect to enter the family
follow- up phase of the Program after delivering their babies. Family follow- up is currently
designed to last for four years to ensure healthy behaviors are being maintained. However,
this lengthy period reduces the amount of time lay health workers have to work with
pregnant clients. For example, of the seven providers who met or exceeded the number
of family follow- up visits in their contracts, only two met or exceeded their contracted
number of prenatal services. No providers met their contracted number of prenatal services,
but not their family follow- up contracts. In addition, during the second year of
implementation, the Program was serving many more clients in family follow- up than in
prenatal. This may reduce the Program's ability to positively impact their communities'
birth outcomes, because a lower percentage of pregnant women in the community are being
served.
To limit family follow- up's impact on reducing services to pregnant women, consideration
should be given to reducing it to two years. Children should be fully immunized by age
two, and this period should be sufficient to provide the educational and referral support
to prepare families to find assistance independently. If further support is needed, families
could be referred to other social service agencies for more intensive family support,
allowing the lay health workers to focus on providing the prenatal and family follow- up
services they are best prepared for.
Smne providers rely heavily on medical clinic and public lzealth oflice connections - Many
clients served by Health Start are referred to the Program by the medical clinics and public
health offices contracted to provide Health Start services. This creates a relationship in
whch the Program is an extension of the services being received through the contractor's
parent organization. In such instances, many clients are those who sought prenatal care
themselves and then were directed to Health Start services, not Health Start finding and
recruiting clients and getting them into prenatal care they would otherwise avoid.
Currently, at least five providers receive between 25 and 73 percent of their clients from
clinics. These same providers receive only 10 to 47 percent of their clients from lay health
workers' direct outreach efforts. These relationships create a different mode of service
delivery and bring into question the Program's mission as provided by these contractors.
It is unclear if Health Start is bringing people who need prenatal care into clinics, or if
clinics are using Health Start resources to supplement services they are already providing.
Large providers are relying on group rather than individual smice delive y - Many of the
larger providers frequently use group classes as opposed to individual visits to provide
services. This differs from the model of a " home- based" system of service delivery. The
classroom- type atmosphere differs dramatically from the intimate, and presumably more
confidential, atmosphere of the clienfs home. While the efficiency of such methods of
delivery may be beneficial, such a public setting brings into question the Program's ability
to address sensitive client needs.
Probleutzs with model ltnve not resulted in poor smices- The concerns about clients being
referred from clinics and services provided in group rather than individual settings does
not mean that clients are receiving poor service, only that there is variation from the service
delivery model that must be controlled by the Health Start administration. Health Start
needs to closely monitor the relationships among these contractors to ensure the benefits
of a home- based, community outreach- type program are not compromised.
Health Start Clients' Poverty
and Cultural Barriers May
Deter Prenatal Care
Despite the deviations from the program model, the Program does appear to provide
services to clients who will benefit from them. Many Health Start clients face obstacles
to receiving adequate prenatal care. These obstacles include poverty, as well as cultural
and language barriers.
Poverty has long been recognized as a leading factor for women not receiving adequate
prenatal care, and many Health Start clients would be defined as living in poverty
according to federal guidelines. Eighty- six percent of Health Start's clients were enrolled
in AHCCCS at the time they gave birth.' In comparison, approximately 12 percent of all
Arizonans are enrolled in AHCCCS. This high percentage of clients enrolled in AHCCCS
suggests Health Start clients are much more likely to have lower incomes than the general
population. Poverty, however, is not the only potential barrier to adequate prenatal care
that Health Start clients face.
Cultural and language barriers are also recognized as deterrents to women receiving
prenatal care. New immigrants to the U. S. often seek medical attention only when illness
occurs, and pregnancy is not considered an illness. Preventive health care is not a familiar
concept to many new Health Start clients. Language barriers also present an obstacle to
prenatal care if women cannot effectively communicate with their medical providers. As
seen in Figure 1 ( see page lo), Health Start is serving a primarily Hispanic population,
requiring many lay health workers to be bilingual.
Health Start Provides Relevant
Information in a Variety of Settings
Health Start providers use various methods to dispense relevant information to clients.
Clients can receive services in their homes, in provider offices, and in various community
locations. Many of the Program's goals are addressed by educating clients about relevant
topics or referring clients to appropriate services.
Semices delivered in a variety of settings- Health Start lay health workers provide services
to clients in various places and in various ways. For example, services are provided at
clients' homes, in clinics, in group classes, and in other locations. Preliminary Health Start
1 AHCCCS, the Arizona Health Care Cost Containment System, is the State's program to provide health care to
the indigent.
Figure 1
Health Start Pilot Program
Client Ethnicity
March 1, 1995 through February 29, 1996
AsianIPacific Islander
0.2%
Native American
18.7%
Number of women = 2,020
Source: Auditor General staff analysis of Health Start database.
data show clients received an average of seven lay health worker visits during their
pregnancies, which exceeds the Program's goal of at least five prenatal visits. Fifty- four
percent of these meetings were at clients' homes, 12 percent were at Health Start offices,
and 17 percent were at other places ( such as community centers, clinics, schools, or
hospitals). Another 17 percent of the services were not provided on an individual basis,
but in group classes.
Services provided relnte to prograns goals- The educational topics discussed during client
visits cover a wide range of subjects, most of which relate to the goals and instruction
specified in the legislation establishing the Program. Health Start program data indicates
at least 27 educational topics were discussed with clients during the Program's first year.
Table 2 ( see page ll), illustrates those topics discussed most often during visits and classes.
Lay health workers refer clients to semices- Part of the Program's mission is to inform
participants how to receive prenatal care and assist them in accessing appropriate prenatal
and social services. During the first meeting with lay health workers, half of the women
are referred to the Women, Infant and Chldren Program ( WIC), whch provides nutritious
food for needy expectant mothers. Over 40 percent are referred to AHCCCS, and one- third
are referred directly to a clinic, hospital, or doctor's office. Nearly 20 percent are referred
to the Department of Economic Security ( DES) for social services. With each visit, clients'
needs are assessed and further needed referrals are made. Helping clients translate forms,
arranging appointments, and even providing bus tokens are also part of the Program's
referral and assistance activities.
Table 2
Health Start Pilot Program
Percentage of Clients Receiving Instruction
by Educational Topics
and Related Legislated Goals
March 1995 through July 1996
Percentaue Educational Topic
Breast feeding
Prenatal care
Emotions/ feelings
Women's health
Children's nutrition
Immunization needs
Transportation
Infant care
Child development
Safety
Finances
Leclislated Goal
Improve the overall health of children through
good nutrition
Increases prenatal services
( No directly stated goal or instruction)
Educate on the benefits of preventive health care
Improve the overall health of children through
good nutrition
Encourage age- appropriate immunization
Increase prenatal care services
Reduce the incidence of children affected by
childhood disease
Promote early identification of developmen-tal
disabilities.
5 Educate on the benefits of preventive health care
Assist in obtaining financial assistance ( a legislated
service but not a goal).
1 Six of the eight Health Start goals mandated by the enabling legislation are covered by these topics. One
goal not covered- reducing the rate of low birth weight babies- is an indirect goal achieved through
increased prenatal visits. The other goal is identifying public and private preschool programs.
Source: Auditor General staff analysis of Health Start database.
However, it is unknown whether other needed self- sufficiency topics were discussed with
clients because Health Start providers never collected data for these key components. For
example, no data exists regarding a client's employment referrals or community service
activities. Ths lack of data prevents any systematic analysis of these important program
efforts. A newly designed form that recently went into effect includes areas for collecting
data on employment and hearing/ vision referrals, and an analysis of efforts in these areas
will be included in the third evaluation due in December 1997.
Health Start Oversees Lay
Health Worker Recruitment
and Training
Health Start oversees the screening, training, and certification of lay health workers to
ensure they are qualified and prepared to perform their duties. The lay health worker often
becomes the conduit through which clients access health, nutritional, and social services.
Lay health workers are representative of the populations they serve; therefore, providers
work to recruit and hire lay health workers with the same ethnic, cultural, and social-economic
characteristics as their clients. Additionally, as required by legislation, all lay
health workers must undergo a background check as a condition of employment and
complete an affidavit that they have not committed a felony or misdemeanor involving
moral turpitude.
Health Start uses a specific training and testing program to ensure lay health workers are
qualified to serve clients. Workers must complete a core training course that contains 60
specific educational objectives. They must then pass a test specific to this training with a
score of 90 percent or better. Prior to the core training workers must also complete eight
hours of orientation training in seven key educational topics:
The lay health worker's role
Pregnancy
Child growth and development
Communication skills
Identifying and accessing community resources
Documentation and confidentiality
Supervised home visits.
Lay health workers must complete orientation training and demonstrate proficiency in
conducting home visits before they are able to work independently. Core training must
be completed withn 90 days after workers are hired. This includes training unique to their
community ( for example, referrals to Indian Health Services in predominantly Native
American communities). ADHS issues a Certification of Completion when training is
finished. Each lay health worker also has an individual continuing education plan, which
includes at least 6 annual hours of ADHS- approved courses.
Recommendations
1. If the Health Start Pilot Program is reauthorized for fiscal year 1999, the Legislature
should consider reducing the family follow- up period to a maximum of two years.
2. The ADHS should monitor the program providers to ensure adherence to the home-based,
community outreach model.
( This Page Intentionally Left Blank)
FINDING II
SOME FACTORS AFFECTING
OUTCOMES ARE UNCLEAR
Preliminary results show Health Start clients are receiving adequate prenatal care and
experiencing a low incidence of low birth weight babies. However, these positive outcomes
cannot, at this time, be attributed to the services provided by the Program. In addition,
analysis of low birth weight and adequate prenatal care rates for communities Health Start
serves reveals no consistent improvements in the community- level rates.
Background
Program outcomes can be viewed at two different levels, the client level and the community
level. Health Start is working to effect improvement at both levels. By educating clients
on how to ensure good reproductive and family health, and directly assisting clients in
accessing medical, nutritional, and social services, the Health Start Program hopes to effect
community change, one client at a time. For this reason, both levels are being examined
and the differences discussed.
Preliminary Analysis Shows
Positive Client- Level
Outcomes
Preliminary data from women delivering babies in the first year of usable Health Start data
suggests positive outcomes are occurring. When compared to the state rate, AHCCCS
clients, or communities that Health Start serves, more Health Start clients are reporting
adequate prenatal care and fewer low birth weight babies. However, it is not known if
Health Start services or other factors are affecting these results.
Health Start clients receive adequate prenatal medical visits- Most clients are receiving
adequate prenatal medical visits. Adequate prenatal care, as defined by five or more
medical prenatal visits, is the standard at both the state and federal level, and is the
Program's goal. Almost all of the Program's clients received five or more prenatal medical
visits. In fact, nine out of ten clients received five or more prenatal visits. As a group, clients
averaged ten prenatal visits.
Health Start clients have fewer low birth weight babies- Health Start clients show a
reduction in low birth weights. ADHS defines low birth weight as less than five- and- a- half
pounds. Health Start clients' babies had a combined low birth weight rate of 4.6 percent,
which compares to a statewide rate of 6.8 percent and a rate for Health Start communities
of 6.7 percent A low birth weight rate of no more than 5 percent is Health Start's goal, and
reflects both state and national standards for the year 2000.
AHCCCS not the reason for low birth weight rates- Most of the women who deliver babies
whle in the Program report being enrolled in AHCCCS ( 86 percent). Even so, AHCCCS
does not appear to be a factor in Health Start clients' reduced low birth weight rate.
AHCCCS clients statewide have a low birth weight rate of 7.4 percent, which is worse than
the state average. AHCCCS participation among Health Start clients cannot explain positive
birth outcomes; however, other sources may still be affecting the Health Start rates and
need to be investigated.
Why outcomes appear better for Health Start clients not yet known- Health Start's low
birth weight rates are better rates for the total population of women in communities served
by Health Start, but why they are better requires further investigation. Self- selection into
the Program, programs such as Baby Arizona working with the same population, and
statistical anomalies are just a few factors that could explain the differences between Health
Start clients and their communities as a whole. A more detailed analysis, including a
comparison group and analysis of factors which may explain differences, will be included
in the third and final evaluation due in December 1997.
Positive Outcomes
Not Appearing at
the Community Level
Whle individual clients in the Program are receiving adequate prenatal care and are having
few low birth weight babies, there are mixed results regarding the Program's impact at
the community level. These community- level data show that the number of women
receiving adequate prenatal services has increased in most communities between 1993 and
1995, whle low birth weight rates are often hgher. It is unlikely that the community- level
impacts of Health Start, if they occur, will be measurable with only one additional year
of data.
At the community level, the number of women receiving adequate prenatal services has
increased in most communities, but low birth weight rates are often worse. From 1993, the
year before Health Start began, to 1995, eight of the ten providers' communities showed
increases in the number of women receiving prenatal care in the first trimester, and seven
of ten communities showed increases in the percentage of women receiving five or more
prenatal visits.
Low birth weight rates, however, were higher for six of the ten providers' communities.
For example, in Guadalupe, the low birth weight rate worsened between 1993 ( 6.2 percent),
and 1995 ( 7.8 percent). During this same period, however, more women received prenatal
care in the first trimester.
It is unlikely that Health Start's effects on community- level outcomes will be measurable
as part of next yeafs final evaluation. Analysis of community- level data will be constrained
by the limited data available on these outcomes. Additionally, the small numbers of
pregnant women in some of the Health Start communities who are served by the Program
make it unlikely that the Program, if it is effective, would show community- level effects
in this limited period.
( This Page Intentionally Left Blank)
FINDING Ill
THE ARIZONA DEPARTMENT OF HEALTH
SERVICES HAS IMPROVED ITS HEALTH
START PILOT PROGRAM CONTRACTS
ADHS has improved its fiscal year 1997 program contracts to include a better reimburse-ment
process. The contract revisions remove some disincentives to providing services and
should allow reimbursement rates to be more appropriately equalized across sites.
However, the changes also result in a higher cost per client and the creation of new
disincentives to contractors.
The Health Start providers are paid a flat dollar amount plus a rate for each client service
to a maximum amount as defined by contract. Contractors do not receive additional
reimbursement if they exceed the maximum number of service units stipulated in their
contracts. As a result, providers maximize revenues by providing as many services as
contractually allowed.
Health Start Pilot Program
Negotiates Improved
Petformance Contracts
The contracts for 1997 are improved over the original contracts used in 1995 and 1996. The
services that contractors are to provide have been reclassified, resulting in the removal of
disincentives to contractors providing services. Additionally, Health Start reimbursement
rates have been equalized across sites.
Restructuring of Health Start reimbursernent system removes disincentives - For fiscal year
1997, ADHS has reclassified the services that contractors are to provide. Under the original
structure, providers were reimbursed for prenatal services based on the number of clients.
For clients in family follow- up, contractors were reimbursed based on the number of
services or visits provided. The contracts for fiscal year 1997 have combined the prenatal
clients and the family follow- up services into a " client visit" service. Under the 1997
structure, providers are reimbursed for each visit for both prenatal and family follow- up
clients.
Under the original contract structure, providers might not be reimbursed for all the services
they provided. The contract limited the amount of money a contractor could be paid for
each service category. Consequently, as described in the following example, a contractor
might not receive reimbursement for all services.
A contractor had a maximum of 200 family follow- up visits and 20 prenatal clients
in their contract. Under the old contracts, if the contractor provided 220 family follow-up
visits and served 15 prenatal clients, the contractor would not be reimbursed for
20 of the 220 family follow- up visits conducted because the allowable number of 200
was exceeded. This would occur even though the contractor had not reached the
maximum dollar amount of the contract because only 15 of the potential 20 prenatal
clients were served. Combining service categories under the new contracts would now
allow this contractor the flexibility to serve clients without concern for whether the
client is prenatal or family follow- up, so long as the contract amount is not exceeded.
As a result, the contractor may now be reimbursed for a higher percentage of the
services provided.
In addition, the original contracts reimbursed providers the same amount regardless of
the number of services clients received. Under the original structure, a contractor received
the same rate for an outreach and prenatal client, whether they had three or more home
visits. Although the Program's goal is to have workers provide at least five client visits
during the prenatal period, previously, contractors had no incentive to provide more than
the minimum requirement of three visits, since they would not receive any additional
reimbursement. However, contractors are now paid for each visit.
1997 reis~ zb~ irseirzarzatt es vanj less tlurrz the 1996 cosztracts- ADHS has modified its Health
Start Pilot Program rate structure. The 1995 and 1996 rates for direct client services varied
substantially among providers. For example, in 1996, rates for " outreach only" clients
ranged from $ 20 to $ 80, and the rate ranged from $ 200 to $ 380 for each " outreach and
prenatal" client. The 1997 rate for " client visits," which replaces the outreach only and
outreach and prenatal services, ranges from $ 50 to $ 60.
Improved Contracts Impact
Program Delivery and Costs
Although the Health Start Pilot Program's new contracts equalize rates for sites and increase
contractors' flexibility in providing services, the changes may result in higher costs per
client, and create an incentive to provide more services to fewer clients. In addition,
reimbursement for group activities based on the number of clients in attendance could
create an incentive to rely more on group classes than on home visits.
Rates increase for outreach and prenatal, and family follow- up- Because contractors can
now be paid for each outreach and prenatal visit, contractors can receive higher
reimbursements for these clients. Contractors averaged 5.7 prenatal visits per client in fiscal
year 1995- 96 but were paid only their base outreach and prenatal or outreach only rate for
these clients. For example, a contractor who received $ 250 per outreach and prenatal client
received this amount whether the client received 3 or 8 home visits. Under the 1997
structure, at $ 50 per visit, the contractor would receive only $ 150 for the client who received
3 visits, but would receive $ 400 for the client who received 8 visits. Using past performance
as a measure, as seen in Table 3 ( see page 22), all but 2 of the sites will receive more for
outreach and prenatal clients in 1997 than in 1996.
In addition to the increased payments for prenatal clients, the rate for family follow- up
visits increased from $ 30 to either $ 50 or $ 60. As a result of these changes, contractors may
now have an incentive to provide more services to fewer clients rather than enrolling more
clients. However, the new client registration payments provide a small incentive to enroll
new clients. Since the Program targets a specific number of visits ( five prenatal, and six
per year for family follow- up), ADHS should set guidelines for the number of prenatal
visits that can be reimbursed for each client and the number of family follow- up visits per
year that can be reimbursed for each client.
Reimbursement of group classes may result in disproportionate compensation for
services- Contractors will receive the same client visit reimbursement for home visits, clinic
visits, or group classes. This means a contractor will be reimbursed at the individual client
visit rate for each client who attends a group class. For a one- or two- hour class that is
attended by ten clients, a contractor can receive $ 500.' Contractors who rely on home visits
would spend about ten hours providing services to ten clients in order to receive the same
reimbursement. While it makes sense to reimburse contractors more for a class than for
a single client visit, and group classes add to the variety of services provided through
Health Start, reimbursing contractors for each client at the same rate as for individual
services may not be necessary. To reinforce the Program's home visit model and to ensure
that contractors who rely more heavily on group classes do not receive disproportionate
compensation, reimbursement rates for classes should be renegotiated. In addition, a lower
rate for individual visits held in the Program's offices versus home visits would further
prevent programs from receiving reimbursement at rates above costs and would help to
maintain the home visit model.
Two of the largest contractors rely heavily on group classes as a form of client visit.
21
Table 3
Health Start Pilot Program
Provider Reimbursements for Prenatal Clients
Years Ended or Ending June 30,1996 and 1997'
( Unaudited)
Provider 1996
County Health Departments
Cochise County
Coconino
Pima
Pinal
Y avapai
Yuma
Area Health Education Center
Northern Arizona
Northern Arizona- West
Community Health CenterslBehavioral Health Centers
Centra de Amistad, Inc. 250
Clinica Adelante, Inc. 250
Indian Community Health Service, Inc. 200
Mariposa Community Health Center 260
Mountain Park Health Center 250
Average reimbursement for each client $ 267
1997
( Projected)
1 Reimbursements are based on each contractofs negotiated and average number of visits provided. The 1996
client rates ranges from $ 200 to $ 380. The 1997 service rates range from $ 50 to $ 60 and the projected average
number of prenatal visits ranges from 3.3 to 9.6.
2 Tlus amount also includes the new " client regstration" payment, whch each contractor receives for enrolling
new clients. No client registration payments were made in 1996.
Source: Auditor General staff analysis of Arizona Department of Health Services Health Start
database and additional information provided by the Arizona Department of Health
Services.
Recommendations
In order to ensure that all clients receive an appropriate number of services and home visits,
and contractors are appropriately and equitably reimbursed for services, the ADHS should:
1. Set guidelines for the number of prenatal visits per year that can be reimbursed for each
client.
2. Set guidelines for the number of family follow- up visits per year that can be reimbursed
for each client.
3. Renegotiate rates for group classes.
4. Renegotiate rates for office visits.
( This Page Intentionally Left Blank)
STATUTORYANNUAL
EVALUATION COMPONENTS
According to Laws 1994, Ninth S. S, Chapter 1, § 9 the Office of the Auditor General is to
address ten factors in annual programmatic evaluations of the Health Start Pilot Program.
Responses to these factors are listed below.
1. Information on the number and characteristics of the program participants.
Health Start served approximately 2,000 primarily low- income, minority populations
between March 1995 and February 1996. Only 10 percent of program clients are non-
Hispanic whites ( see Figure 1, page 10). As seen in Figure 286 percent were enrolled
in AHCCCS at the time they delivered their babies.
Figure 2
Health Start Pilot Program
Client Health Insurance Coverage
at Time of Giving Birth
March 1, 1995 through February 29, 1996
Enrolled in
AHCCCS
86.0% -,
, Pri vate insurancelcare
7.6%
Applied for but denied
AHCCCS coverane
Number of women = 1,105
Source: Auditor General staff analysis of Health Start database.
Additionally, most of Health Start's clients are single. Only 37 percent are married
and 15 percent are cohabitating, leaving nearly half without a partner.
2. lnformation on contractors and program service providers.
The OWCH originally contracted with 13 providers for Health Start services in over
60 urban and rural sites across Anzona. One contract was not renewed for 1996, and
services for that area are now being provided by one of the other original contractors.
The 12 contractors for fiscal years 1996 and 1997 are 6 county health departments and
6 private, not- for- profit providers. Five contractors serve metropolitan areas in Phoenix,
Tucson, and Yuma, and 9 contractors serve rural areas throughout 11 of Arizona's 15
counties. Two of the contractors ( one urban, one rural) serve primarily Native American
participants.
Table 1 ( see Introduction and Background, page 4), shows the contractors and contractor
type, their service areas, and their pilot program contract award for fiscal years 1995,
1996, and 1997.
As seen in Table 4 ( see page 27), contractors did not provide all of the services
contracted for in their fiscal year 1996 contracts. While only one- fourth or less of the
12 providers met the prenatal and outreach, outreach only, and sibling immunizations
contract amounts, seven providers exceeded and one met the contracted number of
family follow- up visits.
The targeted number of client visits in fiscal year 1997 ranges from 350 client visits
for the smallest contractor to 2,184 for the largest contractor. Contracts for fiscal year
1997 also define a " client registration" service unit.' Client registration goals vary across
contractors, with between 80 and 400 new clients being targeted.
3. lnformation on program revenues and expenditures.
Revenues for the Program include $ 1.4 million in Family Stability Act appropriations,
$ 200,000 appropriated to ADHS in a separate line item to provide prenatal services,
and $ 18,793 in federal monies for a total of $ 1,618,793. The budget for the 1996 revenues
is presented in Table 5 ( see page 27). Of the total amount available, the Program spent
$ 1,386,249.
" Client regstration" is enrolling a client into the Program.
26
Service
Table 4
Health Start Pilot Program
Services Contracted and Provided
Year Ended June 30,1996
Number Number Percentage
Contracted Provided Provided
Family follow- up visits 4,850 5,077 105 %
Outreach only clients 1,145 848 74
Outreach and prenatal clients 1,325 942 71
Sibling immunization verifications 2,254 981 44
Source: Auditor General staff analysis of invoice records provided by the Department of Health Services.
Category
Monthly contractor base rate
Outreach and prenatal services
Sibling immunizations
Family follow- up services
ADHS personnel
Professional and outside services
Outreach only services
Other operating costs
ADHS employee- related costs
Unallocated
Travel
Total
Table 5
Health Start Pilot Program
Budget
Year Ended June 30,1996
( Unaudited)
Percentage
of Budget Amount
Source: Auditor General staff analysis of budget and contract information provided by Health Start.
4. lnformation on the number and characteristics of enrollment and disenrollment.
Of the approximately 1,100 women delivering babies while in Health Start's first year
of available data, over 93 percent chose to continue in the family follow- up phase of
the Program. Figure 3 shows enrollment and disenrollment patterns of women
immediately after giving birth while in the Program.
5. lnformation on the average cost for each participant in the Program.
As seen in Table 6 ( see page 29), the cost per visit has significantly decreased in the
program year ended June 30, 1996. The cost per prenatal and outreach clients has
increased because the average number of prenatal visits increased from 2.7 in 1995 to
5.7 in 1996, which is closer to the program goal of 5. The figures for 1995 are for a limited
four- month service period, whereas the 1996 figures cover a full 12 months of services
and, consequently, a longer prenatal period.
Figure 3
Health Start Pilot Program
Percentage of Clients Entering
Postnatal Family Follow- up or Disenrolling
March 1, 1995 through February 29, 1996
Family follow- up
93.2% '
Number of women = 1,127
Source: Auditor General staff analysis of Health Start database.
Withdrew or dropped 1.2%
'/
' R efused further services at time of child's birth 4.3%
Moved 1 . I%
Miscarried 0.2%
Expenditures
Contractor
ADHS
Total
Table 6
Health Start Pilot Program
Cost per Client and Visit
Years Ended June 30,1995 ' and 1996
Visits
1995 1996
Prenatal and
Outreach Clients
1995 1996
1995 costs are for a four- month service period only.
Source: Auditor General staff calculations are based on data provided by the Department of Health
Services.
6. Information concerning progress of program participants in achieving goals
and objectives.
In Finding I1 ( see pages 15 through 17), we report on prenatal medical care visits and
reduced low birth weight rates for Health Start participants. In Finding I ( see pages
7 through 13), we also report on the services that are being provided to the Health
Start clients.
The thrd and final evaluation report will focus primarily on determining if the Health
Start Pilot Program has effectively achieved the Program's goals. Measures for the
following six goals are straightforward.
( 1) Increasing prenatal care services;
( 2) Reducing the incidence of low birth weight babies;
( 3) Increasing the number of children receiving age- appropriate immunizations by
age two;
( 4) Educating families on developmental assessments to promote early identification
of learning disabilities.
( 5) Educating families on the benefits of preventative health care and the need for
screening examinations such as hearing and vision.
( 6) Educating families on the importance of good nutritional habits to improve the
overall health of their children.
Whle the measures for these goals are straightforward, it may be difficult to attribute
outcomes specifically to the Health Start Program. For example, while the incidence
of low birth weight babies can be determined, we may not be able to identify the exact
mechanisms that contribute to the rate or associate Health Start services in a casual
manner to differences between the rate for Health Start clients and the rate for the State.
Additionally, while we can measure the number of women in Health Start who receive
adequate prenatal care, effects from programs such as Baby Arizona may confound
interpretation of the results.
The Program's effectiveness in reachng the other two goals will be more problematic
to measure due to the limited evaluation time. These two goals, assisting families to
identify private and public school readiness programs, and reducing the incidence
of chldren affected by chldhood diseases, will be difficult to capture since the oldest
children in the Program will be only about 2 years old when data collection is complete
for the evaluation, and these goals are more applicable to older children.
Finally, the the Health Start Pilot Program's effectiveness in promoting family unity
and strengthening family relations, reducing dependency on welfare, increasing
employment, and increasing self- sufficiency will be addressed in the final report.
However, the Program's primary purpose and goals, the method of service delivery,
and the low intensity of the program model raise serious questions as to whether the
Health Start Pilot Program will have measurable impacts on these factors.
7. Recommendations regarding program administration.
We have no recommendations regarding program administration at this time.
8. Recommendations regarding informational materials distributed through the
programs.
The Health Start Program has distributed the Kwe Book among its clients. These books
provide parents with information on child development, nutrition, and well- baby care.
The books are not solely for Health Start, and were developed by ADHS and the Sonora,
Mexico, Health Department to serve residents of U. S./ Mexico border communities.
As a result, the Spanish version of the book provides immunization and medical
standards that are used in Mexico rather than in the U. S. The Mexican guidelines are
inadequate and potentially dangerous. Mothers who receive the Spanish version of
the Kwe Book and advice from the lay health worker will receive mixed messages about
proper immunizations for their babies.
The Kare Books should be distributed with U. S. immunization guidelines only.
9. Recommendations pertaining to program expansion.
The Office of the Auditor General has no recommendation pertaining to program
expansion at this time.
10. Recommendations regarding the method used in preparing the Arizona Children
and Families Resource Directory.
The Office of the Auditor General has no recommendation pertaining to the Arizona
Children and Families Resource Directory.
Agency Response
( This Page Intentionally Left Blank)
FIFE SYMINGTON, GOVERNOR
JACK DILLENBERG, D. D. S., M. P. H., DIRECTOR
January 13, 1997
Mr. Douglas R. Norton, CPA
Auditor General
Office of the Auditor General
2910 North 44"' Street, Suite 410
Phoenix, Arizona 85004
Dear Mr. Norton:
Thank you for tlie opportunity to review the preliminary report of the second Annual Evaluation of
the Health Start Pilot Program. This has been a very procluctive year for the Program.
ADHS agrees ulith most of the findings included in the your report and would like to thank you for
identifying some of our successes. \ Ve are pleaset1 that tile preliminary results show that individual
Health Start clients are experiencing positive outcomes by receiving adequate prenatal care and
experiencing a low inciclence of low birth weight babies. Similar results were reported in the
Sum~ naryo f tlie ADHS Health Start Evaluation l? eport, which is attached. We are also pleased that
your report shows that ADHS has improved its Health Start Pilot contracts. ADHS also agrees that
solne of the Success by Six goals are not appropriate for a conimunity based health education and
referral program.
Some statements in your report require additional clarification.
Finding 1. IIcaIth Start Ser\ liccs Adc11. css I'~. ogr;~ rn Goals, Although Val- iations from the Model
Exist
" The four year family follo\ r.- up period may longer t11; ln needed to ; rccomplish the Program's
prim: lry go; lls."
ADHS agrees with this tincling, the original Health Sta- t program was for a period of two years. The
four year period was specified in the 1991 legislation. ADHS would welcome the ability to offer the
Program with only a two year' follow- up to enable more pregnant women to be served.
" In addition, some tIe:~ ltIS~ L : lrt providers ; Ire ussoci:~ tcd with 1le: llth Clinics, and almost half
rcccive more tlliln one Sour- tli of' the cliCilt rc. f'crr:~ l.; f rom t l ~ ecl inics."
Mr. Douglas R. Norton, CPA
January 13, 1997
While ADHS agrees that some providers are receiving referrals from their parent organization, there are
many reasons for this fincling. ( I) Dat:~ is from the one year time period beginning with the initiation
of the Health Start Pilot Program contracts. ( 2) In many instances, for new contractors who are
anxious to begin serving their comnlunities, referrals from agencies, including a parent organization, is
the most efficient method of implementing a new program. ( 3) The door- to door enrollment method
which is possible in small communities is impractical in some communities, such as those served by the
Indian Community Health Center, whose community is pregnant Native Americans in the entire Phoenix
metropolitan area. The ability of providers to tailor the Health Start program to the needs of their
communities is one of the strong points of the Pro=" r am.
" Some of the I; rrge providers rely on providing services through group, rather than individual
encounters."
ADHS agrees with this tincling and thanks you for identifying another success. One site's program
format is preclicatecl on their original promotor prograrn, which was begun in 1988, before the institution
of the Health Start Pilot Program. While program stal'f has questioned this format and that the group
format may not he consistent with the moclel, as does your report, the ADHS Health Start Evaluation
Team tleterminecl that, " Outcome measures fro111 the ADHS evaluation indicate that the site has achieved
oiitc(> mer esi~ ltsf or which tlle benefit is greater than the cost." In addition, one of the strong points of the
Health Start Program is the ability of prc~ viclzrs to moclifj the program to meet the needs of the
community. Program stal'f are investigating group class concerns and addressing these concerns
through collaboration \\! it11 other OWCH Programs ant1 looking at the implications of redefining " Client
Visit".
Finding 2. Some Factors Af'Sccting Outcomes are Unclear
" The numlwr of women receiving adcqu:~ te preni~ tal services hxs increilsed in most communities,
\ vliile the numl, er of lo\ v- birth weight hits also incre:~ sed."
Health Start has not yet aftkctecl perinatal statistics in some communities. ADHS recognizes that the
number of women receiving aclequate prenatal services has increased in some communities, while the
nun~ ber of low birth weight babies has increasecl as well. Current research indicates that an increase in
low birth weight bahies in a community may he accompanied by a decrease in fetal deaths. ADHS is
exploring meclianisms to compare these rates. Overall, tlie Health Start Evaluation Report indicates that
Health Start clients had fewer low- birth weight hahies and very low- birth weight babies than a control
zroup ( n~ atchetlc omparison group) from tlie same communities.
We look forward to working with yoiir staff in tlie coming year and to continuing to provide the
benefits of the Health Start Pilot Program to families in Arizona.
Director
Arizona Department of Health Services Summary of
Health Start Program Evaluation Report
Introduction
The Arizona Department of Health Services ( ADHS) implemented a comprehensive strategy to
strengthen the program evaluation efforts for the Health Start Program. This report includes a
summary of the measures initiated, in the first year of this multi- year project, in data collection
and quality assurance to enhance the capacity required to evaluate program outcomes.
Recommendations made by the Office of the Auditor General in the Program Evaluation
Report of January 1996 concerning data collection and quality assurance have also been
implemented.
The implementation of this comprehensive program evaluation strategy addressed the
following areas:
+ Protocols for Site Visits were developed to collect information about sites; interview
guides were developed for use with the Health Start Program Coordinators and Lay
Health Workers. Site visits were made by ADHS staff to all Contractors to monitor
program implementation and contract compliance.
+ Procedures were developed and initiated to conduct Data Integrity Checks at each
site to compare data in case files with the Health Start data base for completeness and
accuracy.
+ The Health Start data collection forms were reviewed and revised to collect
additional information and resolve past data collection problems, with the new forms
being implemented in July, 1996.
+ Outcome indicators for the Health Start Program were reviewed, and various methods
of measurement explored. One aspect of this process included reviewing instruments
for possible use in the Health Start program. A questionnaire was developed by the
Auditor General's Office, in coordination with ADHS staff, for use by the Lay Health
Workers.
+ A Data lntegration Project was implemented to use existing data bases, maintained
by ADHS, to track Health Start clients and provide a foundation for the measurement of
outcome indicators. Several key data bases, that include Health Start Program
information, birth information from Vital Records and Newborn Intensive Care Program
( NICP) data, were used to construct a matched data set, consisting of Health Start
clients who had given birth during 1995 and a comparison group. Additional data bases
can be incorporated into the merged data set beginning in early 1997, which may
include the Women's Infants and Children Nutrition Program ( WIC) and potentially the
Hospital Discharge Data. Data from the Data lntegration Project was utilized to provide
descriptive information about Health Start clients and provide outcome measures.
+ Work was initiated to develop a Cost Benefit Model and initial analysis for the Health
Start Program. This aspect included examining existing data collected by Contractors,
ADHS, expenditure reports and outcome data.
Background
The Health Start Program was implemented by ADHS in 1992, designed to support the
increasing numbers of women receiving inadequate or no prenatal care, and to promote
primary health care for their children. It stressed activities to educate women on the benefits I of early prenatal care and assist them to obtain this care. It included activities to follow the
family for up to two years to assist women to obtain immunizations for their children. Primarily
a home visiting program, with services provided through Community Lay Health Workers,
Health Start's mission is to educate, support and advocate for families at risk by
I
promoting optimal use of community- based family health and education services to
reduce the incidence of low birth weight babies, increase prenatal services to pregnant
women and improve childhood health through a comprehensive multi- strategy
approach.
The present Health Start Program was created by the Arizona Children and Families
Stability Act, enacted during the 1994 9th Special Session ( Laws 1994, 9th SS., Ch. 1 $ 8).
This Health Start Pilot Program was built upon the previous Health Start program, but with an
expanded scope and extended length of the family follow- up period ( from two to four years
m
after birth). A range of services, delivered using a home visiting model, address the seven
primary program goals. I
rn Increase prenatal services to pregnant women
rn Reduce the incidence of low birth weight babies
Reduce the incidence of children affected by childhood diseases
rn Increase the number of children receiving age appropriate immunizations by two years
of age
I
rn Educate families in the importance of good nutritional habits to improve the overall
health of their children
rn Educate families on developmental assessments to promote the early identification of
I
learning disabilities, physical handicaps or behavioral health needs
Educate families on the benefits of preventive health care and need for screening
examinations such as hearing and vision
I
The services to be provided by the Health Start Program through lay workers, as outlined in I
ARS § 36- 697 include:
F Identify pregnant women in the lay health worker's neighborhood or community, and
enroll them in the program. I
F Inform clients of how to receive prenatal care services.
t Assist clients to access appropriate prenatal care.
t Educate clients on appropriate prenatal and neonatal care, preventive health care and
child wellness, including appropriate nutritional habits to improve the overall health of
their children.
t Assist and encourage clients to provide age appropriate immunizations so that their
children are fully immunized by two years of age.
t Assist and encourage clients and their families to access comprehensive public and
private preschool and other school readiness programs.
t Assist clients to apply for private and public financial assistance.
t Assist clients and their families to access other applicable community and public
services, including employment services.
t Provide clients with a list of local private, both nonprofit and for profit, and public
educational institutions and governmental agencies that provide program and referral
services ( Arizona Children and Families Resource Directory).
t Assist clients to access adult services including, continuing education, employment, and
other community involvement, such as religious or social services, as appropriate
Rationale for the Home - Visiting Approach
Health Start, along with other models of home visiting programs, brings family- centered
services to the home, using an individualized approach according to the needs of each family.
Because home visitors bring services to a family rather than requiring the family to come to an
agency office, home visiting programs can break down barriers to care and reach families who
otherwise might not receive services, as well as connecting families with existing services in
the community, such as medical care, employment or job training.
Evaluations of home visiting programs have assessed a variety of outcomes, depending upon
the goals of the particular home visiting programs studied. These outcomes have included
rates of low birth weight and pre- term births, children's motor or cognitive development,
utilization of health services, rates of child abuse and other benefits for mothers or
communities. '
The Health Start Program model was selected by ADHS to address the diverse needs of
Arizona's target population. The Program was designed to encourage the development of
h he Future of Children- Home Visiting: Analysis and Recommendations, Packard Foundation,
Vol 3 No. 3 Winter 1993 p. 10.)
community- based programs that could be responsive to the specific needs in the community,
while building capacity and community support for improved health of pregnant women and
their children. The home visiting model also provided the opportunity to address barriers to
health care that include:
Lack of Insurance coverage
Fear or distrust of existing health care provider agencies
Lack of education about health care
Lack of access to transportation
Language or Cultural barriers
Low literacy levels
Other Individual client barriers
Health Start Contractors
ADHS contracts with public and private agencies who provide Health Start Services. They
employ Lay Health Workers who focus on outreach to pregnant women in their communities
to enroll them in the program and provide health education, support, advocacy and
referrals to these women and their families. Twelve Health Start Contractors, comprised of
primarily County Health Departments and Community Health Centers, provide Health Start
services in 60 communities around the state. These communities include economically
disadvantaged urban neighborhoods in South Phoenix and Tucson, migrant farm worker
communities, Native American reservations and isolated rural communities.
Contractors have developed and manage a network of resources and referral sources that Lay
Health Workers utilize to serve the Health Start Clients. Within policies and guidelines
specified by ADHS, the Contractors utilize methods that are appropriate for the demographics
and particular characteristics of their community to achieve program standards and desired
outcomes. Within the framework of the Health Start Program is the flexibility for Contractors to
implement the program in a manner that " fits" their neighborhood or community.
Site visits have been made to each Contractor to monitor program implementation and
contract compliance. Topics reviewed included: staff and their recruitment, hiring, job
responsibilities and training process; policies being used at each location; service delivery and
documentation; quality assurance activities; and coordination of services. In general,
Contractors have done a good job in implementing the Health Start Pilot Program.
Comprehensive descriptions were completed for three sites: Mountain Park Health Center,
Pima County Health Department and Yuma County Health Department.
The scope of the information collected for the latter covered the following general areas:
identification and needs of communities being served; program history and evolution; program
participants; contractor/ program agency; staff and their recruitment, hiring and training
process; service delivery, coordination of services and quality assurance activities; program
barriers, challenges, achievements and changes; and community support.
Additional information was also obtained as to the process used for collection of cost data on
health care services and related programs. This information will be critical in providing a basis
for the cost benefit analysis for the program. Resources and Training materials were also
reviewed at each of the three sites. ADHS provides program development and support to
Contractors and purchases educational materials for distribution to program participants.
Characteristics of Health Start Participants
Information on Health Start Program Participants was obtained through two methodologies: 1)
the Health Start data base for clients served in FY 96 ( July 1, 1995- June 30, 1996), and 2)
the Data Integration Project via a data set consisting of Health Start clients who gave birth in
calender year 1995 ( January 1,1995 - December 31, 1995) matched with a control population
of women from Vital Statistics records.
Data from the Health Start Data base
Data can be extracted from the Health Start data base to cover three periods of time:
RegistrationIEnrollment; the Prenatal Period; and the PostnatalIFamily Follow- up Period.
-- Health Start Participants Who RegisteredIEnrolled in FY 96
The women who are initially contacted by the Lay Health Workers may be registered in the
Program, and basic demographic information obtained. These women may later enroll as
Health Start clients if they are pregnant and sign an informed consent form. During FY 96,
Health Start Contractors registered 1,837 women. Of these 1,837 women, 282 were tested
and found not to be pregnant ( 15%), 11 3 declined services ( 6%) and 1,442 ( 79%) were
enrolled in Health Start.
Approximately one third ( 32%) of the clients came in Health Start directly through the outreach
efforts of the Lay Health Workers, making these contacts the most common single source of
referrals to the Health Start program. Clinics and Public Health Nurses accounted for the
second largest source of clients ( 29%), followed by friends, relatives and other clients ( 19%),
self referrals ( 1 0%), and other sources ( 1 0%).
The majority of the new enrollees were Hispanic ( 71%); one fifth were American Indian. More
than half of the Health Start Participants were married or living with a partner ( 59%), 35% had
never married, and 6% were divorced, separated or widowed. Over 50% of the women were
between the ages of 20 and 29, with another 27.5% being below the age of 20. The majority
of participants did not have children ( 43%) or had one other child ( 41%).
One of the roles of the Lay Health Workers is to help their clients to utilize available resources,
as needed. Thus, an important part of their job is making referrals. Lay Health Workers
reported referring 1,168 of the new clients to additional services during the
registration/ enrollment process. An average of 2.4 referrals per participant were made ( range
from 1- 8), for a total of 2,820 referrals overall. The Lay Health Workers made three or more
referrals for almost 35% ( 491) of the new clients, 5% ( 73) needed 5 or more.
Referrals to the WIC program ( 59%) accounted for the majority of referrals reported, followed
by referrals to AHCCCS ( 46%) and to a clinic or hospital ( 37%). More than a third ( 38%) of
the women were not enrolled in AHCCCS at the time of registration into Health Start, as
compared to 62% who were enrolled. Mental health services ( 0.3%), followed by child care
services ( 1.3%), public health nurse ( 2.6%), and social worker services ( 3.3%) were those
utilized least by the Lay Health Workers. Potential reasons for the low utilization of social
workers and mental health services may include cultural biases or accessibility to services.
-- Health Start Participants Who Gave Birth in FY 96
Data available at the end of the prenatal part of the Program include: if the client had a baby,
information about the baby, AHCCCS status, referrals made throughout the prenatal period,
education provided to the client, and the number of times the client met with the Lay Health
Worker.
The Health Start program had 987 clients who delivered babies in FY 96. Of the 984 women
for whom there was data, 845 ( 86%) were enrolled in AHCCCS at the time of delivery. This is
in contrast to only 607 ( 62%) having been on AHCCCS at the time of their enrollment into
Health Start. Another 7% had private insurance. Of the 73 women who were not on AHCCCS
and did not have any insurance, 44% had applied for AHCCCS; 12 of them were waiting for
status notification and 20 of them had been denied.
Lay Health Workers provide health education to clients. The most frequently discussed topics
were Prenatal Care, Emotions and Feelings During Pregnancy, NutritionIDiet, Breastfeeding,
and High Risk Prenatal Conditions. This education provides clients with information which
helps the clients feel more comfortable with the pregnancy and alerts them to topics they could
discuss with their doctors.
A total of 3,020 referrals were recorded throughout the prenatal period, with a mean of 3.1
referrals per client. Approximately 12% of the women had no referrals reported, while over
51% had 1- 3 referrals and 37% had 4 or more. The most common referrals were to WIC,
followed by clinic, doctor or hospital, and then by Arizona Dept. of Economic Security ( DES).
One of the unique features of the Health Start program in relation to other programs serving
pregnant women is a focus on providing services to women in their own homes, i. e., making
home visits. The mean number of home visits recorded was 4.1 3 per participant. However,
the number of visits ranged from no home visits reported to 19 home visits. Of the 976 women
reporting home visits, 65% had 1- 4 home visits reported, while 35% had 5 or more reported.
While home visits are a focus of the Health Start program, they are not the only way that
services are delivered; the ability to use a combination of strategies can usually best serve
clients. The need for home visits varies by case, with teenagers, for example, often needing
more intense services than older women. In addition, not all women welcome Health Start
staff into their homes ( and some do not even have permanent homes), and, thus, may be
better served through office visits.
-- Health Start Participants in Family Follow- up in FY 96
After the baby is born, a client may choose to continue receiving visits from her Lay Health
Worker and enter the Family Follow- up portion of the Health Start program. Of the women
who had babies in FY 96, 94% chose to begin Family Follow- up. Family Follow- up visits were
predominantly home visits ( 83%).
The education provided by the Lay Health Workers to the new mothers covered a variety of
topics. The most frequently discussed were: Immunizations, Family Planning, Child Growth
and Development, Emotions and Feelings of the New Mother, and Infant Care. The Lay
Health Workers also provided referrals to services needed by the clients. The most common
referrals were to a Clinic, Doctor or Hospital, followed by referrals for immunizations or for
Family Planning, and then by referrals to WIC or AHCCCS.
-- Plans for FY 97
During FY 97 ( July 1996 - June 1997), the Health Start Program has expanded the case
documentation and data collection process, which will result in additional information being
collected about program participants, and available for analysis.
Data lnteuration Pro! ect
-- Background and Methodology
Recognizing that the Health Start Program needed to measure outcome indicators and the
extent of involvement of Health Start clients with other public programs in Arizona and that the
Program did not have the capacity to collect all of the necessary data, especially when clients
moved, several data bases managed by ADHS were examined for their potential value in
providing outcome information and measurements for the Health Start Program. The required
information already existed, but accessing this data required matching the Health Start data to
other data files, namely the Vital Statistics files and the Newborn Intensive Care Program
( NICP) and the WIC files. ADHS authorized the merging of data from Vital Statistics, and data
bases maintained for WIC and NICP, with the Health Start data base to set a foundation for
tracking program participants in the various data sets and for the creation of comparison
groups of women giving birth who did not receive the Health Start Services. A Data
Integration Project was implemented to use these existing data bases to track Health
Start clients and provide descriptive information and outcome measures.
For this report, a data set was matched with Vital Records data using Health Start clients who
delivered babies during the period January I, 1995 - December 31, 1995, in order to coincide
with the Vital Records Data Base. The matched data set was used to derive descriptive
characteristics and look at birth outcomes for Health Start. A control was selected for each
participant, and the groups were compared. Variables used for matching included:
Mother's Age, Race, Ethnicity, Marital Status and Education. Additional variables that can be
examined using this combination of data include: Gestation Period, Alcohol Use, Tobacco Use
and Responsible Party.
Additional data bases can be incorporated into the merged data set beginning in early 1997.
Types of analysis that could be conducted for Health Start to look at outcome indicators were
identified. Program differences among the sites were also important to consider to adequately
interpret data. Using this methodology, it will be feasible to analyze outcomes by contractor.
Based on the preliminary analyses for the statewide program, the following trends were
evident.
b Health Start Participants were more likely to receive Prenatal Care earlier ( 62.7%)
in the first trimester, as compared to the comparison group of non participants
( 57.0%).
Health Start participants were less likely to have pre- term babies ( 5.9% as
compared to 7.7% of the comparison group). The Gestation Period of 37 weeks
or more for Health Start participants was 93.7%, as compared to the comparison
group ( 91.6%) .
+ Women enrolled in Health Start were less likely ( 66.7%) to be on AHCCCS, as
compared to the comparison group ( 73.9%), but also less likely to be uninsured
( 3.0%), as compared to the comparison group ( 5.7%).
Health Start participants were more likely to have a normal birth weight baby
( 95%), as compared to the control group ( 93%).
Cost Benefit Analysis
The Health Start program seeks a number of objectives. The data that are currently
available limit the evaluation of benefits to the objective of reducing the incidence of
low birth weight babies. Thus, the benefit calculations do not include the benefits derived
from the immunization of children, nor of the activities occurring during the family follow up
portion of the program. Neither do they include the benefits of services provided to mothers
who participate in the Health Start program during pregnancies that did not come to term
during calender year 1995. The estimates of benefits that are described are, therefore,
lower than the actual benefits produced by Health Start's activities.
Selection of a Control Group
The effectiveness or lack of effectiveness of Health Start's activities in reducing the incidence
of Very Low Birth Weight ( VLBW) or Low Birth Weight ( LBW) infants can only be measured by
comparison to the birth outcomes of women who have the same risk profiles as the Health
Start participants but who were not served by Health Start. Thus, in the simplest terms, the
benefit of Health Start activities directed toward the improvement of birth outcomes can
be represented as the difference in birth outcomes between Health Start participants
and the non- participants or " control cases".
There are 1,306 Health Start participants with 1,306 matched controls in the data set used for
this analysis. The characteristics of the control cases and the participants with whom they are
matched are described in Table # 1.
The results of the match show that exact matches were obtained on age and ethnicity, and
very nearly exact matches were obtained on race, marital status and education. There are, for
example, nine more women who are African American in the Health Start group than in the
control group. Since African American women are generally at a higher risk for LBW or VLBW
babies than women of other races, the difference implies that at least nine of the women in the
Health Start group are at higher risk for LBW or VLBW babies than are the women with whom
they are compared. Thus, the benefits attributable to Health Start are likely to be
understated by the results for this group.
The Health Start participants are very slightly better educated than the controls in that four
more women completed high school than among the controls and three more women
completed college. Since better educated women are generally more likely to have better
birth outcomes than less well educated women ( all else being equal), this difference could
overstate the benefits of Health Start participation in the results.
Birth Weiahts and Participation in Health Start
The birth weights of children born to mothers participating in the Health Start program,
compared to the birth weights of the matched control cases, are included in Table # 1.
In total, four more children with very low birth weights (< I500 grams) were born to controls
than to Health Start participants. Twenty- six more children with low birth weights ( 1501- 2500
grams) were born to control group mothers than to Health Start mothers. The results are
consistent with the hypothesis that participation in Health Start reduced the incidence
of very low birth weight and low birth weight babies among infants born during
calender year 1995.
The criteria used to match the cases does not, however, guarantee that the differences
between the characteristics of the Health Start participants and the control cases that could
influence the outcomes have successfully been eliminated. In addition, variations among
Contractors in their implementation of the Program or in the adequacy of selection criteria for
the control group to account for differences in risk among the populations served by the
Contractor may affect the quality of the match. These factors indicate a need to more carefully
analyze the differences among Contractors in the characteristics of their clients and in the
nature and quantity of the services that were provided to the clients. It is also important to
recognize that identification of differences, if any are found, in the effectiveness of different
approaches to the minimization of undesirable birth outcomes would be an very important
contribution of the Health Start evaluation.
Table # 1
Characteristics of Health Start Participants and Controls with Births in 1995
*( M) indicates characteristic used to match.
Potential Savinas from Health Start
The savings that are reasonably attributable to the differences in the average costs of care
between normal and LBW or VLBW babies at the time of birth may be estimated. It is
important to note that the data on the health care costs of VLBW and LBW babies is lower
than true costs because it omits the costs of hospital care for mothers with VLBW or LBW
children and the costs of care for children who died. The data do not include health care costs
for follow- up care after birth, except for a one year estimate for hospital charges for the first
year after birth for the VLBW children.
The benefits of Health Start can be estimated as the savings that are obtained from the
prevention of the need for health care for VLBW or LBW children. The direct costs of care
for VLBW children are obtained from the estimate in the 1995- 1 996 report by ADHS that
includes the costs of NlCU care, physician time in NlCU and hospital charges for the first year
after a child is born. The estimate from the report is that the cost of health care for an average
VLBW infant was approximately $ 123,000 in 1991. Assuming that the costs have increased at
the rate of the medical care component of the Consumer Price Index ( urban consumers), the
equivalent in terms of 1995 dollars is approximately $ 1 53,000.
The measure of potential savings due to the reduction of the incidence of LBW births in the
Health Start population is the difference between the average hospital charges for LBW and
normal births for all births in Arizona that were recorded in the hospital discharge data base.
The average hospital charges for normal infants were $ 1 , I 00 and the average charges for
LBW infants were $ 7,700. Thus, the savings from the prevention of a LBW baby is an average
of $ 6,600.
Total amounts paid to contractors by the State under the provisions of their contracts include,
the costs of . all services to Health Start clients rather than the costs of services designed to
improve birth outcomes for the clients who bore children in calender year 1995. These costs
substantially overstate the costs of the services that could have produced the benefits
that we measure for 1995. This is enhanced by the omission of the value of the other
outcomes, such as immunizations, that could be produced by Health Start.
Recognizing that the measured benefits do not include costs of maternal care or the future
costs of care for VLBW or LBW children and that charges overstate actual costs, the estimates
imply that the lower bounds of the benefits of the birth weight portion of the Health Start
program range from approximately $ 790,000 to $ 1.4 million.
Conclusion
Overall the performance and outcome indicators for the Health Start appear very
promising. As the program is in the initial stages of development, additional work must be
done to insure reliability of measures and accuracy of data before conclusions can be drawn.
For example, more information as to the characteristics of clients served ( i. e. medical
conditions, substance abuse, etc.) and types and duration of services provided to Health Start
participants must be available for analysis. However, in view of the modifications made to the
Health Start data collection forms and the ability to interface with other data bases, this task
appears more promising for the future. I
In terms of cost benefit analysis, examples of the types of information on the costs and
benefits of Health Start activities that could be used to evaluate the performance of individual
contractors and the program as a whole are being explored. The comparisons that have been
I
presented are explicitly oversimplified with some obvious bias towards overstating the cost
segments of the calculations. These procedures were adopted to be conservative in the
appraisal and to substitute obvious, understandable biases for more subtle problems that
I
could not be resolved from the current data. I
The results, although simplistic, provide some information that has been unavailable in
previous evaluations of the Health Start program. Chief among these is the introduction of
carefully selected control group experimental group comparisons that permit the
definition of the contributions of Health Start to the reduction in the incidence of VLBW
and LBW children. I