Performance audit, department of health services division of behavioral health services substance abuse treatment programs: report to the Arizona Legislature |
Previous | 1 of 4 | Next |
|
This page
All
Subset |
Debra K. Davenport
Auditor General
Performance Audit
Department of
Health Services
Division of Behavioral Health Services—
Substance Abuse Treatment Programs
Performance Audit Division
July • 2009
REPORT NO. 09-07
A REPORT
TO THE
ARIZONA LEGISLATURE
The is appointed by the Joint Legislative Audit Committee, a bipartisan committee composed of five senators
and five representatives. Her mission is to provide independent and impartial information and specific recommendations to
improve the operations of state and local government entities. To this end, she provides financial audits and accounting services
to the State and political subdivisions, investigates possible misuse of public monies, and conducts performance audits of
school districts, state agencies, and the programs they administer.
The Joint Legislative Audit Committee
Audit Staff
Copies of the Auditor General’s reports are free.
You may request them by contacting us at:
Office of the Auditor General
2910 N. 44th Street, Suite 410 • Phoenix, AZ 85018 • (602) 553-0333
Additionally, many of our reports can be found in electronic format at:
www.azauditor.gov
Melanie M. Chesney, Director
Shan Hays, Manager and Contact Person
Anne Hunter, Team Leader
Kerry Howell
Karl Kulick
Senator Thayer Verschoor, Chair Representative Judy Burges, Vice-Chair
Senator Pamela Gorman Representative Tom Boone
Senator John Huppenthal Representative Cloves Campbell, Jr.
Senator Richard Miranda Representative Rich Crandall
Senator Rebecca Rios Representative Kyrsten Sinema
Senator Bob Burns (ex-officio) Representative Kirk Adams (ex-officio)
2910 NORTH 44th STREET • SUITE 410 • PHOENIX, ARIZONA 85018 • (602) 553-0333 • FAX (602) 553-0051
WILLIAM THOMSON
DEPUTY AUDITOR GENERAL
DEBRA K. DAVENPORT, CPA
AUDITOR GENERAL
STATE OF ARIZONA
OFFICE OF THE
AUDITOR GENERAL
July 30, 2009
Members of the Arizona Legislature
The Honorable Janice K. Brewer, Governor
Will Humble, Interim Director
Department of Health Services
Transmitted herewith is a report of the Auditor General, a Performance Audit of the
Department of Health Services, Division of Behavioral Health Services—Substance Abuse
Treatment Programs. This report is in response to an October 5, 2006, resolution of the
Joint Legislative Audit Committee. The performance audit was conducted as part of the
sunset review process prescribed in Arizona Revised Statutes §41-2951 et seq. I am also
transmitting with this report a copy of the Report Highlights for this audit to provide a quick
summary for your convenience.
As outlined in its response, the Department of Health Services agrees with all of the
findings and plans to implement all of the recommendations.
My staff and I will be pleased to discuss or clarify items in the report.
This report will be released to the public on July 31, 2009.
Sincerely,
Debbie Davenport
Auditor General
Attachment
The Office of the Auditor General has conducted a performance audit of the
substance abuse treatment programs provided by the Department of Health
Services (Department), Division of Behavioral Health Services (Division), pursuant to
an October 5, 2006, resolution of the Joint Legislative Audit Committee. This is the
second audit in a series of three reports on the Department and was conducted as
part of the sunset review process prescribed in Arizona Revised Statutes (A.R.S.)
§41-2951 et seq. This audit focuses on substance abuse treatment outcomes and
system oversight. The first audit focused on the Division of Licensing Services, and
the final report will be an analysis of the 12 statutory sunset factors.
According to its Annual Report on Substance Abuse Treatment Programs, the Division
spent more than $121 million for substance abuse services in fiscal year 2008.
Program participants, whom the Division refers to as “consumers,” numbered more
than 63,000 adults and children. Most were enrolled in the Arizona Health Care Cost
Containment System, or AHCCCS, the State’s Medicaid program. They received
alcohol- and drug-related services ranging from counseling and skills training to
crisis intervention and detoxification in hospitals or other inpatient facilities. The
Division provides these services through contracts with four regional behavioral
health authorities, or RBHAs, and three tribal regional behavioral health authorities, or
TRBHAs, which contract with a network of more than 100 substance abuse treatment
service providers throughout the State.
This performance audit focused on the program’s outcomes—that is, the extent to
which services reduced dependency on alcohol and drugs—and on the Division’s
oversight of behavioral health authorities and providers.
Division should focus on strategies that improve
outcomes (see pages 9 through 29)
Although substance abuse is difficult to treat, the Division can take steps to improve
outcomes for individuals who participate in substance abuse treatment. Auditors
analyzed 3 years of data related to four measures commonly used to evaluate
program effectiveness—extent of continuing alcohol or drug use, employment,
criminal activity, and homelessness. The analysis showed that outcomes related to
continued alcohol and drug use were associated with the following two factors:
Office of the Auditor General
SUMMARY
page i
Deciding to abstain from using alcohol or drugs before treatment started—
More than half of all consumers reported that they were abstinent when their
treatment started. Within this group, more than 93 percent reported that they
were still abstinent when they left the program. By contrast, most of those who
reported using alcohol or drugs when they began treatment were still using
these substances at about the same level when they left. About one person in
every four who began treatment while still using alcohol or drugs reported
diminishing his/her use of alcohol or drugs or stopping it altogether by the time
he/she left treatment.
Completing treatment—Overall, 58 percent of consumers did not complete their
treatment. Providers lost contact with many of them, while others refused
treatment or left for other reasons, but rates of continued use of alcohol or drugs
varied substantially between those who completed the treatment and those who
did not. For example, among consumers who reported using alcohol or drugs
when they began treatment, 27 percent of those who completed their treatment
reported abstinence when they left. By contrast, among consumers who
reported using alcohol or drugs when they began treatment, only 17.6 percent
of those consumers who left before completing their treatment reported that
their use had diminished.
The analysis showed little change across the three remaining performance
measures—lack of recent arrests, employment, and stable housing. For example, 21
percent of consumers reported recent arrests upon entering treatment, and 18
percent reported new arrests at the time of their update or disenrollment. Similarly, 38
percent said they were employed when they entered treatment, while 41 percent
reported being employed at their annual update or disenrollment. Finally, slightly
more than 7 percent were homeless upon entering treatment, and slightly less than
7 percent were homeless at their annual update or disenrollment.
Substance abuse is difficult to treat, and auditors’ more detailed case studies of a
limited number of consumers showed that the reasons for success or failure are
complicated and varied. Nonetheless, research and best practices indicate the best
opportunities for increasing success rest in three main strategies:
Focusing on treatment retention—Research corroborates what auditors’
analysis showed: consumers who remain longer in treatment experience better
outcomes. The Division can take several steps to increase consumer retention,
including establishing performance goals, monitoring completion rates, and
using incentives and other case management techniques. Other states that
auditors reviewed have taken such steps, and the Division may be able to adopt
some of these approaches.
Ensuring that consumers have access to a full range of services that can
potentially be used to address their particular needs—This strategy, called
continuum of care, involves incorporating appropriate types of treatment over
State of Arizona
page ii
time and placing the consumer in more or less intensive treatment as needed.
Auditors’ case studies showed that while some consumers showed good
outcomes and received appropriate services, others did not necessarily receive
the services and therapies that might improve the chances of good treatment
outcomes. The Division can take several steps to ensure continuum of care,
including collecting and monitoring data relevant to assessment, better defining
case management, and working with RBHAs to improve the continuum of care
when weaknesses are identified.
Following practices that have been shown to carry the greatest chance of
success—These evidence-based practices have been validated by observation
or experience as improving treatment success. Examples include motivational
interviews, which is a counseling style designed to help consumers recognize
and accept the need for continued care. Although the Division requires the
RBHAs to use evidence-based practices, RBHAs are not necessarily doing so,
and the Division is not ensuring compliance. According to a 2008 federal grant
review that focused on programs for children and adolescents, the Division had
done a good job of establishing evidence-based practices in some areas but
had not identified such practices across the continuum of care and could do
more to ensure sustainability of the emphasis on evidence-based practices. The
Division reached similar conclusions in another study of intensive outpatient
programs for youth. Steps the Division can take to place greater emphasis on
evidence-based practices include encouraging RBHAs to offer a wider variety of
programs, monitoring compliance with its contractual requirements to use
evidence-based practices, and expanding its work with the RBHAs to ensure
that providers have the guidance needed to implement specific evidence-based
practices.
Division should improve oversight of substance abuse
programs (see pages 31 through 42)
The Division should take steps to improve its oversight of the substance abuse
programs administered by RBHAs. These steps take two main forms:
Increasing the use of information about treatment outcomes—Although the
Division collects outcome information to complete certain reports, auditors
found that oversight efforts focused almost entirely on process-related
information, such as the timeliness of services or coordination with a
consumer’s primary care physician as required by the Division’s contract with
AHCCCS. As a result, the Division is largely unable to determine if its substance
abuse treatment programs are achieving positive results or if its resources are
being used effectively. Additionally, because the Division does not compare
Office of the Auditor General
page iii
substance abuse outcome measures across RBHAs or providers, it cannot
assess which providers’ treatment services are resulting in improved client
outcomes or identify underperforming providers. Needed actions include
continuing its efforts to streamline uniform outcome data collection, establishing
relevant performance goals in contracts with the RBHAs, and encouraging the
RBHAs to consider ways to reward providers who meet standards and penalize
those who do not.
Expanding utilization reviews to focus more on service costs, consumer
assessments, and case management—Although its oversight efforts contain
many elements that could potentially help manage costs, the Division could
implement several actions that could improve its ability to do so. Greater
emphasis on cost appears warranted. Auditors’ review of division data from
fiscal years 2006 to 2008, for example, identified 14 substance abuse
consumers with service costs over $100,000. One incurred $82,000 in medical
detoxification costs, during which time he continued to drink and require
detoxification three or four times a month. The Division was not aware of these
cases until auditors brought them to officials’ attention. Actions needed include
(1) regularly reviewing high- and low-cost substance abuse treatment cases, (2)
collecting data to identify consumers who may be overutilizing or underutilizing
certain types of services, which could indicate a lack of alternative forms of
treatment or a need for other changes to improve treatment, (3) comparing
variations in the use of types of treatment at each RBHA to see if the use of such
services positively affects consumer treatment outcomes and adjusting
treatment accordingly, and (4) determining how to best use assessment and
case management to contain costs while maintaining quality of care. Further, to
improve oversight, the Division should continue its efforts to fill vacant positions
in its data systems and analysis and quality management functions, and should
perform follow-up work to ensure that the restructuring it initiated in April 2009
has provided management with the information to do so.
State of Arizona
page iv
Office of the Auditor General
TABLE OF CONTENTS
continued
page v
Introduction & Background 1
Finding 1: Division should focus on strategies that improve
outcomes 9
Substance abuse difficult to treat 9
Most consumers showed little change after treatment 10
Division should increase focus on treatment retention and completion 17
Division should monitor continuum of care 21
Using appropriate evidence-based practices can improve success rate 25
Recommendations 28
Finding 2: Division should improve oversight of substance
abuse programs 31
Division should monitor outcomes 31
Division should monitor monies spent on treatment 35
Recommendations 41
Appendix A: Outcome analysis methodology and results a-i
Appendix B: Methodology b-i
Appendix C: Bibliography c-i
Agency Response
TABLE OF CONTENTS
continued
page vi
State of Arizona
Table:
1 Funding Sources for Substance Abuse Services Expenditures
Fiscal Year 2008
(Unaudited) 4
Figures:
1 Regional and Tribal Behavioral Health Authorities’
Geographical Service Areas (GSA) and
Fiscal Year 2008 Substance Abuse Enrollment 3
2 State-wide Substance Abuse Treatment Enrollment
Fiscal Years 2001 through 2008 6
3 Treatment Completion and Noncompletion Rates by RBHA (GSA)
Fiscal Years 2006 through 2008 19
4 Consumer-Reported Abstinence from Substance Use
By RBHA (GSA) from Admission to Annual Review or Discharge
Fiscal Years 2006 through 2008 a-iv
5 Change in Substance Use Among Consumers
Who Reported Substance Use Before Treatment by RBHA (GSA)
From Admission to Annual Review or Discharge
Fiscal Years 2006 through 2008 a-v
6 Alcohol Abstinence Status of Consumers by State (Western Region)
Fiscal Year 2009 SAPT Block Grant Application
(Unaudited) a-vi
7 Drug Abstinence Status of Consumers by State (Western Region)
Fiscal Year 2009 SAPT Block Grant Application
(Unaudited) a-vii
Office of the Auditor General
TABLE OF CONTENTS
continued
page vii
Figures (continued):
8 Alcohol Abstinence Status of Consumers
Arizona, Western Region, and National Averages
Fiscal Year 2009 SAPT Block Grant Application
(Unaudited) a-viii
9 Drug Abstinence Status of Consumers
Arizona, Western Region, and National Averages
Fiscal Years 2009 SAPT Block Grant Application
(Unaudited) a-ix
10 Employment Status of Consumers by RBHA (GSA)
From Admission to Annual Review or Discharge
Fiscal Years 2006 through 2008 a-xi
11 Employment Status of Consumers by State (Western Region)
Fiscal Year 2009 SAPT Block Grant Application
(Unaudited) a-xii
12 Employment Status of Consumers
Arizona, Western Region, and National Averages
Fiscal Year 2009 SAPT Block Grant Application
(Unaudited) a-xiii
13 Arrest-Free Status of Consumers by RBHA (GSA)
From Admission to Annual Review or Discharge
Fiscal Years 2006 through 2008 a-xv
14 Arrest-Free Status of Consumers by State (Western Region)
Fiscal Year 2009 SAPT Block Grant Application
(Unaudited) a-xvi
15 Arrest-Free Status of Consumers
Arizona, Western Region, and National Averages
Fiscal Year 2009 SAPT Block Grant Application
(Unaudited) a-xvii
State of Arizona
TABLE OF CONTENTS
concluded
page viii
Figures (concluded):
16 Homeless Status of Consumers by RBHA (GSA)
From Admission to Annual Review or Discharge
Fiscal Years 2006 through 2008 a-xix
17 Homeless Status of Consumers by State (Western Region)
Fiscal Year 2009 SAPT Block Grant Application
(Unaudited) a-xx
18 Homeless Status of Consumers
Arizona, Western Region, and National Averages
Fiscal Year 2009 SAPT Block Grant Application
(Unaudited) a-xxi
The Office of the Auditor General has conducted a performance audit of the
substance abuse treatment programs provided by the Department of Health
Services (Department), Division of Behavioral Health Services (Division), pursuant to
an October 5, 2006, resolution of the Joint Legislative Audit Committee. This is the
second audit in a series of three reports on the Department and was conducted as
part of the sunset review process prescribed in Arizona Revised Statutes (A.R.S.)
§41-2951 et seq. This audit focuses on substance abuse treatment outcomes and
system oversight. The first audit focused on the Division of Licensing Services, and
the final report will be an analysis of the 12 statutory sunset factors.
Importance of treating substance abuse
Drug and alcohol abuse are associated with some of society’s most serious and
expensive problems. For example, according to literature, nation-wide:
More than half of all state prison inmates were under the influence of alcohol
or drugs when they were arrested.1
Nearly one in six state inmates committed crimes to support a drug addiction.2
About 20 percent of acute care Medicaid expenditures pay for alcohol- or
drug-related medical costs.3
Drunk driving is a major expense for the police, courts, and emergency
medical systems.4
1 U.S. Department of Justice, Bureau of Justice Statistics, 2005a as cited in Rosenbloom et al., 2006
2 U.S. Department of Justice, Bureau of Justice Statistics, 2005b as cited in Rosenbloom et al., 2006
3 U.S. Department of Health and Human Services, Substance Abuse and Mental Health Services Administration, 2005 as
cited in Rosenbloom et al., 2006
4 Miller, Cox, Zaloshnja, & Taylor, 2002
Office of the Auditor General
INTRODUCTION
& BACKGROUND
page 1
In 2003, an estimated 22.5 million people ages
12 and older nation-wide, or about 9.4 percent
of the population, had a substance abuse
disorder, and an estimated $21 billion was
devoted to substance abuse treatment,
according to a U.S. Department of Health and
Human Services’ Substance Abuse and
Mental Health Services Administration
(SAMHSA) report dated 2007.1 According to
the same report, people with substance use
disorders rely on public sources of financing
far more than do people with other diseases.
The report also states that public sources of
funding, including state and local government,
Medicaid, and other federal spending, such as
block grants, provided 77 percent of total
substance abuse spending in 2003 but only 45
percent of all healthcare spending.
The nature of addiction, such as its quality as
a chronic and recurring problem, and other
factors, such as age and socio-economic
status, affect treatment success. Arizona, like
many other states, struggles with the problem
of substance abuse (see textbox).
Structure and funding of Arizona’s system for providing
substance abuse services
The Division provides substance abuse services mainly to consumers enrolled in the
Arizona Health Care Cost Containment System, or AHCCCS, which is the State’s
Medicaid program. However, others can receive services to the extent funding is
available, particularly if they are members of priority populations such as pregnant
women. AHCCCS enrollees include adults who meet the requirements of Title XIX
(Medicaid) of the Social Security Act (Act), as well as children and their families who
meet the requirements of Title XXI (the federal SCHIP program, which is called
KidsCare in Arizona) of the Act.
Publicly funded substance abuse treatment and prevention services in Arizona are
provided primarily through the behavioral health system administered by the
Division.2 The Division provides these services through contracts with four regional
behavioral health authorities, or RBHAs, and three tribal regional behavioral health
1 Mark et al., 2007
2 Incarcerated individuals may receive publicly funded substance abuse services through the Department of Corrections
or the county jails.
State of Arizona
page 2
Prevalence and impacts of substance abuse in Arizona
Altogether, an estimated 466,000 Arizonans ages 12 and older were
dependent on or abused alcohol, and an estimated 137,000 were
dependent on or abused illicit drugs, according to the 2005 National
Survey on Drug Use and Health.
Drug- and alcohol-related deaths, including those attributable to
motor vehicle and boating fatalities, accounted for nearly 1,700
deaths in Arizona in 2005.
There were significant increases in the number of alcohol and drug-induced
deaths in Arizona between 2000 and 2005. The rates of
drug-induced deaths more than doubled between 2000 and 2005,
and the number of alcohol-induced deaths increased by more than
one-third for the same time period.
A 2008 study of adults arrested in Maricopa County in 2007 found
that over one-third of arrestees interviewed for the study tested
positive for methamphetamine use, with over 40 percent of female
arrestees testing positive.
Source: The Substance Abuse Epidemiology Work Group’s 2007 Arizona Statewide Substance Abuse
Epidemiology Profile, and Arizona State University, Center for Violence Prevention and
Community Safety’s Arizona Arrestee Reporting Information Network Annual Adult Report
2007.
authorities, or TRBHAs, which contract
with a network of more than 100
substance abuse treatment service
providers within six geographical service
areas, or GSAs, throughout the State (see
Figure 1).1 During fiscal year 2008, over
63,000 adults and children participated in
substance abuse treatment programs
offered through the RBHAs and TRBHAs
and their providers throughout the State.
Magellan, the RBHA that serves Maricopa
County, served the largest proportion—38
percent—of individuals receiving
substance abuse treatment in fiscal year
2008, followed by Community Partnership
of Southern Arizona (CPSA) at 32 percent,
Northern Arizona Behavioral Health
Authority (NARBHA) at 15 percent, and
Cenpatico Behavioral Health of Arizona,
the three tribal regional behavioral health
authorities, and the Navajo Nation serving
the remaining approximately 15 percent.
The Navajo Nation serves approximately 2
percent of individuals receiving substance
abuse treatment.
According to the Division’s Annual Report
on Substance Abuse Treatment Programs,
in fiscal year 2008 approximately $121.2
million, or approximately 11 percent of the
Division’s estimated $1.1 billion budget,
was expended for substance abuse
services.2 The majority of this funding
came from Title XIX/XXI monies, with
additional funds provided through the
federal Substance Abuse Prevention and Treatment (SAPT) block grant, state
appropriations, and other funding sources (see Table 1, page 4).
Budget reductions in fiscal year 2009 will probably affect substance abuse treatment.
In addition, as of July 16, 2009, the Department’s 2010 budget had not been
determined. According to the Division, a total of $2.5 million of appropriated funds
that are used for substance abuse services was cut from its fiscal year 2009 budget.
Office of the Auditor General
page 3
1 Magellan replaced ValueOptions as the RBHA for Maricopa County on
September 1, 2007.
Source: Auditor General staff analysis of the Arizona Department of Health Services,
Division of Behavioral Health Services’ Annual Report on Substance Abuse
Treatment Programs for fiscal year 2008 and division-provided enrollment
data.
Figure 1: Regional and Tribal Behavioral Health Authorities’
Geographical Service Areas (GSA) and
Fiscal Year 2008 Substance Abuse Enrollment
Northern Arizona Regional Behavioral Health Authority
Cenpatico Behavioral Health of Arizona
Magellan1
Cenpatico Behavioral Health of Arizona
Community Partnership of Southern Arizona
Community Partnership of Southern Arizona
Tribal Regional Behavioral Health Authorities
1 In addition to its three TRBHA contracts, the Division also contracts with two tribal nations—the Navajo Nation and the
Colorado River Indian Tribes—to deliver substance abuse treatment services to reservation residents.
2 The total amount of funding allocated to the RBHAs in fiscal year 2008 was $989.4 million, including monies for serving
children, adults with serious mental illness, and people with co-occurring mental health and substance abuse disorders,
as well as providing general mental health and substance abuse prevention and treatment services.
In addition, the Division reported fund
sweeps and appropriations reductions
totaling almost $1.1 million in substance
abuse fund monies. These monies
come mainly from fees and fines
collected from criminal offenses and
are used to provide alcohol and drug
screening, education, and treatment for
individuals who are court-ordered to
attend, but do not have sufficient ability
to pay. Because of these reductions,
the Division expects that the number of
consumers served in fiscal year 2009
will be less than the number served in
fiscal year 2008. In addition, to offset
some of the budget reductions, the
Division reported that it is updating its
policy to require co-payments for
consumers who are not eligible for Title
XIX and whose income is a certain
percentage above the federal poverty
level. On April 3, 2009, a draft version of
the policy regarding co-payments was
released for public comments, and as
of July 16, 2009, the policy had not
been finalized.1
Funding is allocated to the RBHAs through a capitated payment arrangement
whereby the RBHA is provided a set amount of funding each month for each
consumer in its geographic service area who is enrolled in AHCCCS. In order to
allow the Division to monitor services and determine funding needs, the RBHAs
submit service data called encounters (see textbox) to the Division. The Division’s
contracts with the RBHAs require them to submit encounters whose total value
equals at least 85 percent of the total service revenue payments they receive on an
annual basis.
Substance abuse services provided
The Division provides many different types of substance abuse services. In fiscal
years 2006 through 2008, the RBHAs reported that they provided services valued at
$139.3 million to substance abuse treatment consumers, excluding substance
abuse services to other consumers such as adults with serious mental illness,
children, and adolescents. The largest service category for those consumers,
1 Specific changes to the policy include: 1) consumers with serious mental illness are no longer excluded from the co-payment
requirement, 2) the types of services excluded from the co-payment requirement were modified, and 3)
providers are allowed to refuse service as well as terminate services to a consumer for non-payment of a co-payment.
State of Arizona
page 4
Encounter—Record of a
service delivered to a consumer
by a provider. Includes specifics
about the type and date of service,
provider that delivered the service,
and dollar value of the service.
Funding Source Amount Percentage
Title XIX/XXI funding $ 75,941,357 62.66%
Substance Abuse Prevention and
Treatment Block Grant 23,545,206 19.43
State appropriations 16,518,610 13.63
Intergovernmental agreements
(IGA) and interagency service
agreement (ISA)1 5,141,081 4.24
Liquor fees2 45,325 0.04
Total $121,191,579 100.00%
Table 1: Funding Sources for Substance Abuse Services Expenditures
Fiscal Year 2008
(Unaudited)
1 In fiscal year 2008 the Division had IGAs with Maricopa County and the City of Phoenix, and
an ISA with the Arizona Department of Corrections, Correctional Officer/Offender Liaison
program (the COOL program). Maricopa County and the City of Phoenix provide monies to
help operate the Local Alcohol Rehabilitation Center, a detoxification center in Phoenix. The
COOL program was eliminated by the Department of Corrections in 2009. However, in 2008
the program provided transitional services for persons with substance abuse issues who were
leaving the penal system. The services included assistance in finding housing, employment,
and behavioral health services.
2 Liquor fees are fees collected from the issuance of special event liquor licenses.
Source: Auditor General staff analysis of the Arizona Department of Health Services, Division of
Behavioral Health Services’ Annual Report on Substance Abuse Treatment Programs for
fiscal year 2008.
treatment services, accounted for approximately 31.5 percent of the total.1 The types
of services were as follows:
Treatment services ($43.9 million)—Individual and group counseling, therapy,
assessment, evaluation, screening, and other professional services.
Residential services ($26.9 million)—Twenty-four-hour residential services,
including structured treatment, which includes room and board, delivered in
residential facilities or supported independent living settings.
Support services ($25.3 million)—Case management, peer support services,
and transportation.
Medical and pharmacy services ($19.0 million)—Medication to assist with
alcohol or drug withdrawal or with co-occurring disorders.
Inpatient ($9.5 million)—Inpatient detoxification and treatment services delivered
in hospitals and other inpatient facilities, including residential treatment centers
that provide a structured treatment with 24-hour supervision, an intensive
treatment program, and on-site medical services.
Crisis intervention services ($5.7 million)—Crisis intervention/stabilization
services provided in the community, hospitals, and residential treatment centers.
Rehabilitation services ($4.9 million)—Education, coaching, training, and other
services, including securing and maintaining employment. Services include
living skills training, cognitive rehabilitation, health promotion, and ongoing
support to help maintain employment.
Day programs ($4.1 million)—Skills training and development, behavioral health
prevention/promotion, medication training and support, ongoing support to
maintain employment, and self-help/peer services to improve consumers’ ability
to function in the community.
The cost of services varies by type of treatment. For example, auditors’ analysis of
encounter data from fiscal years 2006 through 2008 revealed that the average cost
per person, per year, for residential treatment was $1,500 and the average cost of
crisis intervention per person, per year, was $240. Because some consumers may
receive several different types of treatment or receive treatment for longer periods or
at several different times during the year, the total cost per person for all services
received varies widely, with a few consumers receiving services valued at more than
Office of the Auditor General
page 5
1 The figures reported in these bullets reflect auditors’ analysis of services reported for consumers whose primary reason
for enrollment was substance abuse. These figures differ from the total substance abuse funding of $121 million shown
in Table 1 (see page 4) because these figures cover a 3-year period and do not include substance abuse services
provided to other consumers. Further, because the Division uses a capitated system to pay RBHAs in advance for
providing all required services, current-year costs are not reflected in current-year payments as they would be in a fee-for-
service system. Therefore, the total substance abuse funding is not the same amount as the value of the services
provided as reported by the RBHAs for a given time period.
$100,000 per year (See Finding 2, pages 31 through 42, for additional information on
high-utilization consumers). The median cost of treatment per person for fiscal years
2006 through 2008 was $1,090.1
Consumers served
Generally, to obtain substance abuse services through the RBHAs or TRBHAs a
person must be enrolled in AHCCCS. According to the Division’s Annual Report on
Substance Abuse Treatment Programs, in fiscal year 2008, 74 percent of adults and
children receiving substance abuse services, or approximately 47,000 individuals,
were AHCCCS eligible. Substance abuse services for non-Title XIX/XXI-eligible
individuals and families, also delivered by RBHA- and TRBHA-contracted providers,
are provided only if monies are available, and these services are provided first to
people in priority populations. The priority populations are spelled out by grant
requirements and other funding sources. For example, the federal SAPT block grant
designates priority populations as pregnant women, women with dependent
children, and intravenous drug users. In fiscal year 2008, 1,139 pregnant women and
6,238 women with children received substance abuse services. In addition, in fiscal
year 2008, 4,033 people were tested for human immunodeficiency virus (HIV), with
24 identified as being HIV positive and receiving counseling.
From fiscal year 2001 to fiscal year 2008, the
number of people receiving some type of
state-provided substance abuse treatment
increased by nearly 300 percent, from 16,115
to 63,571 (see Figure 2). The Division
attributes this growth primarily to expansion in
AHCCCS eligibility requirements. For
example, in fiscal year 2000, Arizona voters
approved Proposition 204, expanding
eligibility for AHCCCS starting in 2001. The
proposition expanded income eligibility
requirements up to 100 percent of the federal
poverty level.
In fiscal year 2008 nearly all consumers
receiving substance abuse treatment
services through the RBHAs and TRBHAs
and their providers were adults, most were
non-Hispanic, and alcohol was the most
commonly used substance (see textbox,
page 7). That year, according to its Annual
State of Arizona
page 6
16,115
32,772
41,172
36,375
59,661 60,105
65,696 63,571
0
10,000
20,000
30,000
40,000
50,000
60,000
70,000
2001 2002 2003 2004 2005 2006 2007 2008
Enrollment
Fiscal Year
Figure 2: State-wide Substance Abuse Treatment Enrollment
Fiscal Years 2001 through 2008
Source: Auditor General staff analysis of the Arizona Department of Health Services,
Division of Behavioral Health Services’ Annual Report on Substance Abuse
Treatment Programs for fiscal years 2007 and 2008.
1 Median cost per person was calculated using demographic and encounter data from the Division’s Client Information
System for fiscal years 2006 through 2008 for adult substance abuse consumers without a serious mental illness.
Report on Substance Abuse Treatment Programs, the Division provided substance
abuse treatment services to 49,751 adults without a serious mental illness (78
percent of substance abuse treatment consumers), 10,071 adults who had serious
mental illness with a co-occurring substance abuse disorder (16 percent), and 3,929
children or adolescents (6 percent). Consumer characteristics are important in
planning treatment and can affect treatment success. For example, women with
children need specialized help or services, such as childcare.
Consumers entered treatment through a variety of referral sources, with nearly half
seeking treatment on their own or on the advice of a friend or family member. Referral
by court order or involvement with a criminal justice agency varies regionally. In rural
areas, court or criminal justice referrals are as high as 40 percent of the treatment-seeking
population. For example, in fiscal year 2007, in the primarily rural counties of
Yuma and La Paz, 41 percent of people enrolled in substance abuse treatment were
referred by a court or a probation/parole officer.
Office of the Auditor General
page 7
Consumer Characteristics
Fiscal Year 2008
Age: •94% adults; 6% adolescents
•50% of adults were aged 25 to 44
Gender: •56% male; 44% female
Diagnosis: •78% were adults with a substance
abuse disorder
•16% had serious mental illness
with a co-occurring substance
abuse disorder
Race: •86% Caucasian
•7% African American
•5% Native American
•2% other or multi-race
Ethnicity: •26% Hispanic/Latino
•74% non-Hispanic/Latino
Primary •37% alcohol
substance: •26% stimulants including
methamphetamine and
cocaine/crack
•24% marijuana
•11% narcotics including
heroin
•2% other
Referral •45% self, friend, or family
source:1 •23% court order or criminal
justice agency
•11% other or unknown
•10% other behavioral health
providers
•6% other state agencies
•3% community agencies
•3% Health Plan or primary
care physician
1 Numbers do not total 100 percent because of rounding.
Source: Auditor General staff analysis of the Arizona Department of Health Services, Division of Behavioral Health Services’ Annual Report on
Substance Abuse Treatment Programs for fiscal year 2008.
Substance abuse program oversight
The Division is primarily an oversight agency that monitors all aspects of behavioral
health services, including substance abuse treatment services, through contract
compliance and other mechanisms. For example, the Division’s contract compliance
unit conducts administrative reviews, as required by the Division’s contract with
AHCCCS, that examine the RBHAs’ compliance with state and federal requirements,
program operations, fiscal operations, and financial status. Although these reviews
encompass overall behavioral health, including substance abuse, they do not focus
on substance abuse.
Additionally, various units within the Division are responsible for overseeing different
aspects of the substance abuse program, as well as the general mental health
program and co-occurring disorders. For example, the grants, clinical practice
improvement, interagency coordination units, and office of the medical director all
have responsibilities that involve some oversight of general mental health treatment,
substance abuse treatment, and treatment for co-occurring disorders. The Division
has one full-time equivalent position, Lead Substance Abuse Clinical Advisor, that is
dedicated solely to the substance abuse program, and as of April 15, 2009, that
position had been vacant for 6 months because of the state hiring freeze. The
responsibilities of the Lead Substance Abuse Clinical Advisor include clinical practice
protocols and oversight of clinical practice improvement for substance abuse and
co-occurring treatment. According to the Division, other staff, such as another clinical
advisor and the Interim Director of Clinical Operations, have taken over some of the
responsibilities of the Lead Substance Abuse Clinical Advisor position.
Scope and objectives
This performance audit focused on steps the Division can take to improve substance
abuse treatment outcomes and its oversight of the substance abuse program. The
audit’s analysis and case studies focused on adult substance abuse consumers and
excluded any consumers who had a serious mental illness in addition to a substance
abuse diagnosis and consumers who were younger than 18 years old when they
entered treatment.
This audit was conducted in accordance with generally accepted government
auditing standards. Those standards require that we plan and perform the audit to
obtain sufficient, appropriate evidence to provide a reasonable basis for our findings
and conclusions based on our audit objectives. We believe that the evidence
obtained provides a reasonable basis for our findings and conclusions based on our
audit objectives.
The Auditor General and staff express appreciation to the Department’s Interim
Director and staff for their cooperation and assistance throughout the audit.
State of Arizona
page 8
Division should focus on strategies that improve
outcomes
Substance abuse is difficult to treat, but the Department of Health Services, Division
of Behavioral Health Services (Division), can employ more effective strategies to
improve outcomes. The difficulty of treatment is borne out in auditors’ analysis of the
available outcome data, which showed little change in consumer outcomes after
treatment. This analysis showed that consumers who entered treatment while already
abstaining from alcohol or drugs had the greatest success as measured in terms of
continued abstinence, whereas those consumers who entered treatment while using
alcohol or drugs showed much lower success rates. Among those consumers who
entered treatment while still using alcohol or drugs, those who actually completed
their treatment tended to do better than those who dropped out. To improve
outcomes, the Division should take steps in three main areas that research has
shown to be effective: (1) helping ensure that consumers complete their treatment,
(2) ensuring that consumers are placed in a level of treatment that is appropriate
based on their assessed needs, and (3) following practices that have been shown to
carry the greatest chance of success.
Substance abuse difficult to treat
Arizona, like many other states, struggles with the problem of treating individuals with
substance abuse problems. Substance abuse is increasingly recognized as a
chronic, relapsing condition that may require multiple episodes of care over many
years. According to the National Institute on Drug Abuse of the National Institutes of
Health, long-term drug use results in changes in brain function that can result in the
compulsion to use drugs despite adverse consequences.1 Research indicates that,
in general, approximately 50 to 60 percent of patients begin using alcohol or drugs
again within 6 months of treatment cessation, regardless of the type of discharge,
patient characteristics, or the particular substance used.2 Successful treatment of
substance abuse relies upon an individual’s ability to change his/her behavior, and
ability and motivation to integrate techniques for disease management into his/her
1 National Institute on Drug Abuse, 2009
2 Institute of Medicine of the National Academies, 2006
Office of the Auditor General
page 9
FINDING 1
lifestyle. The goal is for the individual to manage symptoms independently using
techniques learned in treatment. This is a complex process complicated by a variety
of personal, social, and cultural factors such as socio-economic, legal, family, and
employment situations.
Most consumers showed little change after treatment
Auditors’ analysis of commonly used outcome measures for substance
abuse consumers found that most participants in the Division’s programs
showed little change after entering treatment, although some consumers
improved. Auditors analyzed four National Outcome Measures (NOMs)
developed by the federal government—substance use, criminal activity,
employment, and homelessness (see textbox). Outcomes were tied to two
key factors. If consumers were already doing well in a measured area
before the start of treatment they generally maintained their status,
particularly if they completed treatment. Consumers who were not doing
well in a measured area before the start of treatment were more likely to
improve if they completed treatment, and stayed the same or got worse if
they did not complete treatment.
NOMs measure program success—The federal Substance Abuse and
Mental Health Services Administration (SAMHSA) has developed a set of National
Outcome Measures for use in evaluating treatment effectiveness. It requires states
that receive federal Substance Abuse Prevention and Treatment (SAPT) block
grant monies to report their progress using these measures. According to experts,
for the consumer and other stakeholders, effectiveness of addiction treatment is
measured by its ability to reduce addiction-related problems.1 Therefore, in
addition to substance use, the NOMs measure other outcomes such as criminal
activity, employment, and homelessness. Auditors selected those measures that
SAMHSA has fully developed and that could be measured using division
demographic data.2
Extent of improvement varied—To determine the Division’s success in treating
substance abuse consumers, auditors analyzed data from the Division’s Client
Information System (CIS) for substance abuse consumers who did not have a
serious mental illness and were enrolled with the Division in fiscal years 2006,
2007, or 2008 (see Appendix A, pages a-i through a-xxi).3 The analyses showed
1 McLellan, McKay, Forman, Cacciola, & Kemp, 2005a
2 In addition to the four measures reported in this audit, SAMHSA has identified six other domains for evaluating program
success, some of which are still under development. The other domains are social connectedness, access/capacity,
retention, perception of care, cost-effectiveness of treatment, and use of evidence-based practices.
3 Auditors’ analysis excludes any consumers who had a serious mental illness in addition to a substance abuse diagnosis,
and also excludes any consumers who were younger than 18 years old when they entered treatment. In addition, for each
outcome analyzed, auditors included only the consumers who had valid entries in the outcome data field at both times:
(1) entering treatment and (2) annual update or disenrollment. As a result, the total number of consumers analyzed for
each outcome varies. “Annual update” refers to an annual reassessment each consumer who remains in treatment
should receive. “Disenrollment” means the consumer is removed from enrollment, which occurs when the consumer
completes treatment, refuses further treatment, or stops treatment for another reason, such as moving out of Arizona.
State of Arizona
page 10
National Outcome Measures
(NOMs) analyzed by auditors
• Fewer consumers reporting recent
substance use
• Fewer consumers with recent criminal
activity
• More consumers employed
• Fewer consumers who are homeless
Source: U.S. Department of Health and Human Services
Substance Abuse and Mental Health Services
Administration.
that more than half of the consumers who entered treatment reported abstaining
from alcohol or drugs during the 30 days before starting treatment. Nearly all of
these consumers reported that they were still abstaining at the time they left the
program. However, of those who were not abstaining when they entered treatment,
over 70 percent reported no change in their substance use. Overall, the analyses
found little change in the areas of employment, criminal activity, and
homelessness. Specifically:
Slight overall improvement in substance use—Most consumers who auditors
analyzed did not change their use of drugs or alcohol from entering treatment
to their annual update or discharge. Approximately half of the consumers were
abstinent upon entering treatment and were still abstinent at their update or
discharge, while approximately one-third of the consumers who were using
drugs or alcohol when they entered treatment were
still using drugs or alcohol at their update or
discharge (see first textbox on this page). Out of the
more than 50,000 consumers auditors analyzed, a
total of 5,767 came in using drugs or alcohol and
either stopped or reduced use. Approximately
2,400 consumers got worse.
Almost all consumers abstaining at the start of
treatment remained abstinent—Auditors’ analysis
of consumers’ self-reported data showed that more
than half (54.1 percent) reported they were not
using a substance at the beginning of treatment.
More than 93 percent of these consumers reported
still being abstinent at their update or disenrollment
(see textboxes bottom of this page and top of page
12). According to officials at the Division, RBHAs,
and providers, several factors may explain why so
many consumers report they are already sober
upon entering treatment. For example, some
consumers may have quit drinking or using drugs
but enter treatment when they realize they cannot
maintain sobriety without help. Others may
misrepresent their substance abuse, either to make
themselves look better or because of wishful
thinking if they have been trying to stop for a period
of time. Still other consumers may have a problem
with binge drinking at irregular intervals. Finally,
some consumers enter treatment after spending
time in forced abstinence in jail or prison and
mistakenly report they have been abstinent,
although they are supposed to report on their use
prior to entering jail or prison.
Office of the Auditor General
page 11
Usage outcomes for all consumers1
1 Numbers do not total 100 percent because of rounding.
Source: Auditor General staff analysis of division data for adult substance
abuse consumers without a serious mental illness enrolled during
fiscal years 2006, 2007, or 2008.
Consumers who reported using alcohol or drugs when
treatment started
Reduced use 770 1.5%
Stopped using 4,997 10.0%
Stayed the same 16,504 33.0%
Got worse 693 1.4%
Consumers who reported being abstinent when treatment
started
Stayed abstinent 25,376 50.7%
Did not stay abstinent 1,729 3.5%
Total 50,069
Extent to which consumers reported being
abstinent from alcohol or drug usage
when treatment started
Source: Auditor General staff analysis of division data for adult substance
abuse consumers without a serious mental illness enrolled during
fiscal years 2006, 2007, or 2008.
Abstinent 27,105 54.1%
Using alcohol or drugs 22,964 45.9%
Total 50,069
Most consumers using alcohol or drugs at the start of
treatment continued to do so—Among the consumers who
reported that they were using alcohol or drugs in the 30
days prior to treatment, the majority—nearly 72 percent—
reported no change in the frequency of use of their primary
substance at their update or disenrollment, while 3 percent
actually increased their frequency of use (see textbox,
middle of this page). The remaining approximately 25
percent reported either reducing their use of alcohol or
drugs or stopping use altogether.
For consumers who started treatment using drugs or
alcohol, completing treatment appears to be a factor in
decreasing substance use. Those who completed
treatment had greater success in stopping their drug or
alcohol use than those who left without completing
treatment. More specifically, among this group, only 17.6
percent who did not complete treatment stopped using
drugs or alcohol, but 27 percent of those who completed
treatment stopped (see textbox, bottom of this page).
Percentage of consumers with recent arrests showed little
change—Auditors’ analysis of division data found no
significant change in criminal activity as measured by
recent arrests. Altogether, 21.2 percent of consumers
reported arrests within the past 30 days upon entering
treatment, while 18.1 percent reported new arrests at the
time of their update or disenrollment date. Most consumers
reported not having any arrests within the 30 days prior to
starting treatment and only a small number of those
consumers had a new arrest at the time of their update or
disenrollment (see textbox, page 13). By contrast, of the
consumers who reporting being arrested within 30 days of
entering treatment, although approximately one–third
reported no new arrests as of their annual update or
enrollment, the majority—68.1 percent—reported at least
one more arrest when they had their annual update or
disenrollment. NARBHA, the RBHA for Northern Arizona,
showed the greatest reduction of criminal activity among
consumers who had been arrested within 30 days of
entering treatment. Specifically, 41.2 percent of these
NARBHA consumers did not have a new arrest when they
had their update or disenrollment.
State of Arizona
page 12
Usage outcomes for consumers who
reported using alcohol or drugs when
treatment started1
Source: Auditor General staff analysis of division data for adult substance
abuse consumers without a serious mental illness enrolled during
fiscal years 2006, 2007, or 2008.
1 Numbers do not total 100 percent because of rounding.
Usage outcomes for consumers who
reported being abstinent when treatment
started
Source: Auditor General staff analysis of division data for adult substance
abuse consumers without a serious mental illness enrolled during
fiscal years 2006, 2007, or 2008.
Stayed abstinent 25,376 93.6%
Did not stay abstinent 1,729 6.4%
Total 27,105
Reduced use 770 3.4%
Stopped use 4,997 21.8%
Stayed the same 16,504 71.9%
Increased use 693 3.0%
Total 22,964
Usage outcomes for consumers who
reported using alcohol or drugs when
treatment started and who completed
treatment
Source: Auditor General staff analysis of division data for adult substance
abuse consumers without a serious mental illness enrolled during
fiscal years 2006, 2007, or 2008. Excludes consumers who were
still in treatment or who disenrolled for administrative reasons such
as transfers between RBHAs, and 278 consumers who enrolled as a
result of a crisis and then disenrolled.
Reduced use 121 1.7%
Stopped use 1,894 27.0%
Stayed the same 4,855 69.2%
Got worse 144 2.1%
Total 7,014
Usage outcomes for consumers who
reported using alcohol or drugs before
treatment and who did not complete
treatment
Reduced use 432 3.4%
Stopped use 2,270 17.6%
Stayed the same 9,771 75.8%
Got worse 410 3.2%
Total 12,883
Employment did not significantly change—The Division’s programs have not
led to a significant change in overall consumer employment. Thirty-eight
percent of substance abuse treatment consumers, or 16,360, reported that
they were already employed when they entered treatment, and 41 percent
reported being employed as of their annual update or disenrollment. Of those
consumers who were already employed, more than 92 percent reported being
employed as of their annual update or disenrollment (see textbox, page 14).
Among the almost 27,000 consumers, or 62 percent, who reported not being
employed upon entering treatment, nearly 10 percent reported that they had
gained employment as of their annual update or disenrollment, although 90
percent were still unemployed. NARBHA showed the most improvement in
this outcome area, as 16.1 percent of NARBHA’s unemployed consumers
had gained employment as of their update or disenrollment.
Homelessness showed little change—Overall, the number of consumers who
were homeless did not change from entering treatment to update or
disenrollment, with 7.1 percent homeless upon entering treatment and 6.8
percent homeless at update or disenrollment. Most consumers were not
homeless upon entering treatment, and those who were homeless had some
Office of the Auditor General
page 13
Arrest outcomes for all consumers
Arrest outcomes for consumers with no recent arrests when treatment
started
Arrest outcomes for consumers with recent arrests when treatment
started
Source: Auditor General staff analysis of division data for adult substance abuse consumers without a serious mental illness
enrolled during fiscal years 2006, 2007, or 2008.
Consumers with no recent arrests when treatment started
Had new arrest before leaving treatment 1,907 3.7%
Did not have new arrest 38,565 75.1%
Had new arrest before leaving treatment 7,408 14.4%
Did not have new arrest 3,468 6.8%
Total 51,348
Consumers with recent arrests when treatment started
Had new arrest before leaving treatment 1,907 4.7%
Did not have new arrest 38,565 95.3%
Total 40,472
Had new arrest before leaving treatment 7,408 68.1%
Did not have new arrest 3,468 31.9%
Total 10,876
success moving into stable housing situations. Specifically, almost 46,000, or
93 percent, of substance abuse consumers reported living in a stable housing
situation upon entering treatment, and almost 99 percent of these consumers
reported being in stable housing at their annual update or disenrollment (see
textbox, page 15). Of the more than 3,500 consumers, or 7 percent, who
reported not being in stable housing when they entered treatment,
approximately 21.7 percent of these consumers reported gaining stable
housing by the time of their annual update or disenrollment. Housing services
are generally not available to substance abuse treatment consumers. Such
services are not covered by Title XIX, and state monies only provide the
services for consumers with serious mental illness. Still, increased housing
stability is considered an important measure of substance abuse treatment
success.
Limited comparative information indicates improvements in
Arizona’s program may be low—As part of the analysis, auditors
compared Arizona’s NOMs with those reported by other states. The Division uses
this information to compare Arizona outcomes to other states. The available data,
while limited in quality and consistency, suggests that Arizona’s performance is
below that of substance abuse programs in other states. The limitations in the
State of Arizona
page 14
Employment outcomes for all consumers1
Employment outcomes for consumers who reported being employed
before starting treatment
Employment outcomes for consumers who reported being
unemployed before starting treatment
1 Numbers do not total 100 percent because of rounding.
Source: Auditor General staff analysis of division data for adult substance abuse consumers without a serious mental illness
enrolled during fiscal years 2006, 2007, or 2008.
Consumers who were employed before starting treatment
Unemployed when treatment ended 1,255 2.9%
Employed when treatment ended 15,105 34.9%
Unemployed when treatment ended 24,245 56.0%
Employed when treatment ended 2,656 6.1%
Total 43,261
Consumers who were unemployed before starting treatment
Unemployed when treatment ended 1,255 7.7%
Employed when treatment ended 15,105 92.3%
Total 16,360
Unemployed when treatment ended 24,245 90.1%
Employed when treatment ended 2,656 9.9%
Total 26,901
quality of the comparative data stem from differences in how state programs are
structured and what the states may be reporting. For example, the data that
Arizona reports to SAMHSA includes substance abuse treatment for consumers
with serious mental illness, which probably contributes to the lower percentages of
consumers meeting the measurement. Whether other states similarly report
people with serious mental illness as part of their substance abuse cases is
unknown. Thus, while the comparison may be useful in providing a general
indication of where Arizona stands relative to other states, it should not be taken
as authoritative. Even so, however, Arizona’s apparent ranking relative to other
states is another reason to examine the program carefully to determine if it can be
improved.
Auditors examined the national data collected by SAMHSA at the federal level and
found that both nationally and in the SAMHSA-designated Western Region, the
percentage of Arizona’s substance abuse treatment consumers meeting the
measurement criteria is below average in three of the four areas auditors analyzed:
abstinence, criminal activity, and employment.1 This is the case both for
consumers who were entering treatment and for consumers at the time of their
discharge. Further, the rate of change for Arizona’s consumers from admission to
Office of the Auditor General
page 15
Housing outcomes for all consumers1
Housing outcomes for consumers reporting they were not homeless
before starting treatment
Housing outcomes for consumers reporting they were homeless
before starting treatment
Source: Auditor General staff analysis of division data for adult substance abuse consumers without a serious mental illness
enrolled during fiscal years 2006, 2007, or 2008.
Consumers who were not homeless before starting treatment
Homeless at update or discharge 612 1.2%
Not homeless at update or discharge 45,282 91.6%
Homeless at update or discharge 2,763 5.6%
Not homeless at update or discharge 765 1.5%
Total 49,422
Consumers who were homeless before starting treatment
Homeless at update or discharge 612 1.3%
In stable housing at update or discharge 45,282 98.7%
Total 45,894
Homeless at update or discharge 2,763 78.3%
In stable housing at update or discharge 765 21.7%
Total 3,528
1 Numbers do not total 100 percent because of rounding.
1 SAMHSA’s designated western states reporting NOMs data as part of their fiscal year 2009 SAPT block grant application
were Alaska, Arizona, California, Hawaii, Idaho, Montana, Nevada, Oregon, Washington, and Wyoming.
discharge is below both the national average and the average for western states
on all four measures—that is, the substance abuse programs in other states
appeared to have a greater effect in raising outcomes in these other states. (See
Appendix A, Figures 6 through 9, 11, 12, 14, 15, 17, and 18, pages a-vi through a-xxi,
for comparisons of Arizona’s reported NOMs with other states’ reported
NOMs, both regionally and nationally.)
Case studies help further explain factors affecting treatment—To
provide additional perspective on the program, auditors also analyzed a limited set
of specific cases to understand consumers’ progress or lack of progress and the
factors that affected success. Auditors examined case files from specific treatment
episodes and then interviewed the consumers, treatment providers, and others
involved in the case to determine long-term outcomes and what consumers
thought contributed to their abstinence or use. Two such cases help illustrate the
complicated factors that can affect individual long-term outcomes:
Juan, 44, a married man with children who lives in southwestern Arizona,
entered substance abuse treatment after being arrested for driving with a
blood alcohol level of 0.17 and charged with extreme DUI.1 The court
suspended his driver’s license and ordered him to undergo treatment. The
court referred Juan to a local mental health agency that has a contract with the
RBHA to provide substance abuse counseling in his area. The provider placed
Juan in a 9-week substance abuse and relapse prevention program
consisting of twice-weekly group therapy focused on education and relapse
prevention.
At Juan’s intake session with the provider, he stated that prior to his arrest he
did not drink very often, but would have seven or eight beers when he was out
with his friends. Although the reason he went to substance abuse treatment
was to clear up his legal troubles related to the DUI, Juan reported to auditors
that the program gave him insight into the dangers of drinking and driving and
the impact his drinking had on his family. Juan completed his treatment in
December 2005, and when auditors interviewed him in February 2009, he
reported that he had not had a drink in 3 years.
In this case, the consumer’s NOMs outcomes were unchanged from
beginning treatment to completion of treatment because he had a high level
of achievement before entering treatment. Juan had never been arrested
before. He was employed and living in a stable family situation, and he
reported no alcohol use within the 30 days before he entered treatment as well
as within the 30 days before he was disenrolled.
Dana, 19, entered treatment in northern Arizona after being arrested and
serving 60 days in jail on charges of drug sale, possession of drug
paraphernalia, and shoplifting. Two of her charges were felonies. Her
State of Arizona
page 16
1 All consumer names have been changed.
substance abuse treatment was required as part of her probation. She had
been using alcohol since age 13 and methamphetamine since age 16, and
had had previous substance abuse treatment and periods of abstinence. At
her intake, Dana stated that she needed supervision and did not trust herself
to abstain. Her provider recommended a month of residential treatment and
then moved Dana into a two-stage intensive outpatient program. Over the 11
months of her treatment, she had considerable positive change in her NOMs.
At her enrollment, she had just been released from jail, was unemployed, was
not attending school, and had no stable place to live. At disenrollment she
was self-supporting, enrolled in college, living with a family friend, and had no
additional arrests. She was involved in Narcotics Anonymous and was not
using drugs or alcohol.
In Dana’s case, the NOMs data showed substantial progress on all four
measures. However, Dana was unable to sustain these changes and relapsed
about 1 year after completing treatment. When auditors spoke with her in
February 2009, she was in prison with 5 more years to serve on a drug-related
conviction.
Case studies such as these, together with the results of the auditors’ analyses of the
four NOMs, illustrate the difficulty in making decisions about how to improve program
success. For example, the Division captures NOMs data at various points during
treatment, but the data alone does not adequately indicate whether or not treatment
has been successful. Auditors’ review of studies, best practices, and other materials
showed, however, that improvements in program outcomes are likely to come from
focusing on three main strategies: increasing the number of consumers who
complete their treatment program, providing a continuum of care that addresses
multiple needs, and using practices that have been shown to generate positive
outcomes. The sections that follow discuss each of these strategies in turn.
Division should increase focus on treatment retention and
completion
Research and the Department’s own data indicate that consumers who remain
longer in treatment experience better outcomes. However, most consumers state-wide
do not complete their treatment. As the analysis discussed on page 12
indicated, those who complete treatment experience better outcomes. Several
factors, such as individual motivation and the availability of different treatment
options, can influence whether or not consumers stay in treatment. The Division can
take several steps to increase consumer retention and completion rates, including
establishing performance goals, monitoring completion rates, and using financial
and/or nonfinancial incentives and other case management techniques.
Office of the Auditor General
page 17
Longer treatment yields better results—Research has shown that consumers
who remain longer in treatment, no matter what type of treatment it is, show the best
post-treatment outcomes.1 For example, studies have shown that longer treatment
duration resulted in lower readmission rates and lower rates of drug use and
relapse. One study found that consumers who received long-term residential or
outpatient treatment for 6 months or more showed a lessened likelihood of
engagement in illegal activities than those with shorter lengths of stay.2 Another
reported that longer lengths of stay were positively associated with post-treatment
client earnings across all types of treatment.3 The National Institute on Drug Abuse
states that most consumers need to remain in treatment for about 3 months in order
to achieve significant improvement, and additional progress toward recovery can
be achieved by providing treatment after the 3-month threshold.4 However, a 2003
study suggested that consumers who stay in treatment for longer than 18 months
may start to show reduced amounts of improvement.5
Ensuring treatment completion is an important strategy for improving consumers’
outcomes. Auditors’ analysis of division data found that in addition to being more
likely to reduce or stop substance use, consumers who completed treatment were
more likely to gain employment and avoid criminal activity than those who did not
complete treatment. For example, of the consumers who were unemployed when
they entered treatment, 14.4 percent of those who completed treatment gained
employment, while only 6.2 percent of noncompleters gained employment.
Similarly, of the consumers who had been arrested within the 30 days prior to
entering treatment, 47.5 percent of those who completed treatment had no new
arrests at their update or discharge, but only 27.5 percent of those who did not
complete treatment had no new arrests.
Most consumers drop out or receive short-term treatment—Despite the
importance of retention, auditors’ analysis of division data found that a majority—
58 percent—of the consumers reviewed did not complete their treatment before
they were disenrolled. All six geographic services areas had fewer consumers who
completed treatment than those who did not complete treatment (see Figure 3,
page 19). The most common reason given for disenrolling consumers who did not
complete treatment was that providers lost contact with them (38.3 percent of all
reviewed consumers), while 9.4 percent of reviewed consumers refused further
treatment. According to the Division, without a court order, consumers’ treatment is
voluntary and they can withdraw at any time. Other reasons for disenrollment are
death, incarceration, and a loss of Title XIX status. Further, in contrast to the 3
months suggested in literature, the Division reported that in fiscal year 2007, its
average length of stay for outpatient treatment was 29 days and for long-term
residential care the average length of stay was 78 days. In that fiscal year, more than
60,000 consumers were treated in outpatient programs, while fewer than 700 were
treated in residential, long-term programs.
State of Arizona
page 18
1 McLellan, Chalk, & Bartlett, 2007
2 Etheridge, Craddock, Hubbard, & Rounds-Bryant, 1999 as cited in Koenig, Harwood, Sullivan, & Sen, 2000
3 Koenig, Harwood, Sullivan, & Sen, 2000
4 National Institute on Drug Abuse, 2009
5 Zhang, Friedmann, & Gerstein, 2003
Studies show that
treatment retention and
completion result in
better outcomes.
Office of the Auditor General
page 19
39.4
48.7
37.6
43.3
44.5
42.5
50.8
60.6
51.3
62.4
56.7
55.5
57.5
49.2
0
10
20
30
40
50
60
70
NARBHA (GSA 1) Cenpatico (GSA 2) CPSA (GSA 3) Cenpatico (GSA 4) CPSA (GSA 5) Magellan/VO (GSA 6) Tribes
Percent of Consumers
Completed Treatment Did Not Complete Treatment
Figure 3: Treatment Completion and Noncompletion Rates by RBHA (GSA)
Fiscal Years 2006 through 2008
Source: Auditor General staff analysis of data from the Division’s Client Information System (CIS) for adult substance abuse treatment consumers without a serious mental
illness enrolled in fiscal years 2006, 2007, or 2008.
State of Arizona
page 20
Several factors affect retention—Whether a person stays in treatment
depends on factors associated with both the individual and program, according to
the National Institute on Drug Abuse (Institute). The Institute’s Principles of Drug
Addiction Treatment states that “individual factors related to engagement and
retention include motivation to change drug-using behavior, degree of support
from family and friends, and whether there is pressure to stay in treatment from the
criminal justice system, child protective services, employers, or the family. Within a
treatment program, successful counselors are able to establish a positive,
therapeutic relationship with the patient. These counselors also ensure that a
treatment plan is established and followed so that the individual knows what to
expect during treatment.”1 Similarly, the Treatment Research Institute (TRI) states
that retention can be improved by making treatment attractive, offering options,
increasing monitoring and management, and giving consumers feedback.2 The
TRI points out that consumers who are not in some form of treatment or monitoring
are at increased risk of relapse. Other researchers have also reported that
providing incentives to consumers, such as a reward for a clean urinalysis test, can
help increase retention.
Division can do more to increase retention—The Division does not monitor
treatment retention or length of stay, and although it collects consumer
disenrollment reasons it does not analyze or monitor the data. Its contracts with the
RBHAs do not require them to collect or report information on retention or length
of both inpatient and outpatient stay. The contracts require efforts to re-engage
consumers in treatment—specifically, the RBHAs must require their providers to
attempt re-engagement, and the Division’s provider manuals also establish
specific requirements regarding attempting to re-establish contact with
consumers—and the Division’s Administrative Review monitors RBHA’s
compliance with this standard. However, auditors’ analysis of division data found
that 47.7 percent of consumers could not be re-engaged in treatment (38.3
percent could not be contacted by providers and 9.4 percent refused further
treatment), and the 2007 Administrative Review showed all four RBHAs needed
improvement on the re-engagement measure. One of the auditors’ case examples
illustrates this. Dana—the methamphetamine user who relapsed after treatment
completion and was in prison when auditors interviewed her in February 2009—
had spotty attendance during her treatment, including a period in May 2006 where
she attended less than half her group sessions. Her case manager did not
address her attendance issue until Dana asked about being released from
treatment. Then, she was informed that her probation required her to maintain
acceptable attendance in order to be considered for graduation from the program.
Some other states have adopted goals and incentives related to retention and
treatment completion. For example:
1 National Institute on Drug Abuse, 2009
2 McLellan, 2006
Office of the Auditor General
page 21
Colorado—To meet performance goals, providers must maintain or improve the
average percentage of clients from the previous 2 fiscal years who remain in
outpatient treatment longer than 90 days, and maintain or decrease the
percentage of consumers who left treatment against professional advice.
Maine—Performance goals stipulate a minimum of 50 percent of outpatient and
85 percent of intensive outpatient consumers stay at least four sessions.
Additionally, 30 percent of outpatient consumers must stay in treatment 90 days
or longer, and 50 percent of intensive outpatient treatment consumers must
complete treatment. Providers who exceed these expectations are awarded
incentive bonuses, while underperforming providers are subject to disincentive
penalties.
Delaware—Contracts stipulate a $100 bonus incentive to providers for each
consumer who completes at least 60 days in treatment and achieves the major
goals of his or her treatment plan, including submitting four consecutive drug-free
urine screenings randomly administered.
To help improve treatment outcomes, the Division should establish performance
goals and financial or nonfinancial incentives and disincentives related to retention
and treatment completion in its RBHA contracts, and collect and monitor data on
retention and completion. In establishing benchmarks and incentives, the Division
should consider the potential for unintended consequences such as retaining
consumers inappropriately and take steps to ensure that the incentives achieve the
desired goals. For example, it could follow Delaware’s example of combining
retention with other goals in order to determine whether providers qualify for
incentives. In addition, the Division should use its existing oversight practices such
as its quarterly case reviews to determine whether providers are taking appropriate
steps to retain and engage consumers in treatment. Based on the results of these
reviews, the Division should work with the RBHAs to address weaknesses through
mechanisms such as technical assistance, training, contract requirements, and/or
policy and procedural changes.
Division should monitor continuum of care
A second strategy for improving consumer outcomes is to provide a continuum of
care. Under a continuum of care approach, consumers are placed in treatment
based on the types of services they need and the phase of recovery they are in. After
initial placement, consumers then step up to more intensive treatment or down to
less intense treatment as needed. However, as illustrated by case studies, not all
consumers have their treatment needs met or receive the services and therapies that
may improve the chances of good treatment outcomes. The Division can do more to
ensure continuum of care for consumers by taking several steps, such as collecting
and monitoring data relevant to assessment and case management, and working
with the RBHAs to make improvements when weaknesses are identified.
State of Arizona
page 22
Continuum of care matches consumers to appropriate treatment
based on their problems and stage of recovery—Because no single
treatment is appropriate for all individuals, providers need to be able to match
treatment settings, interventions, and services to each individual’s particular
problems. Continuum of care is a treatment system in which consumers entering
treatment are placed in a level of treatment appropriate to their assessed needs.
Individuals are first assessed to determine appropriate treatment services.
Successful matching of the consumers to the appropriate treatments for them
helps to ensure positive outcomes.1 After initial placement, consumers then step
up to more intensive treatment or down to less intense treatment as needed. As
the consumers recover, and further assessments are made of consumers’
progress, the treatment is phased down and consumers can be moved into
alternative community-based services.
Some consumers receive continuum of care while others do not—
One of the auditors’ case studies illustrates how treatment that is appropriate to a
consumer’s needs can yield long-term results:
Mandy, 29, was in treatment for nearly a year. When she contacted a provider,
she had already had previous experience with treatment, she was pregnant,
her husband was in jail, and her two children were living with relatives. She
reported that she had used methamphetamines, alcohol, and marijuana
within the past several weeks. After completing a month of treatment with her
first provider, she was referred to a specialized residential treatment center for
women who are pregnant or have children that offers integrated services
designed to teach them to be better parents and give them skills they need to
get jobs, take care of their children, and stay sober. As a pregnant woman and
therefore part of a priority population, Mandy’s treatment was eligible for SAPT
funding. During almost 1 year of treatment with this second provider, Mandy
made the decision to end her marriage and place her baby with adoptive
parents in an open adoption. At the end of treatment, Mandy had a job, was
living with her mother and one of her sons, and was working on her
relationship with her other son.
When auditors interviewed her in April 2009, 2 years after she completed
treatment, Mandy stated that long-term treatment that focused on all her
issues gave her a solid foundation for a sober life. She said she no longer had
any cravings or desire to use drugs. During her year of integrated treatment,
she learned to address the issues that led to her addiction. Her previous short-term
treatments had not given her the ability to reflect on the information
provided during treatment. Mandy now works as a peer support specialist in
the substance abuse field, has custody of one of her sons, and as of April
2009 was about to get custody of her other son. In addition, she was living
independently and was in contact with her daughter’s adoptive parents.
Continuum of care is an
important strategy in
ensuring treatment is
successful.
1 Mee-Lee, n.d.
Office of the Auditor General
page 23
By contrast, another example, identified through auditors’ analysis of high-cost
consumers, illustrates a failure to provide services along the continuum from acute
detoxification and stabilization through rehabilitation to continuing recovery support:
Rodney, 43, used medical detoxification services 54 times during the period July
2005 through June 2008—approximately every 3 weeks—receiving services
valued at a total of more than $82,000. (See Finding 2, pages 31 through 42, for
a discussion of service costs). Rodney was homeless and reported using
alcohol daily. Although he received some case management and other crisis
services besides detoxification, he received very little in the way of other
services, such as treatment that would reinforce his sobriety after detoxification
and rehabilitation to help him learn to remain sober in the long term. According
to a case manager at the detoxification center, Rodney typically stays in the
center for only 2 to 3 days and chooses not to complete treatment or receive
follow-up care.
A consumer like Rodney who undergoes only the first stage of care is likely to
return to substance abuse. Further, when auditors discussed this case with the
Division’s medical director, he expressed concern that Rodney’s health is
jeopardized by this constant cycling between heavy drinking and medical
detoxification.
Consumers are sometimes placed into services that may not meet their individual
needs—contrary to the continuum-of-care concept—or that duplicate services they
have already received. Auditors’ case studies included three consumers who
received the same treatment program from different providers in different parts of the
State: Juan, discussed previously, and two other consumers, Tom and Victoria. All
three received alcohol education classes designed for people who have received
their first DUI or alcohol-related arrest. These classes are intended to get participants
to recognize problematic behaviors and make better choices in the future. For Juan,
who had his first arrest for DUI at the age of 44, the class was an effective wake-up
call regarding the dangers of impaired driving and how drinking affected his family
relationships. He told the auditors that his 9 weeks of classes were very helpful to him
and improved his life. Tom and Victoria, however, had long histories of alcoholism
and completed the classes after serving prison sentences for drunken driving. Both
Tom and Victoria had received similar classes while in prison. Tom told auditors that
going through the same treatment after his release was not helpful and did not
contribute to his continued sobriety. Instead, Tom and Victoria felt they needed
treatment that would have helped them adjust to life after prison and address other
issues. Both said their continued sobriety resulted from their own desire to avoid
further legal troubles.
In addition to auditors’ case studies that found that continuum of care is not
consistently and appropriately provided, a 2008 study by one RBHA also found
cases where consumers did not receive needed substance abuse services.
Auditors’ case studies
and a RBHA study
revealed that not all
consumers’ needs were
met.
1 De Leon, Melnick, & Cleland, 2008
2 McLellan, Weinstein, Shen, Kendig, & Levine, 2005b
State of Arizona
page 24
Specifically, the Community Partnership of Southern Arizona (CPSA) conducted a
focus review in August 2008 of a psychiatric health facility in its network and found
that there was little evidence to support that the facility provided active treatment to
the ten consumers sampled in the review, including three consumers who required
substance abuse treatment (For more information on this review, see Finding 2,
pages 31 through 42). The review found that none of the three consumers received
substance abuse treatment services during their 7- to 10-day inpatient stays at the
facility. Continuum of care requires that consumers’ needs are assessed and that
they receive treatment that matches their needs.
Division can do more to ensure continuum of care—The Division’s
contracts with the RBHAs already specify that providers are required to develop
treatments and services designed to support long-term recovery and focus on life
factors such as employment, ongoing feedback to the recipient, and re-engagement
into treatment based on the recipient’s changing needs. However,
the Division’s oversight processes have not focused on ensuring that the RBHAs
have implemented these requirements for substance abuse consumers (See
Finding 2, pages 31 through 42, for recommendations on improving oversight of
substance abuse treatment).
As the Division moves to ensure that the RBHAs implement a more effective
continuum of care, it should adopt two approaches shown by research to improve
consumer outcomes. Specifically:
Establishing assessment standards—The Division should establish standards
for assessing the severity of consumers’ substance abuse problems and
referring consumers to appropriate treatment, and monitor implementation of
these standards as part of its regular oversight of RBHA performance. A 2008
study found that assessing the severity of a consumer’s substance abuse
problem and then assigning more severe consumers to more intensive
treatment improves both outcomes and cost-effectiveness of treatment.1 The
study reported that appropriate placement does not require a complex
matching process where individuals are matched with a specific style of
treatment, such as cognitive behavioral therapy versus a 12-step program
(see textbox, page 25).
Defining case management expectations—The Division should define
appropriate expectations for case management of substance abuse
consumers, and consider assigning consumers with severe or complex cases
to a case manager for their complete course of treatment, taking into
consideration the costs of case management. For example, effective clinical
case management can reduce inefficient over-use of detoxification. A 2005
study in Philadelphia concluded that multiple episodes of detoxification
without follow-up care are inappropriate, ineffective, and inefficient.2 Pairing
detoxification with clinical case management resulted in a 55 percent
Office of the Auditor General
page 25
reduction in detoxification admissions and a 70 percent increase in use of
rehabilitation. Clinical case managers were expected to engage consumers
early and assess their substance use, health, and social problems; use
motivational interviewing designed to help consumers recognize and accept
the need for continued care; and play an active, participative role in helping
the consumers access and continue with needed medical and social services
(See Finding 2, page 40, for information on how the Division can use case
management to help minimize costs).
When applying these approaches in its management of the program, the Division
should also collect and monitor data relevant to assessment and case
management and work with the RBHAs to make improvements when its oversight
identifies weaknesses.
Using appropriate evidence-based practices can improve
success rate
A third strategy the Division can employ is to increase
its attention to providers’ use of evidence-based
practices. These practices are validated by
observation or experience as improving treatment
success. Although the Division requires the RBHAs to
use evidence-based practices, RBHAs are not
necessarily doing so, and the Division is not ensuring
compliance. To improve adherence to evidence-based
practices, the Division should enhance its
monitoring of the practices and take appropriate steps
to address any weaknesses it finds.
Evidence-based practices important to
success—Evidence-based practices are
therapies confirmed by scientific research to have a
positive effect in the treatment of substance abuse.
They include motivational interviewing, cognitive
behavioral therapy, community reinforcement
therapy, and 12-step facilitation therapy (see
textbox). There is a growing body of literature on the
cost-effectiveness of well-defined treatments that
have been validated by observation or experience
for substance use disorder, according to a 2007
study.1 SAMHSA has developed the National
Registry of Evidence-based Programs and
Practices, a listing of programs verified by
independent researchers for their ability to achieve
1 Olmstead, Sindelar, Easton, & Carroll, 2007
Examples of Evidence-based Practices
• Motivational Interviewing—A counseling style designed
to help consumers recognize and accept the need for
continued care. (www.motivationalinterview.org)
• Cognitive behavioral therapy—A treatment that seeks to
change the way an individual thinks to feel/act better
even if the situation does not change. (www.nacbt.org)
• Community reinforcement therapy—Therapy that
focuses on events that influence behavior, such as
family and social events, and uses the events to support
successful abstinence. (www.ncsacw.samhsa.gov)
• 12 step facilitation therapy—A brief, structured therapy
that supports initial and ongoing participation in a 12-
step program such as Alcoholics Anonymous.
(www.nrepp.samhsa.gov)
• ASAM criteria—Placement guidelines that focus on
assessing consumers across six dimensions such as
emotional and behavioral conditions and readiness to
change and then placing the consumer in a level of care
to match the severity of his/her condition.
(www.asam.org and www.coce.samhsa.gov)
• Stages of change—A model of behavioral change that
progresses from examining the risks of change to
commitment to action, and then plan, implement, and
maintain the change. (www.coce.samhsa.gov)
Source: Auditor General staff summary of evidence-based practices as defined on
the above Web sites on June 24, 2009.
State of Arizona
page 26
results when properly implemented. The Division’s review of residential treatment
centers supports the use of such evidence-based practices and the results that
can be achieved. The Division contracted with a private firm to review nine
substance abuse residential treatment centers that provide such services mainly
to women to gather baseline data. In that review, all nine centers reported that they
used at least two forms of evidence-based best practices. The three most
commonly reported practice models across all nine centers included the American
Society of Addiction Medicine (ASAM) criteria, motivational interviewing, and
stages of change. The report indicated that 97.5 percent of the cases reviewed
showed evidence of symptomatic improvement.
Evidence-based practices are also used for outpatient treatment. For example,
there are three Methamphetamine Treatment Centers of Excellence located
throughout the State, which specialize in the treatment of methamphetamine use
through techniques such as incentives, community reinforcement, motivational
interviewing, and monitoring behavior through urine testing.
Requirement for evidence-based practices not necessarily followed
or enforced—The Division’s contracts with the RBHAs require the use of
evidence-based practices, but the Division can do more to ensure providers are
using the most up-to-date research and implementing the practices in accordance
with the evidence. The Division has received or conducted three reviews of
substance abuse programs. For example, in one study, a 2008 federal grant review
that focused on programs for children and adolescents, the Division did a good
job of establishing evidence-based practices in some areas, but did not identify
such practices across the continuum of care. It could also do more to ensure
sustainability of the emphasis on evidence-based practices. In another study, the
Division reviewed 11 intensive outpatient programs for youth across the State and
found that, although the Division had an internal definition of what constitutes an
intensive outpatient program, the definition was not documented or adequately
communicated to the providers. Therefore, a number of the providers’ programs
did not meet the Division’s definition.
One example of implementation of an evidence-based practice for substance
abuse treatment is at NARBHA, which received a SAMHSA grant to implement a
program called Matrix for adult treatment services. NARBHA uses a tool provided
by the Matrix Institute to evaluate providers and ensure the program is
implemented properly. The Matrix model incorporates therapist support, group
participation, support group participation, relapse prevention and education,
family involvement, and an explicit structure giving consumers a clear
understanding of treatment. According to staff at one of NARBHA’s providers, the
program has helped them get consumers into the structure of recovery. One
element of the model, contingency management—where consumers are
rewarded for successful participation, for example with prizes such as movie
passes—is seen as helpful for keeping consumers engaged in treatment.
NARBHA uses an
evidence-based
practice for substance
abuse treatment and
monitors it to ensure
proper implementation.
Office of the Auditor General
page 27
The Division may be able to use one evidence-based practice—self-help group
participation—more effectively. Research has shown a strong association between
self-help group participation and long-term recovery following treatment. Although
none of the consumers included in auditors’ case studies continued to participate
regularly in self-help groups after treatment, one of them, Dana—the young
woman serving a drug-related prison sentence—attributed her return to drug use
to when she stopped attending Narcotics Anonymous meetings and continued to
associate with friends who used drugs. She indicated that she found it useful to
have group leaders who were recovering addicts because their familiarity with
addiction makes them good at breaking down addicts’ behavioral defense
mechanisms.
Division can do more to encourage use of evidence-based
practices—Auditors identified two key ways in which the Division can
encourage greater use of evidence-based practices:
More extensive monitoring—The Division examines a sample of substance
abuse consumers’ cases each quarter as part of its overall oversight of the
RBHAs in order to ensure consumers obtain services in a timely manner and
receive appropriate services. In addition, in order to fulfill federal grant
requirements, the Division has conducted a review of substance abuse
programs for adolescents and contracted for a review of women’s substance
abuse residential treatment facilities. Specifically, during 2007 and 2008,
division staff reviewed 11 intensive outpatient programs for adolescents to
determine how the programs could be improved. Also, in 2008, the Health
Services Advisory Group, an external quality review organization contracted by
the Division, conducted a study of women’s residential treatment facilities to
gather baseline data on women’s substance abuse treatment programs.
However, the Division does not routinely review services to determine whether
the RBHAs are applying evidence-based practices or implementing them as
they were designed to be used. Similar to NARBHA’s approach with its
providers that use the Matrix model, the Division should monitor evidence-based
practices either directly or by reviewing the RBHAs’ assessments of
providers’ practices.
Improved guidance—The Division should also continue and expand its work
with RBHAs to ensure that providers have the guidance needed to implement
specific evidence-based practices. When the Division implements new
treatment models, it provides guidance and other assistance to ensure the
RBHAs and providers implement the models correctly. For example, it used
this approach to implement a new system of care for children in the early
2000s, as discussed in the Auditor General’s report on implementation of
HB2003 programs for children (see Report No. 02-12). The Division also
cosponsors an annual summer institute conference for RBHAs and providers.
The Division should use the results of its monitoring of evidence-based
The Division may be
able to use self-help
groups more
effectively.
practices implementation to identify any needed improvements, and then
work with RBHAs to provide technical assistance, training, and guidelines as
appropriate. Finally, the Division should make better use of self-help programs
by developing a method to track and monitor participation and encouraging
RBHAs to offer a wider variety of programs for consumers. Because self-help
group participation can continue the benefits of treatment long after treatment
is completed, the Division should also consider working with the RBHAs to
develop procedures to follow up with consumers after treatment is completed
to encourage continued participation in such groups.
Recommendations:
1.1. To help improve retention, the Division should:
a. Collect and monitor data on retention and completion, including length of
stay and disenrollment reasons;
b. Establish performance goals and financial and/or nonfinancial incentives
and disincentives related to retention and treatment completion in its RBHA
contracts, taking care to avoid encouraging providers to inappropriately
retain consumers in treatment in order to meet the goals;
c. Use its existing oversight practices such as its quarterly case reviews to
determine whether RBHAs are taking appropriate steps to retain and
engage consumers in treatment; and
d. Based on the results of these reviews, the Division should work with the
RBHAs to address weaknesses through mechanisms such as technical
assistance, training, contract requirements, and/or policy and procedural
changes.
1.2. To make better use of the continuum of care to improve treatment outcomes, the
Division should:
a. Establish standards for assessing the severity of consumers’ substance
abuse problems and referring them to appropriate treatment;
b. Using data, monitor implementation of these standards as part of its regular
oversight of RBHA performance;
c. Define appropriate expectations for case management of substance abuse
consumers, taking into consideration costs of case management and the
advantages of monitoring consumers with severe or complex cases;
State of Arizona
page 28
d. Collect and monitor data relevant to assessment and case management;
and
e. Work with the RBHAs to make improvements when its oversight identifies
weaknesses.
1.3. To better ensure the use of appropriate evidence-based practices, the Division
should:
a. Monitor compliance with its contractual requirements to use evidence-based
practices;
b. Work with RBHAs to provide technical assistance, training, and guidelines
as appropriate to ensure that providers have the guidance needed to
implement specific evidence-based practices such as motivational
interviewing, cognitive behavioral therapy, and community reinforcement
therapy;
c. Develop a method to track and monitor self-help group participation;
d. Encourage RBHAs to offer a wider variety of self-help programs for
consumers; and
e. Consider working with the RBHAs to develop procedures to engage
consumers in community and peer support outlets that would reinforce
progress made in treatment once consumers are disenrolled.
Office of the Auditor General
page 29
State of Arizona
page 30
Division should improve oversight of substance
abuse programs
The Department of Health Services, Division of Behavioral Health Services (Division),
should take steps to improve its oversight of the substance abuse programs
administered by the Regional Behavioral Health Authorities (RBHAs). Current
oversight efforts are limited in two key ways. First, although the Division gathers
information on treatment outcomes, it does not use this information to monitor
programs, focusing instead on process-related information such as whether services
are timely. To increase the benefit that treatment participants receive, the Division
should continue its efforts to streamline uniform data collection that focuses on
consumer outcomes, establish relevant performance goals in contracts with the
RBHAs, and encourage the RBHAs to consider mechanisms for awarding financial
and/or nonfinancial incentives to providers who meet these goals and penalizing
those who do not. Second, oversight efforts currently lack any significant focus on
analyzing treatment costs and types. This is particularly true for analyzing high-cost
or low-cost cases that may indicate ineffective or inadequate services. To help better
ensure that the Division’s scarce resources are used effectively, the Division should
implement a systematic approach that includes but is not limited to reviewing high-and
low-dollar cost services by individual users and treatment categories and among
RBHAs, and considering how to best use assessment and case management to
contain costs.
Division should monitor outcomes
The Division should improve oversight of substance abuse treatment by increasing
its focus on consumer outcomes—that is, what the services are accomplishing in
terms of keeping consumers from reverting to abuse of alcohol or drugs. The Division
focuses primarily on treatment process measures for all behavioral health
consumers, including substance abuse consumers, and it is therefore largely unable
to determine if its substance abuse treatment programs are achieving positive results
or if its resources are being used effectively. Focusing more on outcomes involves
taking such steps as streamlining uniform outcome data collection, implementing
minimum performance goals, and establishing performance incentives and penalties
to help ensure that goals are met.
Office of the Auditor General
page 31
FINDING 2
Division monitors process, not treatment outcomes—The Division’s
oversight of the behavioral health treatment system is limited to a number of
process measures and is not focused on treatment results. For example, as
required by its Arizona Health Care Cost Containment System (AHCCCS) contract,
on a quarterly basis, the Division reviews the RBHAs’ ability to provide timely and
appropriate services to consumers and coordination of care with the consumer’s
primary care physician. Such measures are required for all Medicaid-covered
services and provide information that is relevant to assessing whether consumers
receive services, but they do not provide meaningful insight as to whether
consumers are reducing their dependence on alcohol or drugs.
The Division gathers consumer outcome data but does not appear to use it for
monitoring purposes. The Division gathers outcome data such as information on
abstinence and participation in employment or education when developing its
Annual Report on Substance Abuse Treatment Programs, and when applying for
the annual federal Substance Abuse Prevention and Treatment block grant. In
addition, the Division reported that it uses outcome data to compare Arizona’s to
other states’ outcomes. Division policies state that this information allows for the
measurement of behavioral health outcomes, is required of all consumers, and is
used to support quality and utilization management activities. However, auditors
found no indications that the Division uses this information to evaluate RBHA
performance in providing effective substance abuse treatment.
The Division also does not use this outcome information to compare performance
between RBHAs or between individual providers. As a result, the Division is unable
to effectively evaluate system performance and determine which providers’
treatment services are resulting in positive consumer outcomes. Likewise, the
Division is unable to identify underperforming providers and require that they make
necessary improvements. Similarly, at the RBHA level, some RBHA representatives
said that although they are required to gather the outcome data and submit it to
the Division, they do not evaluate the outcome information as thoroughly as they
could because they need more guidance from the Division on how this should be
done. Therefore, the Division should provide more guidance to the RBHAs to help
ensure they can thoroughly evaluate the consumer outcome information.
Division should continue efforts to streamline uniform outcome data
collection—The Division is revising its assessment process, and in doing so it
needs to ensure the new process enables it to uniformly collect the data needed
to assess consumer outcomes. The Division requires RBHAs to gather consumer
outcome data during intake, update, and disenrollment assessments. Historically,
according to division officials, providers have used a uniform core assessment tool
mandated by the Division, but some have expressed concern about the length of
the 32-page tool and the more than 1 hour that it took to fill it out. The tool, which
is used for all the Division’s consumers, includes a full behavioral health
assessment, not just a substance abuse assessment. During the audit, the
Division began developing a new method for collecting data that would streamline
the data collection process.
The Division gathers
consumer outcome data
but does not use it for
monitoring purposes.
State of Arizona
page 32
Because A.R.S. §36-2006(A) and (B)(4) require the Department to establish a
standardized screening assessment for alcohol and other drug education and
treatment programs, and because consistent outcome information is important for
monitoring program success, the Division should continue in its efforts to
streamline uniform collection of required outcome data. Another state has
developed uniform requirements that are short, and the Division may be able to
adopt its solution. Maine has developed a one-page form specifically for gathering
consumer outcome information during assessments that includes standardized
yes-or-no questions or provides a list of the answer options for each question.
Adopting a similar form or modifying its existing method would address providers’
concerns about the form’s length while retaining the Division’s ability to effectively
and uniformly collect, monitor, and easily validate outcome data as well as allowing
it to comply with statute.
Division should establish minimum outcome-based performance
goals, incentives, and penalties—In 2006, a national policy panel
convened to examine how state governments could be most effective in
preventing and treating substance use disorders and problems, and reported that
states should hold agencies and contracted providers accountable for meeting
identified outcome measures and should reward those that meet or exceed
outcome targets and penalize those that consistently fail.1 As discussed in Finding
1 (see pages 9 through 29), three states—Maine, Delaware, and Colorado—
reported having established performance goals for providers on a variety of
measures such as treatment retention, continuation of care, and abstinence.
Delaware and Maine have both set a minimum number of consumers that must
meet the goals while Colorado requires that providers
maintain or improve their performance without specifying
an exact number of consumers. However, a Colorado
Division of Behavioral Health official reported that it will
modify the performance goals in fiscal year 2010 to
require that the providers obtain certain percentages
rather than perform better than the average of their
previous scores for 2 years. In addition, Colorado has
established goals for consumer transition from
detoxification to some other form of treatment (see
textbox). Further, a fourth state, Nevada, is currently
conducting a 3-year pilot program that includes
performance goals. By contrast, although the Division has
some contractual requirements regarding provider
performance, it has not established benchmarks for
clinical performance, or outcome goals that are specific
to substance abuse treatment.
1 Rosenbloom, Leis, Shah, & Ambrogi, 2006
Office of the Auditor General
page 33
Examples of Colorado’s performance
goals
Maintain or improve the percentage of detox
clients who are referred to further treatment
Maintain or improve the percentage of detox
clients who enter any non-detox treatment
setting within 30 days of their detox discharge
Maintain or improve the percentage of
treatment clients who stayed in outpatient
treatment longer than 90 days
Maintain or improve the average of treatment
clients who had a reduction in primary drug
use at discharge
Source: Colorado’s fiscal years 2006 through 2008 state-wide treatment
performance measures for all managed service organizations
provided by an official from Colorado’s Department of Human
Services, Division of Behavioral Health.
In addition to performance goals, two states—Delaware and Maine—have
incorporated financial incentives and disincentives into their performance goals
that could serve as models for Arizona. For example, Delaware has established
incentives for criteria such as patient participation and treatment
retention/completion. For patient participation, the provider receives a 1 percent
bonus if 50 percent of participating patients attend two sessions per week in the
first 30 days of treatment. An incentive payment is also tied to completion, active
participation for 60 days, achievement of the major goals of the treatment plan,
and 4 consecutive weeks with clean urine samples. Maine also includes incentives
and disincentives in its performance-based contract, which stipulates that their
providers can collect a percentage of their quarterly payment each quarter they
exceed their performance goals. Likewise, its providers can lose a percentage for
each underperforming quarter.
Division needs to establish benchmarks first, then incentives—The
Division is not yet in a position to do what other states have done because it lacks
clear benchmarks for measuring performance and incentives to encourage good
performance. However, the Division has some contractual requirements regarding
provider performance and financial incentives. For example, Magellan, the RBHA
that oversees behavioral health services in Maricopa County, is required by
contract to implement financial incentives with its service providers to increase
performance on the National Outcome Measures by September 2009. According
to a Magellan official, the RBHA is working with its providers to implement
performance goals and to put a process in place for awarding incentives or levying
penalties. Other RBHAs’ contracts state that the RBHA will share incentives
originating from the Division with applicable providers. However, before
performance incentives, or penalties, can be levied, the Division must first establish
performance benchmarks or standards for treatment retention and completion
and consumer continuation of treatment from detoxification to some other form of
treatment and modify its contracts with the RBHAs to require such measures.
According to a division official, the Division is exploring possible benchmarks for
performance incentives or penalties, but difficulties in collecting and analyzing the
necessary data have hampered efforts in establishing benchmarks. In April 2009,
the Division re-organized its data management staff in order to address the data
collection and analysis issues. The Division should encourage the RBHAs to
consider contractually implementing a financial or nonfinancial method of
incentivizing substance abuse treatment providers who exceed the goals
established in their contracts and penalizing those providers who continually fail to
meet these standards related to consumer outcomes, treatment retention, and
treatment completion.
Magellan is required by
contract to implement
financial incentives with
its service providers to
increase performance
by September 2009.
State of Arizona
page 34
Division should monitor monies spent on treatment
The second area in which improved monitoring is needed is more reviews of service
costs. Reviews of over- and underutilization of healthcare services are important to
help contain costs and manage quality of substance abuse treatment. Although the
Division contractually requires the RBHAs to perform such reviews of services, the
reviews should be expanded to identify unusually high and low costs of services to
individual consumers and high treatment or services costs at each RBHA, and
monitor variations in treatment across all RBHAs. The Division should continue and
expand on the steps it took during the audit to begin reviewing service costs. In
addition, the Division should also consider how to best use assessment and case
management to contain costs. Lastly, the Division should continue its efforts to fill
vacant positions and should perform follow-up work to ensure that the restructuring
it initiated in April 2009 results in improved oversight.
Utilization reviews can help contain healthcare costs and maintain
quality of care—In a managed care system like Arizona’s, utilization reviews
are critical to help control service costs while ensuring that quality of care remains
high (see textbox). Because the Division uses a capitated system to
pay RBHAs in advance for providing all required services, current-year
costs are not reflected in current-year payments as they would be in a
fee-for-service system. Rather, each year’s capitation rate is
established based on analyzing actual costs of services delivered in
previous years, so future payments are affected by current-year
service costs. Research has shown that the cost-containment goals of
managed care systems can be obtained without sacrificing client
outcomes. However, attention must be paid to both cost-containment
strategies and quality-of-care issues (such as evidence-based
practices) if clients are to benefit from substance abuse treatment.
Specifically, 2005 and 2006 studies that compared one California county that used
managed care to two other counties that used a more traditional fee-for-service
approach to substance abuse treatment found that the managed care approach
was more cost-effective—it achieved the same results at a lower cost.1 The
managed care county established various best practices to manage both cost and
promote better outcomes including clear assessment guidelines, case
management guidelines for complex cases, continuum of care requirements,
performance-based contracting, and utilization management guidelines. For
example, the utilization management guidelines included a standard 45-day length
of stay for stabilization in residential treatment, followed by outpatient rehabilitation,
and a standard 90-day outpatient treatment with intensity guidelines for the
number of sessions.
1 Beattie, Hu, Li, & Bond, 2005; Beattie, McDaniel, & Bond, 2006
Office of the Auditor General
page 35
Utilization review—A process to
evaluate and approve or deny
healthcare services, procedures, or
settings based on medical necessity,
appropriateness, effectiveness, and
efficiency.
Source: AHCCCS Medical Policy Manual Chapter
1000—Medical Management/Utilization
Management.
The Division’s contractual requirements for utilization reviews cover nine
dimensions identified in the AHCCCS utilization management policy (see textbox),
and RBHAs include all nine dimensions in their utilization management plans. As
outlined in these plans, RBHAs review a wide range of issues, including data on
readmissions, average length of stay, and pharmacy utilization, such as monitoring
for multiple medicines taken to treat the same illness. By monitoring these
measures of care, the RBHAs have information that would allow them to help
control costs and identify potential issues with quality of care as illustrated in the
following example:
In August 2008, one RBHA, Community Partnership of Southern Arizona
(CPSA), conducted a focus review of a psychiatric health facility. In that 10-
case file review, CPSA found that there was little evidence to support that the
facility provided active treatment to the consumers, including three consumers
who required substance abuse treatment. As a result, CPSA made
recommendations regarding treatment planning documentation and group
therapy improvement. A CPSA official reported that CPSA will follow up with
the facility in August 2009 to ensure that it implemented these
recommendations.
State of Arizona
page 36
AHCCCS medical/utilization management policy requirements
• Utilization Data Analysis and Data Management—Collect, monitor, analyze, evaluate, and report
utilization data to detect both underutilization and overutilization of services.
• Concurrent Review—Review the medical necessity for a planned institutional admission or
ongoing institutional care.
• Prior Authorization—Make medical decisions regarding the initial authorization of services or
requests for continuation of services in specified time frames.
• Retrospective Review—Provide policies, procedures on how to conduct reviews to determine
medical necessity post delivery of services.
• Adoption and Dissemination of Practice Guidelines—Adopt and disseminate practice
guidelines that are based on valid and reliable clinical evidence or a consensus of healthcare
professionals and include a thorough review of medical journals’ peer-reviewed articles
published in the United States when national practice guidelines are not available.
• New Medical Technologies and New Uses of Existing Technologies—Develop and implement
written policies and procedures for evaluating new technologies and new uses of existing
technologies on both an individual basis and systemic basis.
• Case Management/Care Coordination—Ensure that a member’s needs are appropriately met
through a coordination of cost-effective care involving early identification of health risk factors
or special care needs whether in an acute, home, chronic, or alternative setting.
• Disease/Chronic Care Management—Implement a program that focuses on members with high
risk and/or chronic conditions to increase member self-management and improve providers’
practice patterns and thereby improve members’ healthcare outcome.
• Drug Utilization Review—Review the prescribing, dispensing, and use of medications to ensure
that the medications are clinically appropriate, safe, and cost-effective drug therapy that
improves a member’s health status and quality of care.
Source: AHCCCS Medical Policy Manual Chapter 1000—Medical Management/Utilization Management.
Expanding existing reviews can help manage costs—Although current
utilization reviews contain many elements that could potentially help manage
costs, auditors identified several ways to improve their ability to do so. These
include the following:
Conducting reviews of high- and low-cost substance abuse treatment
cases—Auditors’ analysis of division data identified 14 consumers who
received substance abuse treatment valued at a total of over $100,000 each
during fiscal years 2006 through 2008. The Division was not aware of these
cases until auditors brought them to officials’ attention. In one of these cases,
presented in Finding 1 as Rodney (see Finding 1, pages 9 through 29), the
consumer repeatedly used medical detoxification services valued at a total of
over $82,000 in fiscal years 2006 through 2008.
Rodney’s stay at a Tucson detoxification center in September 2008 was typical
of his pattern. He was brought to a hospital emergency room one evening by
emergency medical services. At triage, he had a blood alcohol level of .396.
The center provided Ativan, a medication for withdrawal symptoms; fed him;
and applied a nicotine patch because the hospital is a nonsmoking campus.
A registered nurse monitored him periodically while he slept. Rodney
transferred to the detoxification center the following day. When the center
conducted an assessment to determine his readiness for treatment to
address his addiction, he said, “I’m not sure what I want to do after detox. I’ve
never been to treatment, so I can’t say if I would like to go there.” Rodney
discharged himself around noon on the third day, against medical advice.
Auditors’ analysis of division data found 54 separate division data records for
detoxification services for Rodney over the 3-year period analyzed, and a case
manager at the center reported that Rodney used detoxification three or four
times a month.
Cases such as Rodney’s involve high costs, but appear to produce limited
outcomes. Altogether, for fiscal years 2006 through 2008, approximately $100
million, or 72 percent of the total encounter value (reported value of services)
for substance abuse treatment, was used to provide services to 20 percent of
the consumer population, and 45 percent of these consumers did not
complete treatment. Reviewing high-cost cases such as Rodney’s could
enable the Division to work with the RBHAs and providers to find ways to treat
consumers more effectively and break the cycle of substance abuse and
detoxification.
During the audit, in March 2009, the Division began reviews that focused on
service costs for consumers with serious mental illness. Specifically, the
Division reviewed Magellan’s fiscal year 2008 healthcare costs for consumers
with serious mental illness and identified the top ten service users as
Office of the Auditor General
page 37
measured by service costs and 49 other clients who were enrolled in fiscal
year 2008 but did not receive services that year, most of whom also had not
received services in fiscal year 2009. Low-dollar service users could indicate
potential quality of care issues or an inadequate disenrollment process. The
Division presented the information to Magellan officials, who replied that the
top ten service users were very complex cases involving safety concerns.
Magellan added that it determined that all services provided to nine of the ten
consumers were clinically appropriate and, as a result, would conduct an in-depth
review of the clinical appropriateness of the services offered to the one
other consumer. Regarding consumers who underutilized services, Magellan
reported that they found an opportunity to improve timeliness of closures and
follow up after crisis episodes. As a result, Magellan reported that it would
implement a claims inactivity report to identify consumers who did not have
encounters (service records) for periods of more than 210 days. However, this
review was limited to one RBHA and consumers with serious mental illness.
To more effectively control costs, the Division needs to continue and expand
these efforts. The Division should perform similar reviews for consumers
receiving treatment for substance abuse in order to identify, research, and
appropriately address all high- and low-dollar service users. In addition, the
Division should follow the example of the California managed-care county
discussed on page 35 and establish benchmarks for length of stay, and
collect and monitor data on length of stay in order to identify consumers who
may be over- or under-utilizing certain types of services, which could indicate
they need a different service to be treated cost-effectively.
Conducting cost-focused reviews of specific treatments or services—In
addition to identifying and addressing high- and low-dollar service users, the
Division should also focus reviews on managing high-cost treatment or
service types. Conducting such reviews would enable the Division to
determine not only if the services are necessary, but help identify other
effective, potentially less-costly treatments. In addition to reviewing the
highest-cost consumers, auditors identified other consumers in various
service categories such as transportation, inpatient, and outpatient treatment.
Auditors specifically looked for cases with costs significantly higher than the
median for that service category. Using division data, auditors identified
consumers whose fiscal years 2006 through 2008 treatment values ranged
from $50,000 to $100,000, and found that the costs of some of the specific
services these consumers received were 15 times more than the median
costs of such services. For example:
• Ron, 52, has an opiate dependence that requires supervised methadone
treatment. According to his treatment center’s clinical director, although
Ron lives in Lake Havasu City, his treatment provider transports him twice
a week over 180 miles round-trip to Bullhead City, which the clinical
State of Arizona
page 38
director added is the only authorized methadone treatment center in rural
Mohave County. His treatment center’s clinical director also explained
that because of federal methadone regulations and licensing
requirements, it is very difficult to incentivize other healthcare facilities that
may be closer to the consumer to become methadone treatment
facilities.1 Therefore, his only option is to transport the client to the
Bullhead City Community Medical Services II center at a value over fiscal
years 2006 through 2008 of almost $48,000, which represents more than
87 percent of the total value of Ron’s treatment for that time period. This
amount greatly exceeds the RBHA’s median transportation value of
approximately $88.
Examining costs such as this can help the Division identify high-cost practices
and examine alternatives for reducing them. Methadone treatment is one area
in which the Division has already begun to study alternatives. From fiscal years
2006 to 2008, approximately 2,600 consumers, or about 5 percent of all
substance abuse consumers served, received methadone treatment valued
at more than $9.5 million. According to a division official, the Division is
transitioning to a new treatment, buprenorphine, which the consumer can take
home and administer daily. In addition, in an effort to reduce transportation
costs for consumers receiving methadone treatment, in February 2009, the
Northern Arizona Regional Behavioral Health Authority (NARBHA) approved
the purchase of a van to provide such services instead of paying a third-party
vendor to do so. To help contain costs, such as transportation costs
associated with methadone treatment, the Division and RBHAs should
continue their efforts to transition to alternative treatments. In addition, all
RBHAs and the Division should identify consumers with higher-than-usual
costs for specific services, and alternative methods or treatments that would
provide the same quality of care at a reduced cost.
Comparing variation in treatment types among RBHAs—Finally, to better
monitor ser
Object Description
| Rating | |
| TITLE | Performance audit, department of health services division of behavioral health services substance abuse treatment programs: report to the Arizona Legislature |
| CREATOR | Office of the Auditor General |
| SUBJECT | Arizona Department of Health Services--Division of Behavioral Health Services--Auditing |
| Browse Topic |
Government and politics |
| DESCRIPTION | This title contains one or more publications |
| Language | English |
| Contributor | Arizona Department of Health Services |
| Publisher | Office of the Auditor General |
| TYPE |
Text |
| Material Collection | State Documents |
| Acquisition Note | Report No. 09-07 |
| Source Identifier | LG 6.2:R 36 |
| Location | o430059294 |
| REPOSITORY | Arizona State Library, Archives and Public Records--Law and Research Library |
Description
| TITLE | Performance audit, department of health services division of behavioral health services substance abuse treatment programs: report to the Arizona Legislature |
| DESCRIPTION | 92 pages (PDF version). File size: 1500 KB |
| TYPE |
Text |
| RIGHTS MANAGEMENT | Copyright to this resource is held by the creating agency and is provided here for educational purposes only. It may not be downloaded, reproduced or distributed in any format without written permission of the creating agency. Any attempt to circumvent the access controls placed on this file is a violation of United States and international copyright laws, and is subject to criminal prosecution. |
| DATE ORIGINAL | 2009-07 |
| Time Period |
2000s (2000-2009) |
| ORIGINAL FORMAT | Born Digital |
| Source Identifier | LG 6.2:R 36 |
| Location | o430059294 |
| DIGITAL IDENTIFIER | 09-07.pdf |
| DIGITAL FORMAT | PDF (Portable Document Format) |
| REPOSITORY | Arizona State Library, Archives and Public Records--Law and Research Library. |
| File Size | 1535955 Bytes |
| Full Text | Debra K. Davenport Auditor General Performance Audit Department of Health Services Division of Behavioral Health Services— Substance Abuse Treatment Programs Performance Audit Division July • 2009 REPORT NO. 09-07 A REPORT TO THE ARIZONA LEGISLATURE The is appointed by the Joint Legislative Audit Committee, a bipartisan committee composed of five senators and five representatives. Her mission is to provide independent and impartial information and specific recommendations to improve the operations of state and local government entities. To this end, she provides financial audits and accounting services to the State and political subdivisions, investigates possible misuse of public monies, and conducts performance audits of school districts, state agencies, and the programs they administer. The Joint Legislative Audit Committee Audit Staff Copies of the Auditor General’s reports are free. You may request them by contacting us at: Office of the Auditor General 2910 N. 44th Street, Suite 410 • Phoenix, AZ 85018 • (602) 553-0333 Additionally, many of our reports can be found in electronic format at: www.azauditor.gov Melanie M. Chesney, Director Shan Hays, Manager and Contact Person Anne Hunter, Team Leader Kerry Howell Karl Kulick Senator Thayer Verschoor, Chair Representative Judy Burges, Vice-Chair Senator Pamela Gorman Representative Tom Boone Senator John Huppenthal Representative Cloves Campbell, Jr. Senator Richard Miranda Representative Rich Crandall Senator Rebecca Rios Representative Kyrsten Sinema Senator Bob Burns (ex-officio) Representative Kirk Adams (ex-officio) 2910 NORTH 44th STREET • SUITE 410 • PHOENIX, ARIZONA 85018 • (602) 553-0333 • FAX (602) 553-0051 WILLIAM THOMSON DEPUTY AUDITOR GENERAL DEBRA K. DAVENPORT, CPA AUDITOR GENERAL STATE OF ARIZONA OFFICE OF THE AUDITOR GENERAL July 30, 2009 Members of the Arizona Legislature The Honorable Janice K. Brewer, Governor Will Humble, Interim Director Department of Health Services Transmitted herewith is a report of the Auditor General, a Performance Audit of the Department of Health Services, Division of Behavioral Health Services—Substance Abuse Treatment Programs. This report is in response to an October 5, 2006, resolution of the Joint Legislative Audit Committee. The performance audit was conducted as part of the sunset review process prescribed in Arizona Revised Statutes §41-2951 et seq. I am also transmitting with this report a copy of the Report Highlights for this audit to provide a quick summary for your convenience. As outlined in its response, the Department of Health Services agrees with all of the findings and plans to implement all of the recommendations. My staff and I will be pleased to discuss or clarify items in the report. This report will be released to the public on July 31, 2009. Sincerely, Debbie Davenport Auditor General Attachment The Office of the Auditor General has conducted a performance audit of the substance abuse treatment programs provided by the Department of Health Services (Department), Division of Behavioral Health Services (Division), pursuant to an October 5, 2006, resolution of the Joint Legislative Audit Committee. This is the second audit in a series of three reports on the Department and was conducted as part of the sunset review process prescribed in Arizona Revised Statutes (A.R.S.) §41-2951 et seq. This audit focuses on substance abuse treatment outcomes and system oversight. The first audit focused on the Division of Licensing Services, and the final report will be an analysis of the 12 statutory sunset factors. According to its Annual Report on Substance Abuse Treatment Programs, the Division spent more than $121 million for substance abuse services in fiscal year 2008. Program participants, whom the Division refers to as “consumers,” numbered more than 63,000 adults and children. Most were enrolled in the Arizona Health Care Cost Containment System, or AHCCCS, the State’s Medicaid program. They received alcohol- and drug-related services ranging from counseling and skills training to crisis intervention and detoxification in hospitals or other inpatient facilities. The Division provides these services through contracts with four regional behavioral health authorities, or RBHAs, and three tribal regional behavioral health authorities, or TRBHAs, which contract with a network of more than 100 substance abuse treatment service providers throughout the State. This performance audit focused on the program’s outcomes—that is, the extent to which services reduced dependency on alcohol and drugs—and on the Division’s oversight of behavioral health authorities and providers. Division should focus on strategies that improve outcomes (see pages 9 through 29) Although substance abuse is difficult to treat, the Division can take steps to improve outcomes for individuals who participate in substance abuse treatment. Auditors analyzed 3 years of data related to four measures commonly used to evaluate program effectiveness—extent of continuing alcohol or drug use, employment, criminal activity, and homelessness. The analysis showed that outcomes related to continued alcohol and drug use were associated with the following two factors: Office of the Auditor General SUMMARY page i Deciding to abstain from using alcohol or drugs before treatment started— More than half of all consumers reported that they were abstinent when their treatment started. Within this group, more than 93 percent reported that they were still abstinent when they left the program. By contrast, most of those who reported using alcohol or drugs when they began treatment were still using these substances at about the same level when they left. About one person in every four who began treatment while still using alcohol or drugs reported diminishing his/her use of alcohol or drugs or stopping it altogether by the time he/she left treatment. Completing treatment—Overall, 58 percent of consumers did not complete their treatment. Providers lost contact with many of them, while others refused treatment or left for other reasons, but rates of continued use of alcohol or drugs varied substantially between those who completed the treatment and those who did not. For example, among consumers who reported using alcohol or drugs when they began treatment, 27 percent of those who completed their treatment reported abstinence when they left. By contrast, among consumers who reported using alcohol or drugs when they began treatment, only 17.6 percent of those consumers who left before completing their treatment reported that their use had diminished. The analysis showed little change across the three remaining performance measures—lack of recent arrests, employment, and stable housing. For example, 21 percent of consumers reported recent arrests upon entering treatment, and 18 percent reported new arrests at the time of their update or disenrollment. Similarly, 38 percent said they were employed when they entered treatment, while 41 percent reported being employed at their annual update or disenrollment. Finally, slightly more than 7 percent were homeless upon entering treatment, and slightly less than 7 percent were homeless at their annual update or disenrollment. Substance abuse is difficult to treat, and auditors’ more detailed case studies of a limited number of consumers showed that the reasons for success or failure are complicated and varied. Nonetheless, research and best practices indicate the best opportunities for increasing success rest in three main strategies: Focusing on treatment retention—Research corroborates what auditors’ analysis showed: consumers who remain longer in treatment experience better outcomes. The Division can take several steps to increase consumer retention, including establishing performance goals, monitoring completion rates, and using incentives and other case management techniques. Other states that auditors reviewed have taken such steps, and the Division may be able to adopt some of these approaches. Ensuring that consumers have access to a full range of services that can potentially be used to address their particular needs—This strategy, called continuum of care, involves incorporating appropriate types of treatment over State of Arizona page ii time and placing the consumer in more or less intensive treatment as needed. Auditors’ case studies showed that while some consumers showed good outcomes and received appropriate services, others did not necessarily receive the services and therapies that might improve the chances of good treatment outcomes. The Division can take several steps to ensure continuum of care, including collecting and monitoring data relevant to assessment, better defining case management, and working with RBHAs to improve the continuum of care when weaknesses are identified. Following practices that have been shown to carry the greatest chance of success—These evidence-based practices have been validated by observation or experience as improving treatment success. Examples include motivational interviews, which is a counseling style designed to help consumers recognize and accept the need for continued care. Although the Division requires the RBHAs to use evidence-based practices, RBHAs are not necessarily doing so, and the Division is not ensuring compliance. According to a 2008 federal grant review that focused on programs for children and adolescents, the Division had done a good job of establishing evidence-based practices in some areas but had not identified such practices across the continuum of care and could do more to ensure sustainability of the emphasis on evidence-based practices. The Division reached similar conclusions in another study of intensive outpatient programs for youth. Steps the Division can take to place greater emphasis on evidence-based practices include encouraging RBHAs to offer a wider variety of programs, monitoring compliance with its contractual requirements to use evidence-based practices, and expanding its work with the RBHAs to ensure that providers have the guidance needed to implement specific evidence-based practices. Division should improve oversight of substance abuse programs (see pages 31 through 42) The Division should take steps to improve its oversight of the substance abuse programs administered by RBHAs. These steps take two main forms: Increasing the use of information about treatment outcomes—Although the Division collects outcome information to complete certain reports, auditors found that oversight efforts focused almost entirely on process-related information, such as the timeliness of services or coordination with a consumer’s primary care physician as required by the Division’s contract with AHCCCS. As a result, the Division is largely unable to determine if its substance abuse treatment programs are achieving positive results or if its resources are being used effectively. Additionally, because the Division does not compare Office of the Auditor General page iii substance abuse outcome measures across RBHAs or providers, it cannot assess which providers’ treatment services are resulting in improved client outcomes or identify underperforming providers. Needed actions include continuing its efforts to streamline uniform outcome data collection, establishing relevant performance goals in contracts with the RBHAs, and encouraging the RBHAs to consider ways to reward providers who meet standards and penalize those who do not. Expanding utilization reviews to focus more on service costs, consumer assessments, and case management—Although its oversight efforts contain many elements that could potentially help manage costs, the Division could implement several actions that could improve its ability to do so. Greater emphasis on cost appears warranted. Auditors’ review of division data from fiscal years 2006 to 2008, for example, identified 14 substance abuse consumers with service costs over $100,000. One incurred $82,000 in medical detoxification costs, during which time he continued to drink and require detoxification three or four times a month. The Division was not aware of these cases until auditors brought them to officials’ attention. Actions needed include (1) regularly reviewing high- and low-cost substance abuse treatment cases, (2) collecting data to identify consumers who may be overutilizing or underutilizing certain types of services, which could indicate a lack of alternative forms of treatment or a need for other changes to improve treatment, (3) comparing variations in the use of types of treatment at each RBHA to see if the use of such services positively affects consumer treatment outcomes and adjusting treatment accordingly, and (4) determining how to best use assessment and case management to contain costs while maintaining quality of care. Further, to improve oversight, the Division should continue its efforts to fill vacant positions in its data systems and analysis and quality management functions, and should perform follow-up work to ensure that the restructuring it initiated in April 2009 has provided management with the information to do so. State of Arizona page iv Office of the Auditor General TABLE OF CONTENTS continued page v Introduction & Background 1 Finding 1: Division should focus on strategies that improve outcomes 9 Substance abuse difficult to treat 9 Most consumers showed little change after treatment 10 Division should increase focus on treatment retention and completion 17 Division should monitor continuum of care 21 Using appropriate evidence-based practices can improve success rate 25 Recommendations 28 Finding 2: Division should improve oversight of substance abuse programs 31 Division should monitor outcomes 31 Division should monitor monies spent on treatment 35 Recommendations 41 Appendix A: Outcome analysis methodology and results a-i Appendix B: Methodology b-i Appendix C: Bibliography c-i Agency Response TABLE OF CONTENTS continued page vi State of Arizona Table: 1 Funding Sources for Substance Abuse Services Expenditures Fiscal Year 2008 (Unaudited) 4 Figures: 1 Regional and Tribal Behavioral Health Authorities’ Geographical Service Areas (GSA) and Fiscal Year 2008 Substance Abuse Enrollment 3 2 State-wide Substance Abuse Treatment Enrollment Fiscal Years 2001 through 2008 6 3 Treatment Completion and Noncompletion Rates by RBHA (GSA) Fiscal Years 2006 through 2008 19 4 Consumer-Reported Abstinence from Substance Use By RBHA (GSA) from Admission to Annual Review or Discharge Fiscal Years 2006 through 2008 a-iv 5 Change in Substance Use Among Consumers Who Reported Substance Use Before Treatment by RBHA (GSA) From Admission to Annual Review or Discharge Fiscal Years 2006 through 2008 a-v 6 Alcohol Abstinence Status of Consumers by State (Western Region) Fiscal Year 2009 SAPT Block Grant Application (Unaudited) a-vi 7 Drug Abstinence Status of Consumers by State (Western Region) Fiscal Year 2009 SAPT Block Grant Application (Unaudited) a-vii Office of the Auditor General TABLE OF CONTENTS continued page vii Figures (continued): 8 Alcohol Abstinence Status of Consumers Arizona, Western Region, and National Averages Fiscal Year 2009 SAPT Block Grant Application (Unaudited) a-viii 9 Drug Abstinence Status of Consumers Arizona, Western Region, and National Averages Fiscal Years 2009 SAPT Block Grant Application (Unaudited) a-ix 10 Employment Status of Consumers by RBHA (GSA) From Admission to Annual Review or Discharge Fiscal Years 2006 through 2008 a-xi 11 Employment Status of Consumers by State (Western Region) Fiscal Year 2009 SAPT Block Grant Application (Unaudited) a-xii 12 Employment Status of Consumers Arizona, Western Region, and National Averages Fiscal Year 2009 SAPT Block Grant Application (Unaudited) a-xiii 13 Arrest-Free Status of Consumers by RBHA (GSA) From Admission to Annual Review or Discharge Fiscal Years 2006 through 2008 a-xv 14 Arrest-Free Status of Consumers by State (Western Region) Fiscal Year 2009 SAPT Block Grant Application (Unaudited) a-xvi 15 Arrest-Free Status of Consumers Arizona, Western Region, and National Averages Fiscal Year 2009 SAPT Block Grant Application (Unaudited) a-xvii State of Arizona TABLE OF CONTENTS concluded page viii Figures (concluded): 16 Homeless Status of Consumers by RBHA (GSA) From Admission to Annual Review or Discharge Fiscal Years 2006 through 2008 a-xix 17 Homeless Status of Consumers by State (Western Region) Fiscal Year 2009 SAPT Block Grant Application (Unaudited) a-xx 18 Homeless Status of Consumers Arizona, Western Region, and National Averages Fiscal Year 2009 SAPT Block Grant Application (Unaudited) a-xxi The Office of the Auditor General has conducted a performance audit of the substance abuse treatment programs provided by the Department of Health Services (Department), Division of Behavioral Health Services (Division), pursuant to an October 5, 2006, resolution of the Joint Legislative Audit Committee. This is the second audit in a series of three reports on the Department and was conducted as part of the sunset review process prescribed in Arizona Revised Statutes (A.R.S.) §41-2951 et seq. This audit focuses on substance abuse treatment outcomes and system oversight. The first audit focused on the Division of Licensing Services, and the final report will be an analysis of the 12 statutory sunset factors. Importance of treating substance abuse Drug and alcohol abuse are associated with some of society’s most serious and expensive problems. For example, according to literature, nation-wide: More than half of all state prison inmates were under the influence of alcohol or drugs when they were arrested.1 Nearly one in six state inmates committed crimes to support a drug addiction.2 About 20 percent of acute care Medicaid expenditures pay for alcohol- or drug-related medical costs.3 Drunk driving is a major expense for the police, courts, and emergency medical systems.4 1 U.S. Department of Justice, Bureau of Justice Statistics, 2005a as cited in Rosenbloom et al., 2006 2 U.S. Department of Justice, Bureau of Justice Statistics, 2005b as cited in Rosenbloom et al., 2006 3 U.S. Department of Health and Human Services, Substance Abuse and Mental Health Services Administration, 2005 as cited in Rosenbloom et al., 2006 4 Miller, Cox, Zaloshnja, & Taylor, 2002 Office of the Auditor General INTRODUCTION & BACKGROUND page 1 In 2003, an estimated 22.5 million people ages 12 and older nation-wide, or about 9.4 percent of the population, had a substance abuse disorder, and an estimated $21 billion was devoted to substance abuse treatment, according to a U.S. Department of Health and Human Services’ Substance Abuse and Mental Health Services Administration (SAMHSA) report dated 2007.1 According to the same report, people with substance use disorders rely on public sources of financing far more than do people with other diseases. The report also states that public sources of funding, including state and local government, Medicaid, and other federal spending, such as block grants, provided 77 percent of total substance abuse spending in 2003 but only 45 percent of all healthcare spending. The nature of addiction, such as its quality as a chronic and recurring problem, and other factors, such as age and socio-economic status, affect treatment success. Arizona, like many other states, struggles with the problem of substance abuse (see textbox). Structure and funding of Arizona’s system for providing substance abuse services The Division provides substance abuse services mainly to consumers enrolled in the Arizona Health Care Cost Containment System, or AHCCCS, which is the State’s Medicaid program. However, others can receive services to the extent funding is available, particularly if they are members of priority populations such as pregnant women. AHCCCS enrollees include adults who meet the requirements of Title XIX (Medicaid) of the Social Security Act (Act), as well as children and their families who meet the requirements of Title XXI (the federal SCHIP program, which is called KidsCare in Arizona) of the Act. Publicly funded substance abuse treatment and prevention services in Arizona are provided primarily through the behavioral health system administered by the Division.2 The Division provides these services through contracts with four regional behavioral health authorities, or RBHAs, and three tribal regional behavioral health 1 Mark et al., 2007 2 Incarcerated individuals may receive publicly funded substance abuse services through the Department of Corrections or the county jails. State of Arizona page 2 Prevalence and impacts of substance abuse in Arizona Altogether, an estimated 466,000 Arizonans ages 12 and older were dependent on or abused alcohol, and an estimated 137,000 were dependent on or abused illicit drugs, according to the 2005 National Survey on Drug Use and Health. Drug- and alcohol-related deaths, including those attributable to motor vehicle and boating fatalities, accounted for nearly 1,700 deaths in Arizona in 2005. There were significant increases in the number of alcohol and drug-induced deaths in Arizona between 2000 and 2005. The rates of drug-induced deaths more than doubled between 2000 and 2005, and the number of alcohol-induced deaths increased by more than one-third for the same time period. A 2008 study of adults arrested in Maricopa County in 2007 found that over one-third of arrestees interviewed for the study tested positive for methamphetamine use, with over 40 percent of female arrestees testing positive. Source: The Substance Abuse Epidemiology Work Group’s 2007 Arizona Statewide Substance Abuse Epidemiology Profile, and Arizona State University, Center for Violence Prevention and Community Safety’s Arizona Arrestee Reporting Information Network Annual Adult Report 2007. authorities, or TRBHAs, which contract with a network of more than 100 substance abuse treatment service providers within six geographical service areas, or GSAs, throughout the State (see Figure 1).1 During fiscal year 2008, over 63,000 adults and children participated in substance abuse treatment programs offered through the RBHAs and TRBHAs and their providers throughout the State. Magellan, the RBHA that serves Maricopa County, served the largest proportion—38 percent—of individuals receiving substance abuse treatment in fiscal year 2008, followed by Community Partnership of Southern Arizona (CPSA) at 32 percent, Northern Arizona Behavioral Health Authority (NARBHA) at 15 percent, and Cenpatico Behavioral Health of Arizona, the three tribal regional behavioral health authorities, and the Navajo Nation serving the remaining approximately 15 percent. The Navajo Nation serves approximately 2 percent of individuals receiving substance abuse treatment. According to the Division’s Annual Report on Substance Abuse Treatment Programs, in fiscal year 2008 approximately $121.2 million, or approximately 11 percent of the Division’s estimated $1.1 billion budget, was expended for substance abuse services.2 The majority of this funding came from Title XIX/XXI monies, with additional funds provided through the federal Substance Abuse Prevention and Treatment (SAPT) block grant, state appropriations, and other funding sources (see Table 1, page 4). Budget reductions in fiscal year 2009 will probably affect substance abuse treatment. In addition, as of July 16, 2009, the Department’s 2010 budget had not been determined. According to the Division, a total of $2.5 million of appropriated funds that are used for substance abuse services was cut from its fiscal year 2009 budget. Office of the Auditor General page 3 1 Magellan replaced ValueOptions as the RBHA for Maricopa County on September 1, 2007. Source: Auditor General staff analysis of the Arizona Department of Health Services, Division of Behavioral Health Services’ Annual Report on Substance Abuse Treatment Programs for fiscal year 2008 and division-provided enrollment data. Figure 1: Regional and Tribal Behavioral Health Authorities’ Geographical Service Areas (GSA) and Fiscal Year 2008 Substance Abuse Enrollment Northern Arizona Regional Behavioral Health Authority Cenpatico Behavioral Health of Arizona Magellan1 Cenpatico Behavioral Health of Arizona Community Partnership of Southern Arizona Community Partnership of Southern Arizona Tribal Regional Behavioral Health Authorities 1 In addition to its three TRBHA contracts, the Division also contracts with two tribal nations—the Navajo Nation and the Colorado River Indian Tribes—to deliver substance abuse treatment services to reservation residents. 2 The total amount of funding allocated to the RBHAs in fiscal year 2008 was $989.4 million, including monies for serving children, adults with serious mental illness, and people with co-occurring mental health and substance abuse disorders, as well as providing general mental health and substance abuse prevention and treatment services. In addition, the Division reported fund sweeps and appropriations reductions totaling almost $1.1 million in substance abuse fund monies. These monies come mainly from fees and fines collected from criminal offenses and are used to provide alcohol and drug screening, education, and treatment for individuals who are court-ordered to attend, but do not have sufficient ability to pay. Because of these reductions, the Division expects that the number of consumers served in fiscal year 2009 will be less than the number served in fiscal year 2008. In addition, to offset some of the budget reductions, the Division reported that it is updating its policy to require co-payments for consumers who are not eligible for Title XIX and whose income is a certain percentage above the federal poverty level. On April 3, 2009, a draft version of the policy regarding co-payments was released for public comments, and as of July 16, 2009, the policy had not been finalized.1 Funding is allocated to the RBHAs through a capitated payment arrangement whereby the RBHA is provided a set amount of funding each month for each consumer in its geographic service area who is enrolled in AHCCCS. In order to allow the Division to monitor services and determine funding needs, the RBHAs submit service data called encounters (see textbox) to the Division. The Division’s contracts with the RBHAs require them to submit encounters whose total value equals at least 85 percent of the total service revenue payments they receive on an annual basis. Substance abuse services provided The Division provides many different types of substance abuse services. In fiscal years 2006 through 2008, the RBHAs reported that they provided services valued at $139.3 million to substance abuse treatment consumers, excluding substance abuse services to other consumers such as adults with serious mental illness, children, and adolescents. The largest service category for those consumers, 1 Specific changes to the policy include: 1) consumers with serious mental illness are no longer excluded from the co-payment requirement, 2) the types of services excluded from the co-payment requirement were modified, and 3) providers are allowed to refuse service as well as terminate services to a consumer for non-payment of a co-payment. State of Arizona page 4 Encounter—Record of a service delivered to a consumer by a provider. Includes specifics about the type and date of service, provider that delivered the service, and dollar value of the service. Funding Source Amount Percentage Title XIX/XXI funding $ 75,941,357 62.66% Substance Abuse Prevention and Treatment Block Grant 23,545,206 19.43 State appropriations 16,518,610 13.63 Intergovernmental agreements (IGA) and interagency service agreement (ISA)1 5,141,081 4.24 Liquor fees2 45,325 0.04 Total $121,191,579 100.00% Table 1: Funding Sources for Substance Abuse Services Expenditures Fiscal Year 2008 (Unaudited) 1 In fiscal year 2008 the Division had IGAs with Maricopa County and the City of Phoenix, and an ISA with the Arizona Department of Corrections, Correctional Officer/Offender Liaison program (the COOL program). Maricopa County and the City of Phoenix provide monies to help operate the Local Alcohol Rehabilitation Center, a detoxification center in Phoenix. The COOL program was eliminated by the Department of Corrections in 2009. However, in 2008 the program provided transitional services for persons with substance abuse issues who were leaving the penal system. The services included assistance in finding housing, employment, and behavioral health services. 2 Liquor fees are fees collected from the issuance of special event liquor licenses. Source: Auditor General staff analysis of the Arizona Department of Health Services, Division of Behavioral Health Services’ Annual Report on Substance Abuse Treatment Programs for fiscal year 2008. treatment services, accounted for approximately 31.5 percent of the total.1 The types of services were as follows: Treatment services ($43.9 million)—Individual and group counseling, therapy, assessment, evaluation, screening, and other professional services. Residential services ($26.9 million)—Twenty-four-hour residential services, including structured treatment, which includes room and board, delivered in residential facilities or supported independent living settings. Support services ($25.3 million)—Case management, peer support services, and transportation. Medical and pharmacy services ($19.0 million)—Medication to assist with alcohol or drug withdrawal or with co-occurring disorders. Inpatient ($9.5 million)—Inpatient detoxification and treatment services delivered in hospitals and other inpatient facilities, including residential treatment centers that provide a structured treatment with 24-hour supervision, an intensive treatment program, and on-site medical services. Crisis intervention services ($5.7 million)—Crisis intervention/stabilization services provided in the community, hospitals, and residential treatment centers. Rehabilitation services ($4.9 million)—Education, coaching, training, and other services, including securing and maintaining employment. Services include living skills training, cognitive rehabilitation, health promotion, and ongoing support to help maintain employment. Day programs ($4.1 million)—Skills training and development, behavioral health prevention/promotion, medication training and support, ongoing support to maintain employment, and self-help/peer services to improve consumers’ ability to function in the community. The cost of services varies by type of treatment. For example, auditors’ analysis of encounter data from fiscal years 2006 through 2008 revealed that the average cost per person, per year, for residential treatment was $1,500 and the average cost of crisis intervention per person, per year, was $240. Because some consumers may receive several different types of treatment or receive treatment for longer periods or at several different times during the year, the total cost per person for all services received varies widely, with a few consumers receiving services valued at more than Office of the Auditor General page 5 1 The figures reported in these bullets reflect auditors’ analysis of services reported for consumers whose primary reason for enrollment was substance abuse. These figures differ from the total substance abuse funding of $121 million shown in Table 1 (see page 4) because these figures cover a 3-year period and do not include substance abuse services provided to other consumers. Further, because the Division uses a capitated system to pay RBHAs in advance for providing all required services, current-year costs are not reflected in current-year payments as they would be in a fee-for- service system. Therefore, the total substance abuse funding is not the same amount as the value of the services provided as reported by the RBHAs for a given time period. $100,000 per year (See Finding 2, pages 31 through 42, for additional information on high-utilization consumers). The median cost of treatment per person for fiscal years 2006 through 2008 was $1,090.1 Consumers served Generally, to obtain substance abuse services through the RBHAs or TRBHAs a person must be enrolled in AHCCCS. According to the Division’s Annual Report on Substance Abuse Treatment Programs, in fiscal year 2008, 74 percent of adults and children receiving substance abuse services, or approximately 47,000 individuals, were AHCCCS eligible. Substance abuse services for non-Title XIX/XXI-eligible individuals and families, also delivered by RBHA- and TRBHA-contracted providers, are provided only if monies are available, and these services are provided first to people in priority populations. The priority populations are spelled out by grant requirements and other funding sources. For example, the federal SAPT block grant designates priority populations as pregnant women, women with dependent children, and intravenous drug users. In fiscal year 2008, 1,139 pregnant women and 6,238 women with children received substance abuse services. In addition, in fiscal year 2008, 4,033 people were tested for human immunodeficiency virus (HIV), with 24 identified as being HIV positive and receiving counseling. From fiscal year 2001 to fiscal year 2008, the number of people receiving some type of state-provided substance abuse treatment increased by nearly 300 percent, from 16,115 to 63,571 (see Figure 2). The Division attributes this growth primarily to expansion in AHCCCS eligibility requirements. For example, in fiscal year 2000, Arizona voters approved Proposition 204, expanding eligibility for AHCCCS starting in 2001. The proposition expanded income eligibility requirements up to 100 percent of the federal poverty level. In fiscal year 2008 nearly all consumers receiving substance abuse treatment services through the RBHAs and TRBHAs and their providers were adults, most were non-Hispanic, and alcohol was the most commonly used substance (see textbox, page 7). That year, according to its Annual State of Arizona page 6 16,115 32,772 41,172 36,375 59,661 60,105 65,696 63,571 0 10,000 20,000 30,000 40,000 50,000 60,000 70,000 2001 2002 2003 2004 2005 2006 2007 2008 Enrollment Fiscal Year Figure 2: State-wide Substance Abuse Treatment Enrollment Fiscal Years 2001 through 2008 Source: Auditor General staff analysis of the Arizona Department of Health Services, Division of Behavioral Health Services’ Annual Report on Substance Abuse Treatment Programs for fiscal years 2007 and 2008. 1 Median cost per person was calculated using demographic and encounter data from the Division’s Client Information System for fiscal years 2006 through 2008 for adult substance abuse consumers without a serious mental illness. Report on Substance Abuse Treatment Programs, the Division provided substance abuse treatment services to 49,751 adults without a serious mental illness (78 percent of substance abuse treatment consumers), 10,071 adults who had serious mental illness with a co-occurring substance abuse disorder (16 percent), and 3,929 children or adolescents (6 percent). Consumer characteristics are important in planning treatment and can affect treatment success. For example, women with children need specialized help or services, such as childcare. Consumers entered treatment through a variety of referral sources, with nearly half seeking treatment on their own or on the advice of a friend or family member. Referral by court order or involvement with a criminal justice agency varies regionally. In rural areas, court or criminal justice referrals are as high as 40 percent of the treatment-seeking population. For example, in fiscal year 2007, in the primarily rural counties of Yuma and La Paz, 41 percent of people enrolled in substance abuse treatment were referred by a court or a probation/parole officer. Office of the Auditor General page 7 Consumer Characteristics Fiscal Year 2008 Age: •94% adults; 6% adolescents •50% of adults were aged 25 to 44 Gender: •56% male; 44% female Diagnosis: •78% were adults with a substance abuse disorder •16% had serious mental illness with a co-occurring substance abuse disorder Race: •86% Caucasian •7% African American •5% Native American •2% other or multi-race Ethnicity: •26% Hispanic/Latino •74% non-Hispanic/Latino Primary •37% alcohol substance: •26% stimulants including methamphetamine and cocaine/crack •24% marijuana •11% narcotics including heroin •2% other Referral •45% self, friend, or family source:1 •23% court order or criminal justice agency •11% other or unknown •10% other behavioral health providers •6% other state agencies •3% community agencies •3% Health Plan or primary care physician 1 Numbers do not total 100 percent because of rounding. Source: Auditor General staff analysis of the Arizona Department of Health Services, Division of Behavioral Health Services’ Annual Report on Substance Abuse Treatment Programs for fiscal year 2008. Substance abuse program oversight The Division is primarily an oversight agency that monitors all aspects of behavioral health services, including substance abuse treatment services, through contract compliance and other mechanisms. For example, the Division’s contract compliance unit conducts administrative reviews, as required by the Division’s contract with AHCCCS, that examine the RBHAs’ compliance with state and federal requirements, program operations, fiscal operations, and financial status. Although these reviews encompass overall behavioral health, including substance abuse, they do not focus on substance abuse. Additionally, various units within the Division are responsible for overseeing different aspects of the substance abuse program, as well as the general mental health program and co-occurring disorders. For example, the grants, clinical practice improvement, interagency coordination units, and office of the medical director all have responsibilities that involve some oversight of general mental health treatment, substance abuse treatment, and treatment for co-occurring disorders. The Division has one full-time equivalent position, Lead Substance Abuse Clinical Advisor, that is dedicated solely to the substance abuse program, and as of April 15, 2009, that position had been vacant for 6 months because of the state hiring freeze. The responsibilities of the Lead Substance Abuse Clinical Advisor include clinical practice protocols and oversight of clinical practice improvement for substance abuse and co-occurring treatment. According to the Division, other staff, such as another clinical advisor and the Interim Director of Clinical Operations, have taken over some of the responsibilities of the Lead Substance Abuse Clinical Advisor position. Scope and objectives This performance audit focused on steps the Division can take to improve substance abuse treatment outcomes and its oversight of the substance abuse program. The audit’s analysis and case studies focused on adult substance abuse consumers and excluded any consumers who had a serious mental illness in addition to a substance abuse diagnosis and consumers who were younger than 18 years old when they entered treatment. This audit was conducted in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. The Auditor General and staff express appreciation to the Department’s Interim Director and staff for their cooperation and assistance throughout the audit. State of Arizona page 8 Division should focus on strategies that improve outcomes Substance abuse is difficult to treat, but the Department of Health Services, Division of Behavioral Health Services (Division), can employ more effective strategies to improve outcomes. The difficulty of treatment is borne out in auditors’ analysis of the available outcome data, which showed little change in consumer outcomes after treatment. This analysis showed that consumers who entered treatment while already abstaining from alcohol or drugs had the greatest success as measured in terms of continued abstinence, whereas those consumers who entered treatment while using alcohol or drugs showed much lower success rates. Among those consumers who entered treatment while still using alcohol or drugs, those who actually completed their treatment tended to do better than those who dropped out. To improve outcomes, the Division should take steps in three main areas that research has shown to be effective: (1) helping ensure that consumers complete their treatment, (2) ensuring that consumers are placed in a level of treatment that is appropriate based on their assessed needs, and (3) following practices that have been shown to carry the greatest chance of success. Substance abuse difficult to treat Arizona, like many other states, struggles with the problem of treating individuals with substance abuse problems. Substance abuse is increasingly recognized as a chronic, relapsing condition that may require multiple episodes of care over many years. According to the National Institute on Drug Abuse of the National Institutes of Health, long-term drug use results in changes in brain function that can result in the compulsion to use drugs despite adverse consequences.1 Research indicates that, in general, approximately 50 to 60 percent of patients begin using alcohol or drugs again within 6 months of treatment cessation, regardless of the type of discharge, patient characteristics, or the particular substance used.2 Successful treatment of substance abuse relies upon an individual’s ability to change his/her behavior, and ability and motivation to integrate techniques for disease management into his/her 1 National Institute on Drug Abuse, 2009 2 Institute of Medicine of the National Academies, 2006 Office of the Auditor General page 9 FINDING 1 lifestyle. The goal is for the individual to manage symptoms independently using techniques learned in treatment. This is a complex process complicated by a variety of personal, social, and cultural factors such as socio-economic, legal, family, and employment situations. Most consumers showed little change after treatment Auditors’ analysis of commonly used outcome measures for substance abuse consumers found that most participants in the Division’s programs showed little change after entering treatment, although some consumers improved. Auditors analyzed four National Outcome Measures (NOMs) developed by the federal government—substance use, criminal activity, employment, and homelessness (see textbox). Outcomes were tied to two key factors. If consumers were already doing well in a measured area before the start of treatment they generally maintained their status, particularly if they completed treatment. Consumers who were not doing well in a measured area before the start of treatment were more likely to improve if they completed treatment, and stayed the same or got worse if they did not complete treatment. NOMs measure program success—The federal Substance Abuse and Mental Health Services Administration (SAMHSA) has developed a set of National Outcome Measures for use in evaluating treatment effectiveness. It requires states that receive federal Substance Abuse Prevention and Treatment (SAPT) block grant monies to report their progress using these measures. According to experts, for the consumer and other stakeholders, effectiveness of addiction treatment is measured by its ability to reduce addiction-related problems.1 Therefore, in addition to substance use, the NOMs measure other outcomes such as criminal activity, employment, and homelessness. Auditors selected those measures that SAMHSA has fully developed and that could be measured using division demographic data.2 Extent of improvement varied—To determine the Division’s success in treating substance abuse consumers, auditors analyzed data from the Division’s Client Information System (CIS) for substance abuse consumers who did not have a serious mental illness and were enrolled with the Division in fiscal years 2006, 2007, or 2008 (see Appendix A, pages a-i through a-xxi).3 The analyses showed 1 McLellan, McKay, Forman, Cacciola, & Kemp, 2005a 2 In addition to the four measures reported in this audit, SAMHSA has identified six other domains for evaluating program success, some of which are still under development. The other domains are social connectedness, access/capacity, retention, perception of care, cost-effectiveness of treatment, and use of evidence-based practices. 3 Auditors’ analysis excludes any consumers who had a serious mental illness in addition to a substance abuse diagnosis, and also excludes any consumers who were younger than 18 years old when they entered treatment. In addition, for each outcome analyzed, auditors included only the consumers who had valid entries in the outcome data field at both times: (1) entering treatment and (2) annual update or disenrollment. As a result, the total number of consumers analyzed for each outcome varies. “Annual update” refers to an annual reassessment each consumer who remains in treatment should receive. “Disenrollment” means the consumer is removed from enrollment, which occurs when the consumer completes treatment, refuses further treatment, or stops treatment for another reason, such as moving out of Arizona. State of Arizona page 10 National Outcome Measures (NOMs) analyzed by auditors • Fewer consumers reporting recent substance use • Fewer consumers with recent criminal activity • More consumers employed • Fewer consumers who are homeless Source: U.S. Department of Health and Human Services Substance Abuse and Mental Health Services Administration. that more than half of the consumers who entered treatment reported abstaining from alcohol or drugs during the 30 days before starting treatment. Nearly all of these consumers reported that they were still abstaining at the time they left the program. However, of those who were not abstaining when they entered treatment, over 70 percent reported no change in their substance use. Overall, the analyses found little change in the areas of employment, criminal activity, and homelessness. Specifically: Slight overall improvement in substance use—Most consumers who auditors analyzed did not change their use of drugs or alcohol from entering treatment to their annual update or discharge. Approximately half of the consumers were abstinent upon entering treatment and were still abstinent at their update or discharge, while approximately one-third of the consumers who were using drugs or alcohol when they entered treatment were still using drugs or alcohol at their update or discharge (see first textbox on this page). Out of the more than 50,000 consumers auditors analyzed, a total of 5,767 came in using drugs or alcohol and either stopped or reduced use. Approximately 2,400 consumers got worse. Almost all consumers abstaining at the start of treatment remained abstinent—Auditors’ analysis of consumers’ self-reported data showed that more than half (54.1 percent) reported they were not using a substance at the beginning of treatment. More than 93 percent of these consumers reported still being abstinent at their update or disenrollment (see textboxes bottom of this page and top of page 12). According to officials at the Division, RBHAs, and providers, several factors may explain why so many consumers report they are already sober upon entering treatment. For example, some consumers may have quit drinking or using drugs but enter treatment when they realize they cannot maintain sobriety without help. Others may misrepresent their substance abuse, either to make themselves look better or because of wishful thinking if they have been trying to stop for a period of time. Still other consumers may have a problem with binge drinking at irregular intervals. Finally, some consumers enter treatment after spending time in forced abstinence in jail or prison and mistakenly report they have been abstinent, although they are supposed to report on their use prior to entering jail or prison. Office of the Auditor General page 11 Usage outcomes for all consumers1 1 Numbers do not total 100 percent because of rounding. Source: Auditor General staff analysis of division data for adult substance abuse consumers without a serious mental illness enrolled during fiscal years 2006, 2007, or 2008. Consumers who reported using alcohol or drugs when treatment started Reduced use 770 1.5% Stopped using 4,997 10.0% Stayed the same 16,504 33.0% Got worse 693 1.4% Consumers who reported being abstinent when treatment started Stayed abstinent 25,376 50.7% Did not stay abstinent 1,729 3.5% Total 50,069 Extent to which consumers reported being abstinent from alcohol or drug usage when treatment started Source: Auditor General staff analysis of division data for adult substance abuse consumers without a serious mental illness enrolled during fiscal years 2006, 2007, or 2008. Abstinent 27,105 54.1% Using alcohol or drugs 22,964 45.9% Total 50,069 Most consumers using alcohol or drugs at the start of treatment continued to do so—Among the consumers who reported that they were using alcohol or drugs in the 30 days prior to treatment, the majority—nearly 72 percent— reported no change in the frequency of use of their primary substance at their update or disenrollment, while 3 percent actually increased their frequency of use (see textbox, middle of this page). The remaining approximately 25 percent reported either reducing their use of alcohol or drugs or stopping use altogether. For consumers who started treatment using drugs or alcohol, completing treatment appears to be a factor in decreasing substance use. Those who completed treatment had greater success in stopping their drug or alcohol use than those who left without completing treatment. More specifically, among this group, only 17.6 percent who did not complete treatment stopped using drugs or alcohol, but 27 percent of those who completed treatment stopped (see textbox, bottom of this page). Percentage of consumers with recent arrests showed little change—Auditors’ analysis of division data found no significant change in criminal activity as measured by recent arrests. Altogether, 21.2 percent of consumers reported arrests within the past 30 days upon entering treatment, while 18.1 percent reported new arrests at the time of their update or disenrollment date. Most consumers reported not having any arrests within the 30 days prior to starting treatment and only a small number of those consumers had a new arrest at the time of their update or disenrollment (see textbox, page 13). By contrast, of the consumers who reporting being arrested within 30 days of entering treatment, although approximately one–third reported no new arrests as of their annual update or enrollment, the majority—68.1 percent—reported at least one more arrest when they had their annual update or disenrollment. NARBHA, the RBHA for Northern Arizona, showed the greatest reduction of criminal activity among consumers who had been arrested within 30 days of entering treatment. Specifically, 41.2 percent of these NARBHA consumers did not have a new arrest when they had their update or disenrollment. State of Arizona page 12 Usage outcomes for consumers who reported using alcohol or drugs when treatment started1 Source: Auditor General staff analysis of division data for adult substance abuse consumers without a serious mental illness enrolled during fiscal years 2006, 2007, or 2008. 1 Numbers do not total 100 percent because of rounding. Usage outcomes for consumers who reported being abstinent when treatment started Source: Auditor General staff analysis of division data for adult substance abuse consumers without a serious mental illness enrolled during fiscal years 2006, 2007, or 2008. Stayed abstinent 25,376 93.6% Did not stay abstinent 1,729 6.4% Total 27,105 Reduced use 770 3.4% Stopped use 4,997 21.8% Stayed the same 16,504 71.9% Increased use 693 3.0% Total 22,964 Usage outcomes for consumers who reported using alcohol or drugs when treatment started and who completed treatment Source: Auditor General staff analysis of division data for adult substance abuse consumers without a serious mental illness enrolled during fiscal years 2006, 2007, or 2008. Excludes consumers who were still in treatment or who disenrolled for administrative reasons such as transfers between RBHAs, and 278 consumers who enrolled as a result of a crisis and then disenrolled. Reduced use 121 1.7% Stopped use 1,894 27.0% Stayed the same 4,855 69.2% Got worse 144 2.1% Total 7,014 Usage outcomes for consumers who reported using alcohol or drugs before treatment and who did not complete treatment Reduced use 432 3.4% Stopped use 2,270 17.6% Stayed the same 9,771 75.8% Got worse 410 3.2% Total 12,883 Employment did not significantly change—The Division’s programs have not led to a significant change in overall consumer employment. Thirty-eight percent of substance abuse treatment consumers, or 16,360, reported that they were already employed when they entered treatment, and 41 percent reported being employed as of their annual update or disenrollment. Of those consumers who were already employed, more than 92 percent reported being employed as of their annual update or disenrollment (see textbox, page 14). Among the almost 27,000 consumers, or 62 percent, who reported not being employed upon entering treatment, nearly 10 percent reported that they had gained employment as of their annual update or disenrollment, although 90 percent were still unemployed. NARBHA showed the most improvement in this outcome area, as 16.1 percent of NARBHA’s unemployed consumers had gained employment as of their update or disenrollment. Homelessness showed little change—Overall, the number of consumers who were homeless did not change from entering treatment to update or disenrollment, with 7.1 percent homeless upon entering treatment and 6.8 percent homeless at update or disenrollment. Most consumers were not homeless upon entering treatment, and those who were homeless had some Office of the Auditor General page 13 Arrest outcomes for all consumers Arrest outcomes for consumers with no recent arrests when treatment started Arrest outcomes for consumers with recent arrests when treatment started Source: Auditor General staff analysis of division data for adult substance abuse consumers without a serious mental illness enrolled during fiscal years 2006, 2007, or 2008. Consumers with no recent arrests when treatment started Had new arrest before leaving treatment 1,907 3.7% Did not have new arrest 38,565 75.1% Had new arrest before leaving treatment 7,408 14.4% Did not have new arrest 3,468 6.8% Total 51,348 Consumers with recent arrests when treatment started Had new arrest before leaving treatment 1,907 4.7% Did not have new arrest 38,565 95.3% Total 40,472 Had new arrest before leaving treatment 7,408 68.1% Did not have new arrest 3,468 31.9% Total 10,876 success moving into stable housing situations. Specifically, almost 46,000, or 93 percent, of substance abuse consumers reported living in a stable housing situation upon entering treatment, and almost 99 percent of these consumers reported being in stable housing at their annual update or disenrollment (see textbox, page 15). Of the more than 3,500 consumers, or 7 percent, who reported not being in stable housing when they entered treatment, approximately 21.7 percent of these consumers reported gaining stable housing by the time of their annual update or disenrollment. Housing services are generally not available to substance abuse treatment consumers. Such services are not covered by Title XIX, and state monies only provide the services for consumers with serious mental illness. Still, increased housing stability is considered an important measure of substance abuse treatment success. Limited comparative information indicates improvements in Arizona’s program may be low—As part of the analysis, auditors compared Arizona’s NOMs with those reported by other states. The Division uses this information to compare Arizona outcomes to other states. The available data, while limited in quality and consistency, suggests that Arizona’s performance is below that of substance abuse programs in other states. The limitations in the State of Arizona page 14 Employment outcomes for all consumers1 Employment outcomes for consumers who reported being employed before starting treatment Employment outcomes for consumers who reported being unemployed before starting treatment 1 Numbers do not total 100 percent because of rounding. Source: Auditor General staff analysis of division data for adult substance abuse consumers without a serious mental illness enrolled during fiscal years 2006, 2007, or 2008. Consumers who were employed before starting treatment Unemployed when treatment ended 1,255 2.9% Employed when treatment ended 15,105 34.9% Unemployed when treatment ended 24,245 56.0% Employed when treatment ended 2,656 6.1% Total 43,261 Consumers who were unemployed before starting treatment Unemployed when treatment ended 1,255 7.7% Employed when treatment ended 15,105 92.3% Total 16,360 Unemployed when treatment ended 24,245 90.1% Employed when treatment ended 2,656 9.9% Total 26,901 quality of the comparative data stem from differences in how state programs are structured and what the states may be reporting. For example, the data that Arizona reports to SAMHSA includes substance abuse treatment for consumers with serious mental illness, which probably contributes to the lower percentages of consumers meeting the measurement. Whether other states similarly report people with serious mental illness as part of their substance abuse cases is unknown. Thus, while the comparison may be useful in providing a general indication of where Arizona stands relative to other states, it should not be taken as authoritative. Even so, however, Arizona’s apparent ranking relative to other states is another reason to examine the program carefully to determine if it can be improved. Auditors examined the national data collected by SAMHSA at the federal level and found that both nationally and in the SAMHSA-designated Western Region, the percentage of Arizona’s substance abuse treatment consumers meeting the measurement criteria is below average in three of the four areas auditors analyzed: abstinence, criminal activity, and employment.1 This is the case both for consumers who were entering treatment and for consumers at the time of their discharge. Further, the rate of change for Arizona’s consumers from admission to Office of the Auditor General page 15 Housing outcomes for all consumers1 Housing outcomes for consumers reporting they were not homeless before starting treatment Housing outcomes for consumers reporting they were homeless before starting treatment Source: Auditor General staff analysis of division data for adult substance abuse consumers without a serious mental illness enrolled during fiscal years 2006, 2007, or 2008. Consumers who were not homeless before starting treatment Homeless at update or discharge 612 1.2% Not homeless at update or discharge 45,282 91.6% Homeless at update or discharge 2,763 5.6% Not homeless at update or discharge 765 1.5% Total 49,422 Consumers who were homeless before starting treatment Homeless at update or discharge 612 1.3% In stable housing at update or discharge 45,282 98.7% Total 45,894 Homeless at update or discharge 2,763 78.3% In stable housing at update or discharge 765 21.7% Total 3,528 1 Numbers do not total 100 percent because of rounding. 1 SAMHSA’s designated western states reporting NOMs data as part of their fiscal year 2009 SAPT block grant application were Alaska, Arizona, California, Hawaii, Idaho, Montana, Nevada, Oregon, Washington, and Wyoming. discharge is below both the national average and the average for western states on all four measures—that is, the substance abuse programs in other states appeared to have a greater effect in raising outcomes in these other states. (See Appendix A, Figures 6 through 9, 11, 12, 14, 15, 17, and 18, pages a-vi through a-xxi, for comparisons of Arizona’s reported NOMs with other states’ reported NOMs, both regionally and nationally.) Case studies help further explain factors affecting treatment—To provide additional perspective on the program, auditors also analyzed a limited set of specific cases to understand consumers’ progress or lack of progress and the factors that affected success. Auditors examined case files from specific treatment episodes and then interviewed the consumers, treatment providers, and others involved in the case to determine long-term outcomes and what consumers thought contributed to their abstinence or use. Two such cases help illustrate the complicated factors that can affect individual long-term outcomes: Juan, 44, a married man with children who lives in southwestern Arizona, entered substance abuse treatment after being arrested for driving with a blood alcohol level of 0.17 and charged with extreme DUI.1 The court suspended his driver’s license and ordered him to undergo treatment. The court referred Juan to a local mental health agency that has a contract with the RBHA to provide substance abuse counseling in his area. The provider placed Juan in a 9-week substance abuse and relapse prevention program consisting of twice-weekly group therapy focused on education and relapse prevention. At Juan’s intake session with the provider, he stated that prior to his arrest he did not drink very often, but would have seven or eight beers when he was out with his friends. Although the reason he went to substance abuse treatment was to clear up his legal troubles related to the DUI, Juan reported to auditors that the program gave him insight into the dangers of drinking and driving and the impact his drinking had on his family. Juan completed his treatment in December 2005, and when auditors interviewed him in February 2009, he reported that he had not had a drink in 3 years. In this case, the consumer’s NOMs outcomes were unchanged from beginning treatment to completion of treatment because he had a high level of achievement before entering treatment. Juan had never been arrested before. He was employed and living in a stable family situation, and he reported no alcohol use within the 30 days before he entered treatment as well as within the 30 days before he was disenrolled. Dana, 19, entered treatment in northern Arizona after being arrested and serving 60 days in jail on charges of drug sale, possession of drug paraphernalia, and shoplifting. Two of her charges were felonies. Her State of Arizona page 16 1 All consumer names have been changed. substance abuse treatment was required as part of her probation. She had been using alcohol since age 13 and methamphetamine since age 16, and had had previous substance abuse treatment and periods of abstinence. At her intake, Dana stated that she needed supervision and did not trust herself to abstain. Her provider recommended a month of residential treatment and then moved Dana into a two-stage intensive outpatient program. Over the 11 months of her treatment, she had considerable positive change in her NOMs. At her enrollment, she had just been released from jail, was unemployed, was not attending school, and had no stable place to live. At disenrollment she was self-supporting, enrolled in college, living with a family friend, and had no additional arrests. She was involved in Narcotics Anonymous and was not using drugs or alcohol. In Dana’s case, the NOMs data showed substantial progress on all four measures. However, Dana was unable to sustain these changes and relapsed about 1 year after completing treatment. When auditors spoke with her in February 2009, she was in prison with 5 more years to serve on a drug-related conviction. Case studies such as these, together with the results of the auditors’ analyses of the four NOMs, illustrate the difficulty in making decisions about how to improve program success. For example, the Division captures NOMs data at various points during treatment, but the data alone does not adequately indicate whether or not treatment has been successful. Auditors’ review of studies, best practices, and other materials showed, however, that improvements in program outcomes are likely to come from focusing on three main strategies: increasing the number of consumers who complete their treatment program, providing a continuum of care that addresses multiple needs, and using practices that have been shown to generate positive outcomes. The sections that follow discuss each of these strategies in turn. Division should increase focus on treatment retention and completion Research and the Department’s own data indicate that consumers who remain longer in treatment experience better outcomes. However, most consumers state-wide do not complete their treatment. As the analysis discussed on page 12 indicated, those who complete treatment experience better outcomes. Several factors, such as individual motivation and the availability of different treatment options, can influence whether or not consumers stay in treatment. The Division can take several steps to increase consumer retention and completion rates, including establishing performance goals, monitoring completion rates, and using financial and/or nonfinancial incentives and other case management techniques. Office of the Auditor General page 17 Longer treatment yields better results—Research has shown that consumers who remain longer in treatment, no matter what type of treatment it is, show the best post-treatment outcomes.1 For example, studies have shown that longer treatment duration resulted in lower readmission rates and lower rates of drug use and relapse. One study found that consumers who received long-term residential or outpatient treatment for 6 months or more showed a lessened likelihood of engagement in illegal activities than those with shorter lengths of stay.2 Another reported that longer lengths of stay were positively associated with post-treatment client earnings across all types of treatment.3 The National Institute on Drug Abuse states that most consumers need to remain in treatment for about 3 months in order to achieve significant improvement, and additional progress toward recovery can be achieved by providing treatment after the 3-month threshold.4 However, a 2003 study suggested that consumers who stay in treatment for longer than 18 months may start to show reduced amounts of improvement.5 Ensuring treatment completion is an important strategy for improving consumers’ outcomes. Auditors’ analysis of division data found that in addition to being more likely to reduce or stop substance use, consumers who completed treatment were more likely to gain employment and avoid criminal activity than those who did not complete treatment. For example, of the consumers who were unemployed when they entered treatment, 14.4 percent of those who completed treatment gained employment, while only 6.2 percent of noncompleters gained employment. Similarly, of the consumers who had been arrested within the 30 days prior to entering treatment, 47.5 percent of those who completed treatment had no new arrests at their update or discharge, but only 27.5 percent of those who did not complete treatment had no new arrests. Most consumers drop out or receive short-term treatment—Despite the importance of retention, auditors’ analysis of division data found that a majority— 58 percent—of the consumers reviewed did not complete their treatment before they were disenrolled. All six geographic services areas had fewer consumers who completed treatment than those who did not complete treatment (see Figure 3, page 19). The most common reason given for disenrolling consumers who did not complete treatment was that providers lost contact with them (38.3 percent of all reviewed consumers), while 9.4 percent of reviewed consumers refused further treatment. According to the Division, without a court order, consumers’ treatment is voluntary and they can withdraw at any time. Other reasons for disenrollment are death, incarceration, and a loss of Title XIX status. Further, in contrast to the 3 months suggested in literature, the Division reported that in fiscal year 2007, its average length of stay for outpatient treatment was 29 days and for long-term residential care the average length of stay was 78 days. In that fiscal year, more than 60,000 consumers were treated in outpatient programs, while fewer than 700 were treated in residential, long-term programs. State of Arizona page 18 1 McLellan, Chalk, & Bartlett, 2007 2 Etheridge, Craddock, Hubbard, & Rounds-Bryant, 1999 as cited in Koenig, Harwood, Sullivan, & Sen, 2000 3 Koenig, Harwood, Sullivan, & Sen, 2000 4 National Institute on Drug Abuse, 2009 5 Zhang, Friedmann, & Gerstein, 2003 Studies show that treatment retention and completion result in better outcomes. Office of the Auditor General page 19 39.4 48.7 37.6 43.3 44.5 42.5 50.8 60.6 51.3 62.4 56.7 55.5 57.5 49.2 0 10 20 30 40 50 60 70 NARBHA (GSA 1) Cenpatico (GSA 2) CPSA (GSA 3) Cenpatico (GSA 4) CPSA (GSA 5) Magellan/VO (GSA 6) Tribes Percent of Consumers Completed Treatment Did Not Complete Treatment Figure 3: Treatment Completion and Noncompletion Rates by RBHA (GSA) Fiscal Years 2006 through 2008 Source: Auditor General staff analysis of data from the Division’s Client Information System (CIS) for adult substance abuse treatment consumers without a serious mental illness enrolled in fiscal years 2006, 2007, or 2008. State of Arizona page 20 Several factors affect retention—Whether a person stays in treatment depends on factors associated with both the individual and program, according to the National Institute on Drug Abuse (Institute). The Institute’s Principles of Drug Addiction Treatment states that “individual factors related to engagement and retention include motivation to change drug-using behavior, degree of support from family and friends, and whether there is pressure to stay in treatment from the criminal justice system, child protective services, employers, or the family. Within a treatment program, successful counselors are able to establish a positive, therapeutic relationship with the patient. These counselors also ensure that a treatment plan is established and followed so that the individual knows what to expect during treatment.”1 Similarly, the Treatment Research Institute (TRI) states that retention can be improved by making treatment attractive, offering options, increasing monitoring and management, and giving consumers feedback.2 The TRI points out that consumers who are not in some form of treatment or monitoring are at increased risk of relapse. Other researchers have also reported that providing incentives to consumers, such as a reward for a clean urinalysis test, can help increase retention. Division can do more to increase retention—The Division does not monitor treatment retention or length of stay, and although it collects consumer disenrollment reasons it does not analyze or monitor the data. Its contracts with the RBHAs do not require them to collect or report information on retention or length of both inpatient and outpatient stay. The contracts require efforts to re-engage consumers in treatment—specifically, the RBHAs must require their providers to attempt re-engagement, and the Division’s provider manuals also establish specific requirements regarding attempting to re-establish contact with consumers—and the Division’s Administrative Review monitors RBHA’s compliance with this standard. However, auditors’ analysis of division data found that 47.7 percent of consumers could not be re-engaged in treatment (38.3 percent could not be contacted by providers and 9.4 percent refused further treatment), and the 2007 Administrative Review showed all four RBHAs needed improvement on the re-engagement measure. One of the auditors’ case examples illustrates this. Dana—the methamphetamine user who relapsed after treatment completion and was in prison when auditors interviewed her in February 2009— had spotty attendance during her treatment, including a period in May 2006 where she attended less than half her group sessions. Her case manager did not address her attendance issue until Dana asked about being released from treatment. Then, she was informed that her probation required her to maintain acceptable attendance in order to be considered for graduation from the program. Some other states have adopted goals and incentives related to retention and treatment completion. For example: 1 National Institute on Drug Abuse, 2009 2 McLellan, 2006 Office of the Auditor General page 21 Colorado—To meet performance goals, providers must maintain or improve the average percentage of clients from the previous 2 fiscal years who remain in outpatient treatment longer than 90 days, and maintain or decrease the percentage of consumers who left treatment against professional advice. Maine—Performance goals stipulate a minimum of 50 percent of outpatient and 85 percent of intensive outpatient consumers stay at least four sessions. Additionally, 30 percent of outpatient consumers must stay in treatment 90 days or longer, and 50 percent of intensive outpatient treatment consumers must complete treatment. Providers who exceed these expectations are awarded incentive bonuses, while underperforming providers are subject to disincentive penalties. Delaware—Contracts stipulate a $100 bonus incentive to providers for each consumer who completes at least 60 days in treatment and achieves the major goals of his or her treatment plan, including submitting four consecutive drug-free urine screenings randomly administered. To help improve treatment outcomes, the Division should establish performance goals and financial or nonfinancial incentives and disincentives related to retention and treatment completion in its RBHA contracts, and collect and monitor data on retention and completion. In establishing benchmarks and incentives, the Division should consider the potential for unintended consequences such as retaining consumers inappropriately and take steps to ensure that the incentives achieve the desired goals. For example, it could follow Delaware’s example of combining retention with other goals in order to determine whether providers qualify for incentives. In addition, the Division should use its existing oversight practices such as its quarterly case reviews to determine whether providers are taking appropriate steps to retain and engage consumers in treatment. Based on the results of these reviews, the Division should work with the RBHAs to address weaknesses through mechanisms such as technical assistance, training, contract requirements, and/or policy and procedural changes. Division should monitor continuum of care A second strategy for improving consumer outcomes is to provide a continuum of care. Under a continuum of care approach, consumers are placed in treatment based on the types of services they need and the phase of recovery they are in. After initial placement, consumers then step up to more intensive treatment or down to less intense treatment as needed. However, as illustrated by case studies, not all consumers have their treatment needs met or receive the services and therapies that may improve the chances of good treatment outcomes. The Division can do more to ensure continuum of care for consumers by taking several steps, such as collecting and monitoring data relevant to assessment and case management, and working with the RBHAs to make improvements when weaknesses are identified. State of Arizona page 22 Continuum of care matches consumers to appropriate treatment based on their problems and stage of recovery—Because no single treatment is appropriate for all individuals, providers need to be able to match treatment settings, interventions, and services to each individual’s particular problems. Continuum of care is a treatment system in which consumers entering treatment are placed in a level of treatment appropriate to their assessed needs. Individuals are first assessed to determine appropriate treatment services. Successful matching of the consumers to the appropriate treatments for them helps to ensure positive outcomes.1 After initial placement, consumers then step up to more intensive treatment or down to less intense treatment as needed. As the consumers recover, and further assessments are made of consumers’ progress, the treatment is phased down and consumers can be moved into alternative community-based services. Some consumers receive continuum of care while others do not— One of the auditors’ case studies illustrates how treatment that is appropriate to a consumer’s needs can yield long-term results: Mandy, 29, was in treatment for nearly a year. When she contacted a provider, she had already had previous experience with treatment, she was pregnant, her husband was in jail, and her two children were living with relatives. She reported that she had used methamphetamines, alcohol, and marijuana within the past several weeks. After completing a month of treatment with her first provider, she was referred to a specialized residential treatment center for women who are pregnant or have children that offers integrated services designed to teach them to be better parents and give them skills they need to get jobs, take care of their children, and stay sober. As a pregnant woman and therefore part of a priority population, Mandy’s treatment was eligible for SAPT funding. During almost 1 year of treatment with this second provider, Mandy made the decision to end her marriage and place her baby with adoptive parents in an open adoption. At the end of treatment, Mandy had a job, was living with her mother and one of her sons, and was working on her relationship with her other son. When auditors interviewed her in April 2009, 2 years after she completed treatment, Mandy stated that long-term treatment that focused on all her issues gave her a solid foundation for a sober life. She said she no longer had any cravings or desire to use drugs. During her year of integrated treatment, she learned to address the issues that led to her addiction. Her previous short-term treatments had not given her the ability to reflect on the information provided during treatment. Mandy now works as a peer support specialist in the substance abuse field, has custody of one of her sons, and as of April 2009 was about to get custody of her other son. In addition, she was living independently and was in contact with her daughter’s adoptive parents. Continuum of care is an important strategy in ensuring treatment is successful. 1 Mee-Lee, n.d. Office of the Auditor General page 23 By contrast, another example, identified through auditors’ analysis of high-cost consumers, illustrates a failure to provide services along the continuum from acute detoxification and stabilization through rehabilitation to continuing recovery support: Rodney, 43, used medical detoxification services 54 times during the period July 2005 through June 2008—approximately every 3 weeks—receiving services valued at a total of more than $82,000. (See Finding 2, pages 31 through 42, for a discussion of service costs). Rodney was homeless and reported using alcohol daily. Although he received some case management and other crisis services besides detoxification, he received very little in the way of other services, such as treatment that would reinforce his sobriety after detoxification and rehabilitation to help him learn to remain sober in the long term. According to a case manager at the detoxification center, Rodney typically stays in the center for only 2 to 3 days and chooses not to complete treatment or receive follow-up care. A consumer like Rodney who undergoes only the first stage of care is likely to return to substance abuse. Further, when auditors discussed this case with the Division’s medical director, he expressed concern that Rodney’s health is jeopardized by this constant cycling between heavy drinking and medical detoxification. Consumers are sometimes placed into services that may not meet their individual needs—contrary to the continuum-of-care concept—or that duplicate services they have already received. Auditors’ case studies included three consumers who received the same treatment program from different providers in different parts of the State: Juan, discussed previously, and two other consumers, Tom and Victoria. All three received alcohol education classes designed for people who have received their first DUI or alcohol-related arrest. These classes are intended to get participants to recognize problematic behaviors and make better choices in the future. For Juan, who had his first arrest for DUI at the age of 44, the class was an effective wake-up call regarding the dangers of impaired driving and how drinking affected his family relationships. He told the auditors that his 9 weeks of classes were very helpful to him and improved his life. Tom and Victoria, however, had long histories of alcoholism and completed the classes after serving prison sentences for drunken driving. Both Tom and Victoria had received similar classes while in prison. Tom told auditors that going through the same treatment after his release was not helpful and did not contribute to his continued sobriety. Instead, Tom and Victoria felt they needed treatment that would have helped them adjust to life after prison and address other issues. Both said their continued sobriety resulted from their own desire to avoid further legal troubles. In addition to auditors’ case studies that found that continuum of care is not consistently and appropriately provided, a 2008 study by one RBHA also found cases where consumers did not receive needed substance abuse services. Auditors’ case studies and a RBHA study revealed that not all consumers’ needs were met. 1 De Leon, Melnick, & Cleland, 2008 2 McLellan, Weinstein, Shen, Kendig, & Levine, 2005b State of Arizona page 24 Specifically, the Community Partnership of Southern Arizona (CPSA) conducted a focus review in August 2008 of a psychiatric health facility in its network and found that there was little evidence to support that the facility provided active treatment to the ten consumers sampled in the review, including three consumers who required substance abuse treatment (For more information on this review, see Finding 2, pages 31 through 42). The review found that none of the three consumers received substance abuse treatment services during their 7- to 10-day inpatient stays at the facility. Continuum of care requires that consumers’ needs are assessed and that they receive treatment that matches their needs. Division can do more to ensure continuum of care—The Division’s contracts with the RBHAs already specify that providers are required to develop treatments and services designed to support long-term recovery and focus on life factors such as employment, ongoing feedback to the recipient, and re-engagement into treatment based on the recipient’s changing needs. However, the Division’s oversight processes have not focused on ensuring that the RBHAs have implemented these requirements for substance abuse consumers (See Finding 2, pages 31 through 42, for recommendations on improving oversight of substance abuse treatment). As the Division moves to ensure that the RBHAs implement a more effective continuum of care, it should adopt two approaches shown by research to improve consumer outcomes. Specifically: Establishing assessment standards—The Division should establish standards for assessing the severity of consumers’ substance abuse problems and referring consumers to appropriate treatment, and monitor implementation of these standards as part of its regular oversight of RBHA performance. A 2008 study found that assessing the severity of a consumer’s substance abuse problem and then assigning more severe consumers to more intensive treatment improves both outcomes and cost-effectiveness of treatment.1 The study reported that appropriate placement does not require a complex matching process where individuals are matched with a specific style of treatment, such as cognitive behavioral therapy versus a 12-step program (see textbox, page 25). Defining case management expectations—The Division should define appropriate expectations for case management of substance abuse consumers, and consider assigning consumers with severe or complex cases to a case manager for their complete course of treatment, taking into consideration the costs of case management. For example, effective clinical case management can reduce inefficient over-use of detoxification. A 2005 study in Philadelphia concluded that multiple episodes of detoxification without follow-up care are inappropriate, ineffective, and inefficient.2 Pairing detoxification with clinical case management resulted in a 55 percent Office of the Auditor General page 25 reduction in detoxification admissions and a 70 percent increase in use of rehabilitation. Clinical case managers were expected to engage consumers early and assess their substance use, health, and social problems; use motivational interviewing designed to help consumers recognize and accept the need for continued care; and play an active, participative role in helping the consumers access and continue with needed medical and social services (See Finding 2, page 40, for information on how the Division can use case management to help minimize costs). When applying these approaches in its management of the program, the Division should also collect and monitor data relevant to assessment and case management and work with the RBHAs to make improvements when its oversight identifies weaknesses. Using appropriate evidence-based practices can improve success rate A third strategy the Division can employ is to increase its attention to providers’ use of evidence-based practices. These practices are validated by observation or experience as improving treatment success. Although the Division requires the RBHAs to use evidence-based practices, RBHAs are not necessarily doing so, and the Division is not ensuring compliance. To improve adherence to evidence-based practices, the Division should enhance its monitoring of the practices and take appropriate steps to address any weaknesses it finds. Evidence-based practices important to success—Evidence-based practices are therapies confirmed by scientific research to have a positive effect in the treatment of substance abuse. They include motivational interviewing, cognitive behavioral therapy, community reinforcement therapy, and 12-step facilitation therapy (see textbox). There is a growing body of literature on the cost-effectiveness of well-defined treatments that have been validated by observation or experience for substance use disorder, according to a 2007 study.1 SAMHSA has developed the National Registry of Evidence-based Programs and Practices, a listing of programs verified by independent researchers for their ability to achieve 1 Olmstead, Sindelar, Easton, & Carroll, 2007 Examples of Evidence-based Practices • Motivational Interviewing—A counseling style designed to help consumers recognize and accept the need for continued care. (www.motivationalinterview.org) • Cognitive behavioral therapy—A treatment that seeks to change the way an individual thinks to feel/act better even if the situation does not change. (www.nacbt.org) • Community reinforcement therapy—Therapy that focuses on events that influence behavior, such as family and social events, and uses the events to support successful abstinence. (www.ncsacw.samhsa.gov) • 12 step facilitation therapy—A brief, structured therapy that supports initial and ongoing participation in a 12- step program such as Alcoholics Anonymous. (www.nrepp.samhsa.gov) • ASAM criteria—Placement guidelines that focus on assessing consumers across six dimensions such as emotional and behavioral conditions and readiness to change and then placing the consumer in a level of care to match the severity of his/her condition. (www.asam.org and www.coce.samhsa.gov) • Stages of change—A model of behavioral change that progresses from examining the risks of change to commitment to action, and then plan, implement, and maintain the change. (www.coce.samhsa.gov) Source: Auditor General staff summary of evidence-based practices as defined on the above Web sites on June 24, 2009. State of Arizona page 26 results when properly implemented. The Division’s review of residential treatment centers supports the use of such evidence-based practices and the results that can be achieved. The Division contracted with a private firm to review nine substance abuse residential treatment centers that provide such services mainly to women to gather baseline data. In that review, all nine centers reported that they used at least two forms of evidence-based best practices. The three most commonly reported practice models across all nine centers included the American Society of Addiction Medicine (ASAM) criteria, motivational interviewing, and stages of change. The report indicated that 97.5 percent of the cases reviewed showed evidence of symptomatic improvement. Evidence-based practices are also used for outpatient treatment. For example, there are three Methamphetamine Treatment Centers of Excellence located throughout the State, which specialize in the treatment of methamphetamine use through techniques such as incentives, community reinforcement, motivational interviewing, and monitoring behavior through urine testing. Requirement for evidence-based practices not necessarily followed or enforced—The Division’s contracts with the RBHAs require the use of evidence-based practices, but the Division can do more to ensure providers are using the most up-to-date research and implementing the practices in accordance with the evidence. The Division has received or conducted three reviews of substance abuse programs. For example, in one study, a 2008 federal grant review that focused on programs for children and adolescents, the Division did a good job of establishing evidence-based practices in some areas, but did not identify such practices across the continuum of care. It could also do more to ensure sustainability of the emphasis on evidence-based practices. In another study, the Division reviewed 11 intensive outpatient programs for youth across the State and found that, although the Division had an internal definition of what constitutes an intensive outpatient program, the definition was not documented or adequately communicated to the providers. Therefore, a number of the providers’ programs did not meet the Division’s definition. One example of implementation of an evidence-based practice for substance abuse treatment is at NARBHA, which received a SAMHSA grant to implement a program called Matrix for adult treatment services. NARBHA uses a tool provided by the Matrix Institute to evaluate providers and ensure the program is implemented properly. The Matrix model incorporates therapist support, group participation, support group participation, relapse prevention and education, family involvement, and an explicit structure giving consumers a clear understanding of treatment. According to staff at one of NARBHA’s providers, the program has helped them get consumers into the structure of recovery. One element of the model, contingency management—where consumers are rewarded for successful participation, for example with prizes such as movie passes—is seen as helpful for keeping consumers engaged in treatment. NARBHA uses an evidence-based practice for substance abuse treatment and monitors it to ensure proper implementation. Office of the Auditor General page 27 The Division may be able to use one evidence-based practice—self-help group participation—more effectively. Research has shown a strong association between self-help group participation and long-term recovery following treatment. Although none of the consumers included in auditors’ case studies continued to participate regularly in self-help groups after treatment, one of them, Dana—the young woman serving a drug-related prison sentence—attributed her return to drug use to when she stopped attending Narcotics Anonymous meetings and continued to associate with friends who used drugs. She indicated that she found it useful to have group leaders who were recovering addicts because their familiarity with addiction makes them good at breaking down addicts’ behavioral defense mechanisms. Division can do more to encourage use of evidence-based practices—Auditors identified two key ways in which the Division can encourage greater use of evidence-based practices: More extensive monitoring—The Division examines a sample of substance abuse consumers’ cases each quarter as part of its overall oversight of the RBHAs in order to ensure consumers obtain services in a timely manner and receive appropriate services. In addition, in order to fulfill federal grant requirements, the Division has conducted a review of substance abuse programs for adolescents and contracted for a review of women’s substance abuse residential treatment facilities. Specifically, during 2007 and 2008, division staff reviewed 11 intensive outpatient programs for adolescents to determine how the programs could be improved. Also, in 2008, the Health Services Advisory Group, an external quality review organization contracted by the Division, conducted a study of women’s residential treatment facilities to gather baseline data on women’s substance abuse treatment programs. However, the Division does not routinely review services to determine whether the RBHAs are applying evidence-based practices or implementing them as they were designed to be used. Similar to NARBHA’s approach with its providers that use the Matrix model, the Division should monitor evidence-based practices either directly or by reviewing the RBHAs’ assessments of providers’ practices. Improved guidance—The Division should also continue and expand its work with RBHAs to ensure that providers have the guidance needed to implement specific evidence-based practices. When the Division implements new treatment models, it provides guidance and other assistance to ensure the RBHAs and providers implement the models correctly. For example, it used this approach to implement a new system of care for children in the early 2000s, as discussed in the Auditor General’s report on implementation of HB2003 programs for children (see Report No. 02-12). The Division also cosponsors an annual summer institute conference for RBHAs and providers. The Division should use the results of its monitoring of evidence-based The Division may be able to use self-help groups more effectively. practices implementation to identify any needed improvements, and then work with RBHAs to provide technical assistance, training, and guidelines as appropriate. Finally, the Division should make better use of self-help programs by developing a method to track and monitor participation and encouraging RBHAs to offer a wider variety of programs for consumers. Because self-help group participation can continue the benefits of treatment long after treatment is completed, the Division should also consider working with the RBHAs to develop procedures to follow up with consumers after treatment is completed to encourage continued participation in such groups. Recommendations: 1.1. To help improve retention, the Division should: a. Collect and monitor data on retention and completion, including length of stay and disenrollment reasons; b. Establish performance goals and financial and/or nonfinancial incentives and disincentives related to retention and treatment completion in its RBHA contracts, taking care to avoid encouraging providers to inappropriately retain consumers in treatment in order to meet the goals; c. Use its existing oversight practices such as its quarterly case reviews to determine whether RBHAs are taking appropriate steps to retain and engage consumers in treatment; and d. Based on the results of these reviews, the Division should work with the RBHAs to address weaknesses through mechanisms such as technical assistance, training, contract requirements, and/or policy and procedural changes. 1.2. To make better use of the continuum of care to improve treatment outcomes, the Division should: a. Establish standards for assessing the severity of consumers’ substance abuse problems and referring them to appropriate treatment; b. Using data, monitor implementation of these standards as part of its regular oversight of RBHA performance; c. Define appropriate expectations for case management of substance abuse consumers, taking into consideration costs of case management and the advantages of monitoring consumers with severe or complex cases; State of Arizona page 28 d. Collect and monitor data relevant to assessment and case management; and e. Work with the RBHAs to make improvements when its oversight identifies weaknesses. 1.3. To better ensure the use of appropriate evidence-based practices, the Division should: a. Monitor compliance with its contractual requirements to use evidence-based practices; b. Work with RBHAs to provide technical assistance, training, and guidelines as appropriate to ensure that providers have the guidance needed to implement specific evidence-based practices such as motivational interviewing, cognitive behavioral therapy, and community reinforcement therapy; c. Develop a method to track and monitor self-help group participation; d. Encourage RBHAs to offer a wider variety of self-help programs for consumers; and e. Consider working with the RBHAs to develop procedures to engage consumers in community and peer support outlets that would reinforce progress made in treatment once consumers are disenrolled. Office of the Auditor General page 29 State of Arizona page 30 Division should improve oversight of substance abuse programs The Department of Health Services, Division of Behavioral Health Services (Division), should take steps to improve its oversight of the substance abuse programs administered by the Regional Behavioral Health Authorities (RBHAs). Current oversight efforts are limited in two key ways. First, although the Division gathers information on treatment outcomes, it does not use this information to monitor programs, focusing instead on process-related information such as whether services are timely. To increase the benefit that treatment participants receive, the Division should continue its efforts to streamline uniform data collection that focuses on consumer outcomes, establish relevant performance goals in contracts with the RBHAs, and encourage the RBHAs to consider mechanisms for awarding financial and/or nonfinancial incentives to providers who meet these goals and penalizing those who do not. Second, oversight efforts currently lack any significant focus on analyzing treatment costs and types. This is particularly true for analyzing high-cost or low-cost cases that may indicate ineffective or inadequate services. To help better ensure that the Division’s scarce resources are used effectively, the Division should implement a systematic approach that includes but is not limited to reviewing high-and low-dollar cost services by individual users and treatment categories and among RBHAs, and considering how to best use assessment and case management to contain costs. Division should monitor outcomes The Division should improve oversight of substance abuse treatment by increasing its focus on consumer outcomes—that is, what the services are accomplishing in terms of keeping consumers from reverting to abuse of alcohol or drugs. The Division focuses primarily on treatment process measures for all behavioral health consumers, including substance abuse consumers, and it is therefore largely unable to determine if its substance abuse treatment programs are achieving positive results or if its resources are being used effectively. Focusing more on outcomes involves taking such steps as streamlining uniform outcome data collection, implementing minimum performance goals, and establishing performance incentives and penalties to help ensure that goals are met. Office of the Auditor General page 31 FINDING 2 Division monitors process, not treatment outcomes—The Division’s oversight of the behavioral health treatment system is limited to a number of process measures and is not focused on treatment results. For example, as required by its Arizona Health Care Cost Containment System (AHCCCS) contract, on a quarterly basis, the Division reviews the RBHAs’ ability to provide timely and appropriate services to consumers and coordination of care with the consumer’s primary care physician. Such measures are required for all Medicaid-covered services and provide information that is relevant to assessing whether consumers receive services, but they do not provide meaningful insight as to whether consumers are reducing their dependence on alcohol or drugs. The Division gathers consumer outcome data but does not appear to use it for monitoring purposes. The Division gathers outcome data such as information on abstinence and participation in employment or education when developing its Annual Report on Substance Abuse Treatment Programs, and when applying for the annual federal Substance Abuse Prevention and Treatment block grant. In addition, the Division reported that it uses outcome data to compare Arizona’s to other states’ outcomes. Division policies state that this information allows for the measurement of behavioral health outcomes, is required of all consumers, and is used to support quality and utilization management activities. However, auditors found no indications that the Division uses this information to evaluate RBHA performance in providing effective substance abuse treatment. The Division also does not use this outcome information to compare performance between RBHAs or between individual providers. As a result, the Division is unable to effectively evaluate system performance and determine which providers’ treatment services are resulting in positive consumer outcomes. Likewise, the Division is unable to identify underperforming providers and require that they make necessary improvements. Similarly, at the RBHA level, some RBHA representatives said that although they are required to gather the outcome data and submit it to the Division, they do not evaluate the outcome information as thoroughly as they could because they need more guidance from the Division on how this should be done. Therefore, the Division should provide more guidance to the RBHAs to help ensure they can thoroughly evaluate the consumer outcome information. Division should continue efforts to streamline uniform outcome data collection—The Division is revising its assessment process, and in doing so it needs to ensure the new process enables it to uniformly collect the data needed to assess consumer outcomes. The Division requires RBHAs to gather consumer outcome data during intake, update, and disenrollment assessments. Historically, according to division officials, providers have used a uniform core assessment tool mandated by the Division, but some have expressed concern about the length of the 32-page tool and the more than 1 hour that it took to fill it out. The tool, which is used for all the Division’s consumers, includes a full behavioral health assessment, not just a substance abuse assessment. During the audit, the Division began developing a new method for collecting data that would streamline the data collection process. The Division gathers consumer outcome data but does not use it for monitoring purposes. State of Arizona page 32 Because A.R.S. §36-2006(A) and (B)(4) require the Department to establish a standardized screening assessment for alcohol and other drug education and treatment programs, and because consistent outcome information is important for monitoring program success, the Division should continue in its efforts to streamline uniform collection of required outcome data. Another state has developed uniform requirements that are short, and the Division may be able to adopt its solution. Maine has developed a one-page form specifically for gathering consumer outcome information during assessments that includes standardized yes-or-no questions or provides a list of the answer options for each question. Adopting a similar form or modifying its existing method would address providers’ concerns about the form’s length while retaining the Division’s ability to effectively and uniformly collect, monitor, and easily validate outcome data as well as allowing it to comply with statute. Division should establish minimum outcome-based performance goals, incentives, and penalties—In 2006, a national policy panel convened to examine how state governments could be most effective in preventing and treating substance use disorders and problems, and reported that states should hold agencies and contracted providers accountable for meeting identified outcome measures and should reward those that meet or exceed outcome targets and penalize those that consistently fail.1 As discussed in Finding 1 (see pages 9 through 29), three states—Maine, Delaware, and Colorado— reported having established performance goals for providers on a variety of measures such as treatment retention, continuation of care, and abstinence. Delaware and Maine have both set a minimum number of consumers that must meet the goals while Colorado requires that providers maintain or improve their performance without specifying an exact number of consumers. However, a Colorado Division of Behavioral Health official reported that it will modify the performance goals in fiscal year 2010 to require that the providers obtain certain percentages rather than perform better than the average of their previous scores for 2 years. In addition, Colorado has established goals for consumer transition from detoxification to some other form of treatment (see textbox). Further, a fourth state, Nevada, is currently conducting a 3-year pilot program that includes performance goals. By contrast, although the Division has some contractual requirements regarding provider performance, it has not established benchmarks for clinical performance, or outcome goals that are specific to substance abuse treatment. 1 Rosenbloom, Leis, Shah, & Ambrogi, 2006 Office of the Auditor General page 33 Examples of Colorado’s performance goals Maintain or improve the percentage of detox clients who are referred to further treatment Maintain or improve the percentage of detox clients who enter any non-detox treatment setting within 30 days of their detox discharge Maintain or improve the percentage of treatment clients who stayed in outpatient treatment longer than 90 days Maintain or improve the average of treatment clients who had a reduction in primary drug use at discharge Source: Colorado’s fiscal years 2006 through 2008 state-wide treatment performance measures for all managed service organizations provided by an official from Colorado’s Department of Human Services, Division of Behavioral Health. In addition to performance goals, two states—Delaware and Maine—have incorporated financial incentives and disincentives into their performance goals that could serve as models for Arizona. For example, Delaware has established incentives for criteria such as patient participation and treatment retention/completion. For patient participation, the provider receives a 1 percent bonus if 50 percent of participating patients attend two sessions per week in the first 30 days of treatment. An incentive payment is also tied to completion, active participation for 60 days, achievement of the major goals of the treatment plan, and 4 consecutive weeks with clean urine samples. Maine also includes incentives and disincentives in its performance-based contract, which stipulates that their providers can collect a percentage of their quarterly payment each quarter they exceed their performance goals. Likewise, its providers can lose a percentage for each underperforming quarter. Division needs to establish benchmarks first, then incentives—The Division is not yet in a position to do what other states have done because it lacks clear benchmarks for measuring performance and incentives to encourage good performance. However, the Division has some contractual requirements regarding provider performance and financial incentives. For example, Magellan, the RBHA that oversees behavioral health services in Maricopa County, is required by contract to implement financial incentives with its service providers to increase performance on the National Outcome Measures by September 2009. According to a Magellan official, the RBHA is working with its providers to implement performance goals and to put a process in place for awarding incentives or levying penalties. Other RBHAs’ contracts state that the RBHA will share incentives originating from the Division with applicable providers. However, before performance incentives, or penalties, can be levied, the Division must first establish performance benchmarks or standards for treatment retention and completion and consumer continuation of treatment from detoxification to some other form of treatment and modify its contracts with the RBHAs to require such measures. According to a division official, the Division is exploring possible benchmarks for performance incentives or penalties, but difficulties in collecting and analyzing the necessary data have hampered efforts in establishing benchmarks. In April 2009, the Division re-organized its data management staff in order to address the data collection and analysis issues. The Division should encourage the RBHAs to consider contractually implementing a financial or nonfinancial method of incentivizing substance abuse treatment providers who exceed the goals established in their contracts and penalizing those providers who continually fail to meet these standards related to consumer outcomes, treatment retention, and treatment completion. Magellan is required by contract to implement financial incentives with its service providers to increase performance by September 2009. State of Arizona page 34 Division should monitor monies spent on treatment The second area in which improved monitoring is needed is more reviews of service costs. Reviews of over- and underutilization of healthcare services are important to help contain costs and manage quality of substance abuse treatment. Although the Division contractually requires the RBHAs to perform such reviews of services, the reviews should be expanded to identify unusually high and low costs of services to individual consumers and high treatment or services costs at each RBHA, and monitor variations in treatment across all RBHAs. The Division should continue and expand on the steps it took during the audit to begin reviewing service costs. In addition, the Division should also consider how to best use assessment and case management to contain costs. Lastly, the Division should continue its efforts to fill vacant positions and should perform follow-up work to ensure that the restructuring it initiated in April 2009 results in improved oversight. Utilization reviews can help contain healthcare costs and maintain quality of care—In a managed care system like Arizona’s, utilization reviews are critical to help control service costs while ensuring that quality of care remains high (see textbox). Because the Division uses a capitated system to pay RBHAs in advance for providing all required services, current-year costs are not reflected in current-year payments as they would be in a fee-for-service system. Rather, each year’s capitation rate is established based on analyzing actual costs of services delivered in previous years, so future payments are affected by current-year service costs. Research has shown that the cost-containment goals of managed care systems can be obtained without sacrificing client outcomes. However, attention must be paid to both cost-containment strategies and quality-of-care issues (such as evidence-based practices) if clients are to benefit from substance abuse treatment. Specifically, 2005 and 2006 studies that compared one California county that used managed care to two other counties that used a more traditional fee-for-service approach to substance abuse treatment found that the managed care approach was more cost-effective—it achieved the same results at a lower cost.1 The managed care county established various best practices to manage both cost and promote better outcomes including clear assessment guidelines, case management guidelines for complex cases, continuum of care requirements, performance-based contracting, and utilization management guidelines. For example, the utilization management guidelines included a standard 45-day length of stay for stabilization in residential treatment, followed by outpatient rehabilitation, and a standard 90-day outpatient treatment with intensity guidelines for the number of sessions. 1 Beattie, Hu, Li, & Bond, 2005; Beattie, McDaniel, & Bond, 2006 Office of the Auditor General page 35 Utilization review—A process to evaluate and approve or deny healthcare services, procedures, or settings based on medical necessity, appropriateness, effectiveness, and efficiency. Source: AHCCCS Medical Policy Manual Chapter 1000—Medical Management/Utilization Management. The Division’s contractual requirements for utilization reviews cover nine dimensions identified in the AHCCCS utilization management policy (see textbox), and RBHAs include all nine dimensions in their utilization management plans. As outlined in these plans, RBHAs review a wide range of issues, including data on readmissions, average length of stay, and pharmacy utilization, such as monitoring for multiple medicines taken to treat the same illness. By monitoring these measures of care, the RBHAs have information that would allow them to help control costs and identify potential issues with quality of care as illustrated in the following example: In August 2008, one RBHA, Community Partnership of Southern Arizona (CPSA), conducted a focus review of a psychiatric health facility. In that 10- case file review, CPSA found that there was little evidence to support that the facility provided active treatment to the consumers, including three consumers who required substance abuse treatment. As a result, CPSA made recommendations regarding treatment planning documentation and group therapy improvement. A CPSA official reported that CPSA will follow up with the facility in August 2009 to ensure that it implemented these recommendations. State of Arizona page 36 AHCCCS medical/utilization management policy requirements • Utilization Data Analysis and Data Management—Collect, monitor, analyze, evaluate, and report utilization data to detect both underutilization and overutilization of services. • Concurrent Review—Review the medical necessity for a planned institutional admission or ongoing institutional care. • Prior Authorization—Make medical decisions regarding the initial authorization of services or requests for continuation of services in specified time frames. • Retrospective Review—Provide policies, procedures on how to conduct reviews to determine medical necessity post delivery of services. • Adoption and Dissemination of Practice Guidelines—Adopt and disseminate practice guidelines that are based on valid and reliable clinical evidence or a consensus of healthcare professionals and include a thorough review of medical journals’ peer-reviewed articles published in the United States when national practice guidelines are not available. • New Medical Technologies and New Uses of Existing Technologies—Develop and implement written policies and procedures for evaluating new technologies and new uses of existing technologies on both an individual basis and systemic basis. • Case Management/Care Coordination—Ensure that a member’s needs are appropriately met through a coordination of cost-effective care involving early identification of health risk factors or special care needs whether in an acute, home, chronic, or alternative setting. • Disease/Chronic Care Management—Implement a program that focuses on members with high risk and/or chronic conditions to increase member self-management and improve providers’ practice patterns and thereby improve members’ healthcare outcome. • Drug Utilization Review—Review the prescribing, dispensing, and use of medications to ensure that the medications are clinically appropriate, safe, and cost-effective drug therapy that improves a member’s health status and quality of care. Source: AHCCCS Medical Policy Manual Chapter 1000—Medical Management/Utilization Management. Expanding existing reviews can help manage costs—Although current utilization reviews contain many elements that could potentially help manage costs, auditors identified several ways to improve their ability to do so. These include the following: Conducting reviews of high- and low-cost substance abuse treatment cases—Auditors’ analysis of division data identified 14 consumers who received substance abuse treatment valued at a total of over $100,000 each during fiscal years 2006 through 2008. The Division was not aware of these cases until auditors brought them to officials’ attention. In one of these cases, presented in Finding 1 as Rodney (see Finding 1, pages 9 through 29), the consumer repeatedly used medical detoxification services valued at a total of over $82,000 in fiscal years 2006 through 2008. Rodney’s stay at a Tucson detoxification center in September 2008 was typical of his pattern. He was brought to a hospital emergency room one evening by emergency medical services. At triage, he had a blood alcohol level of .396. The center provided Ativan, a medication for withdrawal symptoms; fed him; and applied a nicotine patch because the hospital is a nonsmoking campus. A registered nurse monitored him periodically while he slept. Rodney transferred to the detoxification center the following day. When the center conducted an assessment to determine his readiness for treatment to address his addiction, he said, “I’m not sure what I want to do after detox. I’ve never been to treatment, so I can’t say if I would like to go there.” Rodney discharged himself around noon on the third day, against medical advice. Auditors’ analysis of division data found 54 separate division data records for detoxification services for Rodney over the 3-year period analyzed, and a case manager at the center reported that Rodney used detoxification three or four times a month. Cases such as Rodney’s involve high costs, but appear to produce limited outcomes. Altogether, for fiscal years 2006 through 2008, approximately $100 million, or 72 percent of the total encounter value (reported value of services) for substance abuse treatment, was used to provide services to 20 percent of the consumer population, and 45 percent of these consumers did not complete treatment. Reviewing high-cost cases such as Rodney’s could enable the Division to work with the RBHAs and providers to find ways to treat consumers more effectively and break the cycle of substance abuse and detoxification. During the audit, in March 2009, the Division began reviews that focused on service costs for consumers with serious mental illness. Specifically, the Division reviewed Magellan’s fiscal year 2008 healthcare costs for consumers with serious mental illness and identified the top ten service users as Office of the Auditor General page 37 measured by service costs and 49 other clients who were enrolled in fiscal year 2008 but did not receive services that year, most of whom also had not received services in fiscal year 2009. Low-dollar service users could indicate potential quality of care issues or an inadequate disenrollment process. The Division presented the information to Magellan officials, who replied that the top ten service users were very complex cases involving safety concerns. Magellan added that it determined that all services provided to nine of the ten consumers were clinically appropriate and, as a result, would conduct an in-depth review of the clinical appropriateness of the services offered to the one other consumer. Regarding consumers who underutilized services, Magellan reported that they found an opportunity to improve timeliness of closures and follow up after crisis episodes. As a result, Magellan reported that it would implement a claims inactivity report to identify consumers who did not have encounters (service records) for periods of more than 210 days. However, this review was limited to one RBHA and consumers with serious mental illness. To more effectively control costs, the Division needs to continue and expand these efforts. The Division should perform similar reviews for consumers receiving treatment for substance abuse in order to identify, research, and appropriately address all high- and low-dollar service users. In addition, the Division should follow the example of the California managed-care county discussed on page 35 and establish benchmarks for length of stay, and collect and monitor data on length of stay in order to identify consumers who may be over- or under-utilizing certain types of services, which could indicate they need a different service to be treated cost-effectively. Conducting cost-focused reviews of specific treatments or services—In addition to identifying and addressing high- and low-dollar service users, the Division should also focus reviews on managing high-cost treatment or service types. Conducting such reviews would enable the Division to determine not only if the services are necessary, but help identify other effective, potentially less-costly treatments. In addition to reviewing the highest-cost consumers, auditors identified other consumers in various service categories such as transportation, inpatient, and outpatient treatment. Auditors specifically looked for cases with costs significantly higher than the median for that service category. Using division data, auditors identified consumers whose fiscal years 2006 through 2008 treatment values ranged from $50,000 to $100,000, and found that the costs of some of the specific services these consumers received were 15 times more than the median costs of such services. For example: • Ron, 52, has an opiate dependence that requires supervised methadone treatment. According to his treatment center’s clinical director, although Ron lives in Lake Havasu City, his treatment provider transports him twice a week over 180 miles round-trip to Bullhead City, which the clinical State of Arizona page 38 director added is the only authorized methadone treatment center in rural Mohave County. His treatment center’s clinical director also explained that because of federal methadone regulations and licensing requirements, it is very difficult to incentivize other healthcare facilities that may be closer to the consumer to become methadone treatment facilities.1 Therefore, his only option is to transport the client to the Bullhead City Community Medical Services II center at a value over fiscal years 2006 through 2008 of almost $48,000, which represents more than 87 percent of the total value of Ron’s treatment for that time period. This amount greatly exceeds the RBHA’s median transportation value of approximately $88. Examining costs such as this can help the Division identify high-cost practices and examine alternatives for reducing them. Methadone treatment is one area in which the Division has already begun to study alternatives. From fiscal years 2006 to 2008, approximately 2,600 consumers, or about 5 percent of all substance abuse consumers served, received methadone treatment valued at more than $9.5 million. According to a division official, the Division is transitioning to a new treatment, buprenorphine, which the consumer can take home and administer daily. In addition, in an effort to reduce transportation costs for consumers receiving methadone treatment, in February 2009, the Northern Arizona Regional Behavioral Health Authority (NARBHA) approved the purchase of a van to provide such services instead of paying a third-party vendor to do so. To help contain costs, such as transportation costs associated with methadone treatment, the Division and RBHAs should continue their efforts to transition to alternative treatments. In addition, all RBHAs and the Division should identify consumers with higher-than-usual costs for specific services, and alternative methods or treatments that would provide the same quality of care at a reduced cost. Comparing variation in treatment types among RBHAs—Finally, to better monitor ser |
