Performance audit, department of health services division of behavioral health services substance abuse treatment programs: report to the Arizona Legislature |
Previous | 1 of 4 | Next |
|
|
Small
Medium
Large
Extra Large
Full-size
Full-size archival image
|
This page
All
|
Debra K. Davenport Auditor General Performance Audit Department of Health Services Division of Behavioral Health Services— Substance Abuse Treatment Programs Performance Audit Division July • 2009 REPORT NO. 09-07 A REPORT TO THE ARIZONA LEGISLATURE The is appointed by the Joint Legislative Audit Committee, a bipartisan committee composed of five senators and five representatives. Her mission is to provide independent and impartial information and specific recommendations to improve the operations of state and local government entities. To this end, she provides financial audits and accounting services to the State and political subdivisions, investigates possible misuse of public monies, and conducts performance audits of school districts, state agencies, and the programs they administer. The Joint Legislative Audit Committee Audit Staff Copies of the Auditor General’s reports are free. You may request them by contacting us at: Office of the Auditor General 2910 N. 44th Street, Suite 410 • Phoenix, AZ 85018 • (602) 553-0333 Additionally, many of our reports can be found in electronic format at: www.azauditor.gov Melanie M. Chesney, Director Shan Hays, Manager and Contact Person Anne Hunter, Team Leader Kerry Howell Karl Kulick Senator Thayer Verschoor, Chair Representative Judy Burges, Vice-Chair Senator Pamela Gorman Representative Tom Boone Senator John Huppenthal Representative Cloves Campbell, Jr. Senator Richard Miranda Representative Rich Crandall Senator Rebecca Rios Representative Kyrsten Sinema Senator Bob Burns (ex-officio) Representative Kirk Adams (ex-officio) 2910 NORTH 44th STREET • SUITE 410 • PHOENIX, ARIZONA 85018 • (602) 553-0333 • FAX (602) 553-0051 WILLIAM THOMSON DEPUTY AUDITOR GENERAL DEBRA K. DAVENPORT, CPA AUDITOR GENERAL STATE OF ARIZONA OFFICE OF THE AUDITOR GENERAL July 30, 2009 Members of the Arizona Legislature The Honorable Janice K. Brewer, Governor Will Humble, Interim Director Department of Health Services Transmitted herewith is a report of the Auditor General, a Performance Audit of the Department of Health Services, Division of Behavioral Health Services—Substance Abuse Treatment Programs. This report is in response to an October 5, 2006, resolution of the Joint Legislative Audit Committee. The performance audit was conducted as part of the sunset review process prescribed in Arizona Revised Statutes §41-2951 et seq. I am also transmitting with this report a copy of the Report Highlights for this audit to provide a quick summary for your convenience. As outlined in its response, the Department of Health Services agrees with all of the findings and plans to implement all of the recommendations. My staff and I will be pleased to discuss or clarify items in the report. This report will be released to the public on July 31, 2009. Sincerely, Debbie Davenport Auditor General Attachment The Office of the Auditor General has conducted a performance audit of the substance abuse treatment programs provided by the Department of Health Services (Department), Division of Behavioral Health Services (Division), pursuant to an October 5, 2006, resolution of the Joint Legislative Audit Committee. This is the second audit in a series of three reports on the Department and was conducted as part of the sunset review process prescribed in Arizona Revised Statutes (A.R.S.) §41-2951 et seq. This audit focuses on substance abuse treatment outcomes and system oversight. The first audit focused on the Division of Licensing Services, and the final report will be an analysis of the 12 statutory sunset factors. According to its Annual Report on Substance Abuse Treatment Programs, the Division spent more than $121 million for substance abuse services in fiscal year 2008. Program participants, whom the Division refers to as “consumers,” numbered more than 63,000 adults and children. Most were enrolled in the Arizona Health Care Cost Containment System, or AHCCCS, the State’s Medicaid program. They received alcohol- and drug-related services ranging from counseling and skills training to crisis intervention and detoxification in hospitals or other inpatient facilities. The Division provides these services through contracts with four regional behavioral health authorities, or RBHAs, and three tribal regional behavioral health authorities, or TRBHAs, which contract with a network of more than 100 substance abuse treatment service providers throughout the State. This performance audit focused on the program’s outcomes—that is, the extent to which services reduced dependency on alcohol and drugs—and on the Division’s oversight of behavioral health authorities and providers. Division should focus on strategies that improve outcomes (see pages 9 through 29) Although substance abuse is difficult to treat, the Division can take steps to improve outcomes for individuals who participate in substance abuse treatment. Auditors analyzed 3 years of data related to four measures commonly used to evaluate program effectiveness—extent of continuing alcohol or drug use, employment, criminal activity, and homelessness. The analysis showed that outcomes related to continued alcohol and drug use were associated with the following two factors: Office of the Auditor General SUMMARY page i Deciding to abstain from using alcohol or drugs before treatment started— More than half of all consumers reported that they were abstinent when their treatment started. Within this group, more than 93 percent reported that they were still abstinent when they left the program. By contrast, most of those who reported using alcohol or drugs when they began treatment were still using these substances at about the same level when they left. About one person in every four who began treatment while still using alcohol or drugs reported diminishing his/her use of alcohol or drugs or stopping it altogether by the time he/she left treatment. Completing treatment—Overall, 58 percent of consumers did not complete their treatment. Providers lost contact with many of them, while others refused treatment or left for other reasons, but rates of continued use of alcohol or drugs varied substantially between those who completed the treatment and those who did not. For example, among consumers who reported using alcohol or drugs when they began treatment, 27 percent of those who completed their treatment reported abstinence when they left. By contrast, among consumers who reported using alcohol or drugs when they began treatment, only 17.6 percent of those consumers who left before completing their treatment reported that their use had diminished. The analysis showed little change across the three remaining performance measures—lack of recent arrests, employment, and stable housing. For example, 21 percent of consumers reported recent arrests upon entering treatment, and 18 percent reported new arrests at the time of their update or disenrollment. Similarly, 38 percent said they were employed when they entered treatment, while 41 percent reported being employed at their annual update or disenrollment. Finally, slightly more than 7 percent were homeless upon entering treatment, and slightly less than 7 percent were homeless at their annual update or disenrollment. Substance abuse is difficult to treat, and auditors’ more detailed case studies of a limited number of consumers showed that the reasons for success or failure are complicated and varied. Nonetheless, research and best practices indicate the best opportunities for increasing success rest in three main strategies: Focusing on treatment retention—Research corroborates what auditors’ analysis showed: consumers who remain longer in treatment experience better outcomes. The Division can take several steps to increase consumer retention, including establishing performance goals, monitoring completion rates, and using incentives and other case management techniques. Other states that auditors reviewed have taken such steps, and the Division may be able to adopt some of these approaches. Ensuring that consumers have access to a full range of services that can potentially be used to address their particular needs—This strategy, called continuum of care, involves incorporating appropriate types of treatment over State of Arizona page ii time and placing the consumer in more or less intensive treatment as needed. Auditors’ case studies showed that while some consumers showed good outcomes and received appropriate services, others did not necessarily receive the services and therapies that might improve the chances of good treatment outcomes. The Division can take several steps to ensure continuum of care, including collecting and monitoring data relevant to assessment, better defining case management, and working with RBHAs to improve the continuum of care when weaknesses are identified. Following practices that have been shown to carry the greatest chance of success—These evidence-based practices have been validated by observation or experience as improving treatment success. Examples include motivational interviews, which is a counseling style designed to help consumers recognize and accept the need for continued care. Although the Division requires the RBHAs to use evidence-based practices, RBHAs are not necessarily doing so, and the Division is not ensuring compliance. According to a 2008 federal grant review that focused on programs for children and adolescents, the Division had done a good job of establishing evidence-based practices in some areas but had not identified such practices across the continuum of care and could do more to ensure sustainability of the emphasis on evidence-based practices. The Division reached similar conclusions in another study of intensive outpatient programs for youth. Steps the Division can take to place greater emphasis on evidence-based practices include encouraging RBHAs to offer a wider variety of programs, monitoring compliance with its contractual requirements to use evidence-based practices, and expanding its work with the RBHAs to ensure that providers have the guidance needed to implement specific evidence-based practices. Division should improve oversight of substance abuse programs (see pages 31 through 42) The Division should take steps to improve its oversight of the substance abuse programs administered by RBHAs. These steps take two main forms: Increasing the use of information about treatment outcomes—Although the Division collects outcome information to complete certain reports, auditors found that oversight efforts focused almost entirely on process-related information, such as the timeliness of services or coordination with a consumer’s primary care physician as required by the Division’s contract with AHCCCS. As a result, the Division is largely unable to determine if its substance abuse treatment programs are achieving positive results or if its resources are being used effectively. Additionally, because the Division does not compare Office of the Auditor General page iii substance abuse outcome measures across RBHAs or providers, it cannot assess which providers’ treatment services are resulting in improved client outcomes or identify underperforming providers. Needed actions include continuing its efforts to streamline uniform outcome data collection, establishing relevant performance goals in contracts with the RBHAs, and encouraging the RBHAs to consider ways to reward providers who meet standards and penalize those who do not. Expanding utilization reviews to focus more on service costs, consumer assessments, and case management—Although its oversight efforts contain many elements that could potentially help manage costs, the Division could implement several actions that could improve its ability to do so. Greater emphasis on cost appears warranted. Auditors’ review of division data from fiscal years 2006 to 2008, for example, identified 14 substance abuse consumers with service costs over $100,000. One incurred $82,000 in medical detoxification costs, during which time he continued to drink and require detoxification three or four times a month. The Division was not aware of these cases until auditors brought them to officials’ attention. Actions needed include (1) regularly reviewing high- and low-cost substance abuse treatment cases, (2) collecting data to identify consumers who may be overutilizing or underutilizing certain types of services, which could indicate a lack of alternative forms of treatment or a need for other changes to improve treatment, (3) comparing variations in the use of types of treatment at each RBHA to see if the use of such services positively affects consumer treatment outcomes and adjusting treatment accordingly, and (4) determining how to best use assessment and case management to contain costs while maintaining quality of care. Further, to improve oversight, the Division should continue its efforts to fill vacant positions in its data systems and analysis and quality management functions, and should perform follow-up work to ensure that the restructuring it initiated in April 2009 has provided management with the information to do so. State of Arizona page iv Office of the Auditor General TABLE OF CONTENTS continued page v Introduction & Background 1 Finding 1: Division should focus on strategies that improve outcomes 9 Substance abuse difficult to treat 9 Most consumers showed little change after treatment 10 Division should increase focus on treatment retention and completion 17 Division should monitor continuum of care 21 Using appropriate evidence-based practices can improve success rate 25 Recommendations 28 Finding 2: Division should improve oversight of substance abuse programs 31 Division should monitor outcomes 31 Division should monitor monies spent on treatment 35 Recommendations 41 Appendix A: Outcome analysis methodology and results a-i Appendix B: Methodology b-i Appendix C: Bibliography c-i Agency Response TABLE OF CONTENTS continued page vi State of Arizona Table: 1 Funding Sources for Substance Abuse Services Expenditures Fiscal Year 2008 (Unaudited) 4 Figures: 1 Regional and Tribal Behavioral Health Authorities’ Geographical Service Areas (GSA) and Fiscal Year 2008 Substance Abuse Enrollment 3 2 State-wide Substance Abuse Treatment Enrollment Fiscal Years 2001 through 2008 6 3 Treatment Completion and Noncompletion Rates by RBHA (GSA) Fiscal Years 2006 through 2008 19 4 Consumer-Reported Abstinence from Substance Use By RBHA (GSA) from Admission to Annual Review or Discharge Fiscal Years 2006 through 2008 a-iv 5 Change in Substance Use Among Consumers Who Reported Substance Use Before Treatment by RBHA (GSA) From Admission to Annual Review or Discharge Fiscal Years 2006 through 2008 a-v 6 Alcohol Abstinence Status of Consumers by State (Western Region) Fiscal Year 2009 SAPT Block Grant Application (Unaudited) a-vi 7 Drug Abstinence Status of Consumers by State (Western Region) Fiscal Year 2009 SAPT Block Grant Application (Unaudited) a-vii Office of the Auditor General TABLE OF CONTENTS continued page vii Figures (continued): 8 Alcohol Abstinence Status of Consumers Arizona, Western Region, and National Averages Fiscal Year 2009 SAPT Block Grant Application (Unaudited) a-viii 9 Drug Abstinence Status of Consumers Arizona, Western Region, and National Averages Fiscal Years 2009 SAPT Block Grant Application (Unaudited) a-ix 10 Employment Status of Consumers by RBHA (GSA) From Admission to Annual Review or Discharge Fiscal Years 2006 through 2008 a-xi 11 Employment Status of Consumers by State (Western Region) Fiscal Year 2009 SAPT Block Grant Application (Unaudited) a-xii 12 Employment Status of Consumers Arizona, Western Region, and National Averages Fiscal Year 2009 SAPT Block Grant Application (Unaudited) a-xiii 13 Arrest-Free Status of Consumers by RBHA (GSA) From Admission to Annual Review or Discharge Fiscal Years 2006 through 2008 a-xv 14 Arrest-Free Status of Consumers by State (Western Region) Fiscal Year 2009 SAPT Block Grant Application (Unaudited) a-xvi 15 Arrest-Free Status of Consumers Arizona, Western Region, and National Averages Fiscal Year 2009 SAPT Block Grant Application (Unaudited) a-xvii State of Arizona TABLE OF CONTENTS concluded page viii Figures (concluded): 16 Homeless Status of Consumers by RBHA (GSA) From Admission to Annual Review or Discharge Fiscal Years 2006 through 2008 a-xix 17 Homeless Status of Consumers by State (Western Region) Fiscal Year 2009 SAPT Block Grant Application (Unaudited) a-xx 18 Homeless Status of Consumers Arizona, Western Region, and National Averages Fiscal Year 2009 SAPT Block Grant Application (Unaudited) a-xxi The Office of the Auditor General has conducted a performance audit of the substance abuse treatment programs provided by the Department of Health Services (Department), Division of Behavioral Health Services (Division), pursuant to an October 5, 2006, resolution of the Joint Legislative Audit Committee. This is the second audit in a series of three reports on the Department and was conducted as part of the sunset review process prescribed in Arizona Revised Statutes (A.R.S.) §41-2951 et seq. This audit focuses on substance abuse treatment outcomes and system oversight. The first audit focused on the Division of Licensing Services, and the final report will be an analysis of the 12 statutory sunset factors. Importance of treating substance abuse Drug and alcohol abuse are associated with some of society’s most serious and expensive problems. For example, according to literature, nation-wide: More than half of all state prison inmates were under the influence of alcohol or drugs when they were arrested.1 Nearly one in six state inmates committed crimes to support a drug addiction.2 About 20 percent of acute care Medicaid expenditures pay for alcohol- or drug-related medical costs.3 Drunk driving is a major expense for the police, courts, and emergency medical systems.4 1 U.S. Department of Justice, Bureau of Justice Statistics, 2005a as cited in Rosenbloom et al., 2006 2 U.S. Department of Justice, Bureau of Justice Statistics, 2005b as cited in Rosenbloom et al., 2006 3 U.S. Department of Health and Human Services, Substance Abuse and Mental Health Services Administration, 2005 as cited in Rosenbloom et al., 2006 4 Miller, Cox, Zaloshnja, & Taylor, 2002 Office of the Auditor General INTRODUCTION & BACKGROUND page 1 In 2003, an estimated 22.5 million people ages 12 and older nation-wide, or about 9.4 percent of the population, had a substance abuse disorder, and an estimated $21 billion was devoted to substance abuse treatment, according to a U.S. Department of Health and Human Services’ Substance Abuse and Mental Health Services Administration (SAMHSA) report dated 2007.1 According to the same report, people with substance use disorders rely on public sources of financing far more than do people with other diseases. The report also states that public sources of funding, including state and local government, Medicaid, and other federal spending, such as block grants, provided 77 percent of total substance abuse spending in 2003 but only 45 percent of all healthcare spending. The nature of addiction, such as its quality as a chronic and recurring problem, and other factors, such as age and socio-economic status, affect treatment success. Arizona, like many other states, struggles with the problem of substance abuse (see textbox). Structure and funding of Arizona’s system for providing substance abuse services The Division provides substance abuse services mainly to consumers enrolled in the Arizona Health Care Cost Containment System, or AHCCCS, which is the State’s Medicaid program. However, others can receive services to the extent funding is available, particularly if they are members of priority populations such as pregnant women. AHCCCS enrollees include adults who meet the requirements of Title XIX (Medicaid) of the Social Security Act (Act), as well as children and their families who meet the requirements of Title XXI (the federal SCHIP program, which is called KidsCare in Arizona) of the Act. Publicly funded substance abuse treatment and prevention services in Arizona are provided primarily through the behavioral health system administered by the Division.2 The Division provides these services through contracts with four regional behavioral health authorities, or RBHAs, and three tribal regional behavioral health 1 Mark et al., 2007 2 Incarcerated individuals may receive publicly funded substance abuse services through the Department of Corrections or the county jails. State of Arizona page 2 Prevalence and impacts of substance abuse in Arizona Altogether, an estimated 466,000 Arizonans ages 12 and older were dependent on or abused alcohol, and an estimated 137,000 were dependent on or abused illicit drugs, according to the 2005 National Survey on Drug Use and Health. Drug- and alcohol-related deaths, including those attributable to motor vehicle and boating fatalities, accounted for nearly 1,700 deaths in Arizona in 2005. There were significant increases in the number of alcohol and drug-induced deaths in Arizona between 2000 and 2005. The rates of drug-induced deaths more than doubled between 2000 and 2005, and the number of alcohol-induced deaths increased by more than one-third for the same time period. A 2008 study of adults arrested in Maricopa County in 2007 found that over one-third of arrestees interviewed for the study tested positive for methamphetamine use, with over 40 percent of female arrestees testing positive. Source: The Substance Abuse Epidemiology Work Group’s 2007 Arizona Statewide Substance Abuse Epidemiology Profile, and Arizona State University, Center for Violence Prevention and Community Safety’s Arizona Arrestee Reporting Information Network Annual Adult Report 2007. authorities, or TRBHAs, which contract with a network of more than 100 substance abuse treatment service providers within six geographical service areas, or GSAs, throughout the State (see Figure 1).1 During fiscal year 2008, over 63,000 adults and children participated in substance abuse treatment programs offered through the RBHAs and TRBHAs and their providers throughout the State. Magellan, the RBHA that serves Maricopa County, served the largest proportion—38 percent—of individuals receiving substance abuse treatment in fiscal year 2008, followed by Community Partnership of Southern Arizona (CPSA) at 32 percent, Northern Arizona Behavioral Health Authority (NARBHA) at 15 percent, and Cenpatico Behavioral Health of Arizona, the three tribal regional behavioral health authorities, and the Navajo Nation serving the remaining approximately 15 percent. The Navajo Nation serves approximately 2 percent of individuals receiving substance abuse treatment. According to the Division’s Annual Report on Substance Abuse Treatment Programs, in fiscal year 2008 approximately $121.2 million, or approximately 11 percent of the Division’s estimated $1.1 billion budget, was expended for substance abuse services.2 The majority of this funding came from Title XIX/XXI monies, with additional funds provided through the federal Substance Abuse Prevention and Treatment (SAPT) block grant, state appropriations, and other funding sources (see Table 1, page 4). Budget reductions in fiscal year 2009 will probably affect substance abuse treatment. In addition, as of July 16, 2009, the Department’s 2010 budget had not been determined. According to the Division, a total of $2.5 million of appropriated funds that are used for substance abuse services was cut from its fiscal year 2009 budget. Office of the Auditor General page 3 1 Magellan replaced ValueOptions as the RBHA for Maricopa County on September 1, 2007. Source: Auditor General staff analysis of the Arizona Department of Health Services, Division of Behavioral Health Services’ Annual Report on Substance Abuse Treatment Programs for fiscal year 2008 and division-provided enrollment data. Figure 1: Regional and Tribal Behavioral Health Authorities’ Geographical Service Areas (GSA) and Fiscal Year 2008 Substance Abuse Enrollment Northern Arizona Regional Behavioral Health Authority Cenpatico Behavioral Health of Arizona Magellan1 Cenpatico Behavioral Health of Arizona Community Partnership of Southern Arizona Community Partnership of Southern Arizona Tribal Regional Behavioral Health Authorities 1 In addition to its three TRBHA contracts, the Division also contracts with two tribal nations—the Navajo Nation and the Colorado River Indian Tribes—to deliver substance abuse treatment services to reservation residents. 2 The total amount of funding allocated to the RBHAs in fiscal year 2008 was $989.4 million, including monies for serving children, adults with serious mental illness, and people with co-occurring mental health and substance abuse disorders, as well as providing general mental health and substance abuse prevention and treatment services. In addition, the Division reported fund sweeps and appropriations reductions totaling almost $1.1 million in substance abuse fund monies. These monies come mainly from fees and fines collected from criminal offenses and are used to provide alcohol and drug screening, education, and treatment for individuals who are court-ordered to attend, but do not have sufficient ability to pay. Because of these reductions, the Division expects that the number of consumers served in fiscal year 2009 will be less than the number served in fiscal year 2008. In addition, to offset some of the budget reductions, the Division reported that it is updating its policy to require co-payments for consumers who are not eligible for Title XIX and whose income is a certain percentage above the federal poverty level. On April 3, 2009, a draft version of the policy regarding co-payments was released for public comments, and as of July 16, 2009, the policy had not been finalized.1 Funding is allocated to the RBHAs through a capitated payment arrangement whereby the RBHA is provided a set amount of funding each month for each consumer in its geographic service area who is enrolled in AHCCCS. In order to allow the Division to monitor services and determine funding needs, the RBHAs submit service data called encounters (see textbox) to the Division. The Division’s contracts with the RBHAs require them to submit encounters whose total value equals at least 85 percent of the total service revenue payments they receive on an annual basis. Substance abuse services provided The Division provides many different types of substance abuse services. In fiscal years 2006 through 2008, the RBHAs reported that they provided services valued at $139.3 million to substance abuse treatment consumers, excluding substance abuse services to other consumers such as adults with serious mental illness, children, and adolescents. The largest service category for those consumers, 1 Specific changes to the policy include: 1) consumers with serious mental illness are no longer excluded from the co-payment requirement, 2) the types of services excluded from the co-payment requirement were modified, and 3) providers are allowed to refuse service as well as terminate services to a consumer for non-payment of a co-payment. State of Arizona page 4 Encounter—Record of a service delivered to a consumer by a provider. Includes specifics about the type and date of service, provider that delivered the service, and dollar value of the service. Funding Source Amount Percentage Title XIX/XXI funding $ 75,941,357 62.66% Substance Abuse Prevention and Treatment Block Grant 23,545,206 19.43 State appropriations 16,518,610 13.63 Intergovernmental agreements (IGA) and interagency service agreement (ISA)1 5,141,081 4.24 Liquor fees2 45,325 0.04 Total $121,191,579 100.00% Table 1: Funding Sources for Substance Abuse Services Expenditures Fiscal Year 2008 (Unaudited) 1 In fiscal year 2008 the Division had IGAs with Maricopa County and the City of Phoenix, and an ISA with the Arizona Department of Corrections, Correctional Officer/Offender Liaison program (the COOL program). Maricopa County and the City of Phoenix provide monies to help operate the Local Alcohol Rehabilitation Center, a detoxification center in Phoenix. The COOL program was eliminated by the Department of Corrections in 2009. However, in 2008 the program provided transitional services for persons with substance abuse issues who were leaving the penal system. The services included assistance in finding housing, employment, and behavioral health services. 2 Liquor fees are fees collected from the issuance of special event liquor licenses. Source: Auditor General staff analysis of the Arizona Department of Health Services, Division of Behavioral Health Services’ Annual Report on Substance Abuse Treatment Programs for fiscal year 2008. treatment services, accounted for approximately 31.5 percent of the total.1 The types of services were as follows: Treatment services ($43.9 million)—Individual and group counseling, therapy, assessment, evaluation, screening, and other professional services. Residential services ($26.9 million)—Twenty-four-hour residential services, including structured treatment, which includes room and board, delivered in residential facilities or supported independent living settings. Support services ($25.3 million)—Case management, peer support services, and transportation. Medical and pharmacy services ($19.0 million)—Medication to assist with alcohol or drug withdrawal or with co-occurring disorders. Inpatient ($9.5 million)—Inpatient detoxification and treatment services delivered in hospitals and other inpatient facilities, including residential treatment centers that provide a structured treatment with 24-hour supervision, an intensive treatment program, and on-site medical services. Crisis intervention services ($5.7 million)—Crisis intervention/stabilization services provided in the community, hospitals, and residential treatment centers. Rehabilitation services ($4.9 million)—Education, coaching, training, and other services, including securing and maintaining employment. Services include living skills training, cognitive rehabilitation, health promotion, and ongoing support to help maintain employment. Day programs ($4.1 million)—Skills training and development, behavioral health prevention/promotion, medication training and support, ongoing support to maintain employment, and self-help/peer services to improve consumers’ ability to function in the community. The cost of services varies by type of treatment. For example, auditors’ analysis of encounter data from fiscal years 2006 through 2008 revealed that the average cost per person, per year, for residential treatment was $1,500 and the average cost of crisis intervention per person, per year, was $240. Because some consumers may receive several different types of treatment or receive treatment for longer periods or at several different times during the year, the total cost per person for all services received varies widely, with a few consumers receiving services valued at more than Office of the Auditor General page 5 1 The figures reported in these bullets reflect auditors’ analysis of services reported for consumers whose primary reason for enrollment was substance abuse. These figures differ from the total substance abuse funding of $121 million shown in Table 1 (see page 4) because these figures cover a 3-year period and do not include substance abuse services provided to other consumers. Further, because the Division uses a capitated system to pay RBHAs in advance for providing all required services, current-year costs are not reflected in current-year payments as they would be in a fee-for- service system. Therefore, the total substance abuse funding is not the same amount as the value of the services provided as reported by the RBHAs for a given time period. $100,000 per year (See Finding 2, pages 31 through 42, for additional information on high-utilization consumers). The median cost of treatment per person for fiscal years 2006 through 2008 was $1,090.1 Consumers served Generally, to obtain substance abuse services through the RBHAs or TRBHAs a person must be enrolled in AHCCCS. According to the Division’s Annual Report on Substance Abuse Treatment Programs, in fiscal year 2008, 74 percent of adults and children receiving substance abuse services, or approximately 47,000 individuals, were AHCCCS eligible. Substance abuse services for non-Title XIX/XXI-eligible individuals and families, also delivered by RBHA- and TRBHA-contracted providers, are provided only if monies are available, and these services are provided first to people in priority populations. The priority populations are spelled out by grant requirements and other funding sources. For example, the federal SAPT block grant designates priority populations as pregnant women, women with dependent children, and intravenous drug users. In fiscal year 2008, 1,139 pregnant women and 6,238 women with children received substance abuse services. In addition, in fiscal year 2008, 4,033 people were tested for human immunodeficiency virus (HIV), with 24 identified as being HIV positive and receiving counseling. From fiscal year 2001 to fiscal year 2008, the number of people receiving some type of state-provided substance abuse treatment increased by nearly 300 percent, from 16,115 to 63,571 (see Figure 2). The Division attributes this growth primarily to expansion in AHCCCS eligibility requirements. For example, in fiscal year 2000, Arizona voters approved Proposition 204, expanding eligibility for AHCCCS starting in 2001. The proposition expanded income eligibility requirements up to 100 percent of the federal poverty level. In fiscal year 2008 nearly all consumers receiving substance abuse treatment services through the RBHAs and TRBHAs and their providers were adults, most were non-Hispanic, and alcohol was the most commonly used substance (see textbox, page 7). That year, according to its Annual State of Arizona page 6 16,115 32,772 41,172 36,375 59,661 60,105 65,696 63,571 0 10,000 20,000 30,000 40,000 50,000 60,000 70,000 2001 2002 2003 2004 2005 2006 2007 2008 Enrollment Fiscal Year Figure 2: State-wide Substance Abuse Treatment Enrollment Fiscal Years 2001 through 2008 Source: Auditor General staff analysis of the Arizona Department of Health Services, Division of Behavioral Health Services’ Annual Report on Substance Abuse Treatment Programs for fiscal years 2007 and 2008. 1 Median cost per person was calculated using demographic and encounter data from the Division’s Client Information System for fiscal years 2006 through 2008 for adult substance abuse consumers without a serious mental illness. Report on Substance Abuse Treatment Programs, the Division provided substance abuse treatment services to 49,751 adults without a serious mental illness (78 percent of substance abuse treatment consumers), 10,071 adults who had serious mental illness with a co-occurring substance abuse disorder (16 percent), and 3,929 children or adolescents (6 percent). Consumer characteristics are important in planning treatment and can affect treatment success. For example, women with children need specialized help or services, such as childcare. Consumers entered treatment through a variety of referral sources, with nearly half seeking treatment on their own or on the advice of a friend or family member. Referral by court order or involvement with a criminal justice agency varies regionally. In rural areas, court or criminal justice referrals are as high as 40 percent of the treatment-seeking population. For example, in fiscal year 2007, in the primarily rural counties of Yuma and La Paz, 41 percent of people enrolled in substance abuse treatment were referred by a court or a probation/parole officer. Office of the Auditor General page 7 Consumer Characteristics Fiscal Year 2008 Age: •94% adults; 6% adolescents •50% of adults were aged 25 to 44 Gender: •56% male; 44% female Diagnosis: •78% were adults with a substance abuse disorder •16% had serious mental illness with a co-occurring substance abuse disorder Race: •86% Caucasian •7% African American •5% Native American •2% other or multi-race Ethnicity: •26% Hispanic/Latino •74% non-Hispanic/Latino Primary •37% alcohol substance: •26% stimulants including methamphetamine and cocaine/crack •24% marijuana •11% narcotics including heroin •2% other Referral •45% self, friend, or family source:1 •23% court order or criminal justice agency •11% other or unknown •10% other behavioral health providers •6% other state agencies •3% community agencies •3% Health Plan or primary care physician 1 Numbers do not total 100 percent because of rounding. Source: Auditor General staff analysis of the Arizona Department of Health Services, Division of Behavioral Health Services’ Annual Report on Substance Abuse Treatment Programs for fiscal year 2008. Substance abuse program oversight The Division is primarily an oversight agency that monitors all aspects of behavioral health services, including substance abuse treatment services, through contract compliance and other mechanisms. For example, the Division’s contract compliance unit conducts administrative reviews, as required by the Division’s contract with AHCCCS, that examine the RBHAs’ compliance with state and federal requirements, program operations, fiscal operations, and financial status. Although these reviews encompass overall behavioral health, including substance abuse, they do not focus on substance abuse. Additionally, various units within the Division are responsible for overseeing different aspects of the substance abuse program, as well as the general mental health program and co-occurring disorders. For example, the grants, clinical practice improvement, interagency coordination units, and office of the medical director all have responsibilities that involve some oversight of general mental health treatment, substance abuse treatment, and treatment for co-occurring disorders. The Division has one full-time equivalent position, Lead Substance Abuse Clinical Advisor, that is dedicated solely to the substance abuse program, and as of April 15, 2009, that position had been vacant for 6 months because of the state hiring freeze. The responsibilities of the Lead Substance Abuse Clinical Advisor include clinical practice protocols and oversight of clinical practice improvement for substance abuse and co-occurring treatment. According to the Division, other staff, such as another clinical advisor and the Interim Director of Clinical Operations, have taken over some of the responsibilities of the Lead Substance Abuse Clinical Advisor position. Scope and objectives This performance audit focused on steps the Division can take to improve substance abuse treatment outcomes and its oversight of the substance abuse program. The audit’s analysis and case studies focused on adult substance abuse consumers and excluded any consumers who had a serious mental illness in addition to a substance abuse diagnosis and consumers who were younger than 18 years old when they entered treatment. This audit was conducted in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. The Auditor General and staff express appreciation to the Department’s Interim Director and staff for their cooperation and assistance throughout the audit. State of Arizona page 8 Division should focus on strategies that improve outcomes Substance abuse is difficult to treat, but the Department of Health Services, Division of Behavioral Health Services (Division), can employ more effective strategies to improve outcomes. The difficulty of treatment is borne out in auditors’ analysis of the available outcome data, which showed little change in consumer outcomes after treatment. This analysis showed that consumers who entered treatment while already abstaining from alcohol or drugs had the greatest success as measured in terms of continued abstinence, whereas those consumers who entered treatment while using alcohol or drugs showed much lower success rates. Among those consumers who entered treatment while still using alcohol or drugs, those who actually completed their treatment tended to do better than those who dropped out. To improve outcomes, the Division should take steps in three main areas that research has shown to be effective: (1) helping ensure that consumers complete their treatment, (2) ensuring that consumers are placed in a level of treatment that is appropriate based on their assessed needs, and (3) following practices that have been shown to carry the greatest chance of success. Substance abuse difficult to treat Arizona, like many other states, struggles with the problem of treating individuals with substance abuse problems. Substance abuse is increasingly recognized as a chronic, relapsing condition that may require multiple episodes of care over many years. According to the National Institute on Drug Abuse of the National Institutes of Health, long-term drug use results in changes in brain function that can result in the compulsion to use drugs despite adverse consequences.1 Research indicates that, in general, approximately 50 to 60 percent of patients begin using alcohol or drugs again within 6 months of treatment cessation, regardless of the type of discharge, patient characteristics, or the particular substance used.2 Successful treatment of substance abuse relies upon an individual’s ability to change his/her behavior, and ability and motivation to integrate techniques for disease management into his/her 1 National Institute on Drug Abuse, 2009 2 Institute of Medicine of the National Academies, 2006 Office of the Auditor General page 9 FINDING 1 lifestyle. The goal is for the individual to manage symptoms independently using techniques learned in treatment. This is a complex process complicated by a variety of personal, social, and cultural factors such as socio-economic, legal, family, and employment situations. Most consumers showed little change after treatment Auditors’ analysis of commonly used outcome measures for substance abuse consumers found that most participants in the Division’s programs showed little change after entering treatment, although some consumers improved. Auditors analyzed four National Outcome Measures (NOMs) developed by the federal government—substance use, criminal activity, employment, and homelessness (see textbox). Outcomes were tied to two key factors. If consumers were already doing well in a measured area before the start of treatment they generally maintained their status, particularly if they completed treatment. Consumers who were not doing well in a measured area before the start of treatment were more likely to improve if they completed treatment, and stayed the same or got worse if they did not complete treatment. NOMs measure program success—The federal Substance Abuse and Mental Health Services Administration (SAMHSA) has developed a set of National Outcome Measures for use in evaluating treatment effectiveness. It requires states that receive federal Substance Abuse Prevention and Treatment (SAPT) block grant monies to report their progress using these measures. According to experts, for the consumer and other stakeholders, effectiveness of addiction treatment is measured by its ability to reduce addiction-related problems.1 Therefore, in addition to substance use, the NOMs measure other outcomes such as criminal activity, employment, and homelessness. Auditors selected those measures that SAMHSA has fully developed and that could be measured using division demographic data.2 Extent of improvement varied—To determine the Division’s success in treating substance abuse consumers, auditors analyzed data from the Division’s Client Information System (CIS) for substance abuse consumers who did not have a serious mental illness and were enrolled with the Division in fiscal years 2006, 2007, or 2008 (see Appendix A, pages a-i through a-xxi).3 The analyses showed 1 McLellan, McKay, Forman, Cacciola, & Kemp, 2005a 2 In addition to the four measures reported in this audit, SAMHSA has identified six other domains for evaluating program success, some of which are still under development. The other domains are social connectedness, access/capacity, retention, perception of care, cost-effectiveness of treatment, and use of evidence-based practices. 3 Auditors’ analysis excludes any consumers who had a serious mental illness in addition to a substance abuse diagnosis, and also excludes any consumers who were younger than 18 years old when they entered treatment. In addition, for each outcome analyzed, auditors included only the consumers who had valid entries in the outcome data field at both times: (1) entering treatment and (2) annual update or disenrollment. As a result, the total number of consumers analyzed for each outcome varies. “Annual update” refers to an annual reassessment each consumer who remains in treatment should receive. “Disenrollment” means the consumer is removed from enrollment, which occurs when the consumer completes treatment, refuses further treatment, or stops treatment for another reason, such as moving out of Arizona. State of Arizona page 10 National Outcome Measures (NOMs) analyzed by auditors • Fewer consumers reporting recent substance use • Fewer consumers with recent criminal activity • More consumers employed • Fewer consumers who are homeless Source: U.S. Department of Health and Human Services Substance Abuse and Mental Health Services Administration. that more than half of the consumers who entered treatment reported abstaining from alcohol or drugs during the 30 days before starting treatment. Nearly all of these consumers reported that they were still abstaining at the time they left the program. However, of those who were not abstaining when they entered treatment, over 70 percent reported no change in their substance use. Overall, the analyses found little change in the areas of employment, criminal activity, and homelessness. Specifically: Slight overall improvement in substance use—Most consumers who auditors analyzed did not change their use of drugs or alcohol from entering treatment to their annual update or discharge. Approximately half of the consumers were abstinent upon entering treatment and were still abstinent at their update or discharge, while approximately one-third of the consumers who were using drugs or alcohol when they entered treatment were still using drugs or alcohol at their update or discharge (see first textbox on this page). Out of the more than 50,000 consumers auditors analyzed, a total of 5,767 came in using drugs or alcohol and either stopped or reduced use. Approximately 2,400 consumers got worse. Almost all consumers abstaining at the start of treatment remained abstinent—Auditors’ analysis of consumers’ self-reported data showed that more than half (54.1 percent) reported they were not using a substance at the beginning of treatment. More than 93 percent of these consumers reported still being abstinent at their update or disenrollment (see textboxes bottom of this page and top of page 12). According to officials at the Division, RBHAs, and providers, several factors may explain why so many consumers report they are already sober upon entering treatment. For example, some consumers may have quit drinking or using drugs but enter treatment when they realize they cannot maintain sobriety without help. Others may misrepresent their substance abuse, either to make themselves look better or because of wishful thinking if they have been trying to stop for a period of time. Still other consumers may have a problem with binge drinking at irregular intervals. Finally, some consumers enter treatment after spending time in forced abstinence in jail or prison and mistakenly report they have been abstinent, although they are supposed to report on their use prior to entering jail or prison. Office of the Auditor General page 11 Usage outcomes for all consumers1 1 Numbers do not total 100 percent because of rounding. Source: Auditor General staff analysis of division data for adult substance abuse consumers without a serious mental illness enrolled during fiscal years 2006, 2007, or 2008. Consumers who reported using alcohol or drugs when treatment started Reduced use 770 1.5% Stopped using 4,997 10.0% Stayed the same 16,504 33.0% Got worse 693 1.4% Consumers who reported being abstinent when treatment started Stayed abstinent 25,376 50.7% Did not stay abstinent 1,729 3.5% Total 50,069 Extent to which consumers reported being abstinent from alcohol or drug usage when treatment started Source: Auditor General staff analysis of division data for adult substance abuse consumers without a serious mental illness enrolled during fiscal years 2006, 2007, or 2008. Abstinent 27,105 54.1% Using alcohol or drugs 22,964 45.9% Total 50,069 Most consumers using alcohol or drugs at the start of treatment continued to do so—Among the consumers who reported that they were using alcohol or drugs in the 30 days prior to treatment, the majority—nearly 72 percent— reported no change in the frequency of use of their primary substance at their update or disenrollment, while 3 percent actually increased their frequency of use (see textbox, middle of this page). The remaining approximately 25 percent reported either reducing their use of alcohol or drugs or stopping use altogether. For consumers who started treatment using drugs or alcohol, completing treatment appears to be a factor in decreasing substance use. Those who completed treatment had greater success in stopping their drug or alcohol use than those who left without completing treatment. More specifically, among this group, only 17.6 percent who did not complete treatment stopped using drugs or alcohol, but 27 percent of those who completed treatment stopped (see textbox, bottom of this page). Percentage of consumers with recent arrests showed little change—Auditors’ analysis of division data found no significant change in criminal activity as measured by recent arrests. Altogether, 21.2 percent of consumers reported arrests within the past 30 days upon entering treatment, while 18.1 percent reported new arrests at the time of their update or disenrollment date. Most consumers reported not having any arrests within the 30 days prior to starting treatment and only a small number of those consumers had a new arrest at the time of their update or disenrollment (see textbox, page 13). By contrast, of the consumers who reporting being arrested within 30 days of entering treatment, although approximately one–third reported no new arrests as of their annual update or enrollment, the majority—68.1 percent—reported at least one more arrest when they had their annual update or disenrollment. NARBHA, the RBHA for Northern Arizona, showed the greatest reduction of criminal activity among consumers who had been arrested within 30 days of entering treatment. Specifically, 41.2 percent of these NARBHA consumers did not have a new arrest when they had their update or disenrollment. State of Arizona page 12 Usage outcomes for consumers who reported using alcohol or drugs when treatment started1 Source: Auditor General staff analysis of division data for adult substance abuse consumers without a serious mental illness enrolled during fiscal years 2006, 2007, or 2008. 1 Numbers do not total 100 percent because of rounding. Usage outcomes for consumers who reported being abstinent when treatment started Source: Auditor General staff analysis of division data for adult substance abuse consumers without a serious mental illness enrolled during fiscal years 2006, 2007, or 2008. Stayed abstinent 25,376 93.6% Did not stay abstinent 1,729 6.4% Total 27,105 Reduced use 770 3.4% Stopped use 4,997 21.8% Stayed the same 16,504 71.9% Increased use 693 3.0% Total 22,964 Usage outcomes for consumers who reported using alcohol or drugs when treatment started and who completed treatment Source: Auditor General staff analysis of division data for adult substance abuse consumers without a serious mental illness enrolled during fiscal years 2006, 2007, or 2008. Excludes consumers who were still in treatment or who disenrolled for administrative reasons such as transfers between RBHAs, and 278 consumers who enrolled as a result of a crisis and then disenrolled. Reduced use 121 1.7% Stopped use 1,894 27.0% Stayed the same 4,855 69.2% Got worse 144 2.1% Total 7,014 Usage outcomes for consumers who reported using alcohol or drugs before treatment and who did not complete treatment Reduced use 432 3.4% Stopped use 2,270 17.6% Stayed the same 9,771 75.8% Got worse 410 3.2% Total 12,883 Employment did not significantly change—The Division’s programs have not led to a significant change in overall consumer employment. Thirty-eight percent of substance abuse treatment consumers, or 16,360, reported that they were already employed when they entered treatment, and 41 percent reported being employed as of their annual update or disenrollment. Of those consumers who were already employed, more than 92 percent reported being employed as of their annual update or disenrollment (see textbox, page 14). Among the almost 27,000 consumers, or 62 percent, who reported not being employed upon entering treatment, nearly 10 percent reported that they had gained employment as of their annual update or disenrollment, although 90 percent were still unemployed. NARBHA showed the most improvement in this outcome area, as 16.1 percent of NARBHA’s unemployed consumers had gained employment as of their update or disenrollment. Homelessness showed little change—Overall, the number of consumers who were homeless did not change from entering treatment to update or disenrollment, with 7.1 percent homeless upon entering treatment and 6.8 percent homeless at update or disenrollment. Most consumers were not homeless upon entering treatment, and those who were homeless had some Office of the Auditor General page 13 Arrest outcomes for all consumers Arrest outcomes for consumers with no recent arrests when treatment started Arrest outcomes for consumers with recent arrests when treatment started Source: Auditor General staff analysis of division data for adult substance abuse consumers without a serious mental illness enrolled during fiscal years 2006, 2007, or 2008. Consumers with no recent arrests when treatment started Had new arrest before leaving treatment 1,907 3.7% Did not have new arrest 38,565 75.1% Had new arrest before leaving treatment 7,408 14.4% Did not have new arrest 3,468 6.8% Total 51,348 Consumers with recent arrests when treatment started Had new arrest before leaving treatment 1,907 4.7% Did not have new arrest 38,565 95.3% Total 40,472 Had new arrest before leaving treatment 7,408 68.1% Did not have new arrest 3,468 31.9% Total 10,876 success moving into stable housing situations. Specifically, almost 46,000, or 93 percent, of substance abuse consumers reported living in a stable housing situation upon entering treatment, and almost 99 percent of these consumers reported being in stable housing at their annual update or disenrollment (see textbox, page 15). Of the more than 3,500 consumers, or 7 percent, who reported not being in stable housing when they entered treatment, approximately 21.7 percent of these consumers reported gaining stable housing by the time of their annual update or disenrollment. Housing services are generally not available to substance abuse treatment consumers. Such services are not covered by Title XIX, and state monies only provide the services for consumers with serious mental illness. Still, increased housing stability is considered an important measure of substance abuse treatment success. Limited comparative information indicates improvements in Arizona’s program may be low—As part of the analysis, auditors compared Arizona’s NOMs with those reported by other states. The Division uses this information to compare Arizona outcomes to other states. The available data, while limited in quality and consistency, suggests that Arizona’s performance is below that of substance abuse programs in other states. The limitations in the State of Arizona page 14 Employment outcomes for all consumers1 Employment outcomes for consumers who reported being employed before starting treatment Employment outcomes for consumers who reported being unemployed before starting treatment 1 Numbers do not total 100 percent because of rounding. Source: Auditor General staff analysis of division data for adult substance abuse consumers without a serious mental illness enrolled during fiscal years 2006, 2007, or 2008. Consumers who were employed before starting treatment Unemployed when treatment ended 1,255 2.9% Employed when treatment ended 15,105 34.9% Unemployed when treatment ended 24,245 56.0% Employed when treatment ended 2,656 6.1% Total 43,261 Consumers who were unemployed before starting treatment Unemployed when treatment ended 1,255 7.7% Employed when treatment ended 15,105 92.3% Total 16,360 Unemployed when treatment ended 24,245 90.1% Employed when treatment ended 2,656 9.9% Total 26,901 quality of the comparative data stem from differences in how state programs are structured and what the states may be reporting. For example, the data that Arizona reports to SAMHSA includes substance abuse treatment for consumers with serious mental illness, which probably contributes to the lower percentages of consumers meeting the measurement. Whether other states similarly report people with serious mental illness as part of their substance abuse cases is unknown. Thus, while the comparison may be useful in providing a general indication of where Arizona stands relative to other states, it should not be taken as authoritative. Even so, however, Arizona’s apparent ranking relative to other states is another reason to examine the program carefully to determine if it can be improved. Auditors examined the national data collected by SAMHSA at the federal level and found that both nationally and in the SAMHSA-designated Western Region, the percentage of Arizona’s substance abuse treatment consumers meeting the measurement criteria is below average in three of the four areas auditors analyzed: abstinence, criminal activity, and employment.1 This is the case both for consumers who were entering treatment and for consumers at the time of their discharge. Further, the rate of change for Arizona’s consumers from admission to Office of the Auditor General page 15 Housing outcomes for all consumers1 Housing outcomes for consumers reporting they were not homeless before starting treatment Housing outcomes for consumers reporting they were homeless before starting treatment Source: Auditor General staff analysis of division data for adult substance abuse consumers without a serious mental illness enrolled during fiscal years 2006, 2007, or 2008. Consumers who were not homeless before starting treatment Homeless at update or discharge 612 1.2% Not homeless at update or discharge 45,282 91.6% Homeless at update or discharge 2,763 5.6% Not homeless at update or discharge 765 1.5% Total 49,422 Consumers who were homeless before starting treatment Homeless at update or discharge 612 1.3% In stable housing at update or discharge 45,282 98.7% Total 45,894 Homeless at update or discharge 2,763 78.3% In stable housing at update or discharge 765 21.7% Total 3,528 1 Numbers do not total 100 percent because of rounding. 1 SAMHSA’s designated western states reporting NOMs data as part of their fiscal year 2009 SAPT block grant application were Alaska, Arizona, California, Hawaii, Idaho, Montana, Nevada, Oregon, Washington, and Wyoming. discharge is below both the national average and the average for western states on all four measures—that is, the substance abuse programs in other states appeared to have a greater effect in raising outcomes in these other states. (See Appendix A, Figures 6 through 9, 11, 12, 14, 15, 17, and 18, pages a-vi through a-xxi, for comparisons of Arizona’s reported NOMs with other states’ reported NOMs, both regionally and nationally.) Case studies help further explain factors affecting treatment—To provide additional perspective on the program, auditors also analyzed a limited set of specific cases to understand consumers’ progress or lack of progress and the factors that affected success. Auditors examined case files from specific treatment episodes and then interviewed the consumers, treatment providers, and others involved in the case to determine long-term outcomes and what consumers thought contributed to their abstinence or use. Two such cases help illustrate the complicated factors that can affect individual long-term outcomes: Juan, 44, a married man with children who lives in southwestern Arizona, entered substance abuse treatment after being arrested for driving with a blood alcohol level of 0.17 and charged with extreme DUI.1 The court suspended his driver’s license and ordered him to undergo treatment. The court referred Juan to a local mental health agency that has a contract with the RBHA to provide substance abuse counseling in his area. The provider placed Juan in a 9-week substance abuse and relapse prevention program consisting of twice-weekly group therapy focused on education and relapse prevention. At Juan’s intake session with the provider, he stated that prior to his arrest he did not drink very often, but would have seven or eight beers when he was out with his friends. Although the reason he went to substance abuse treatment was to clear up his legal troubles related to the DUI, Juan reported to auditors that the program gave him insight into the dangers of drinking and driving and the impact his drinking had on his family. Juan completed his treatment in December 2005, and when auditors interviewed him in February 2009, he reported that he had not had a drink in 3 years. In this case, the consumer’s NOMs outcomes were unchanged from beginning treatment to completion of treatment because he had a high level of achievement before entering treatment. Juan had never been arrested before. He was employed and living in a stable family situation, and he reported no alcohol use within the 30 days before he entered treatment as well as within the 30 days before he was disenrolled. Dana, 19, entered treatment in northern Arizona after being arrested and serving 60 days in jail on charges of drug sale, possession of drug paraphernalia, and shoplifting. Two of her charges were felonies. Her State of Arizona page 16 1 All consumer names have been changed. substance abuse treatment was required as part of her probation. She had been using alcohol since age 13 and methamphetamine since age 16, and had had previous substance abuse treatment and periods of abstinence. At her intake, Dana stated that she needed supervision and did not trust herself to abstain. Her provider recommended a month of residential treatment and then moved Dana into a two-stage intensive outpatient program. Over the 11 months of her treatment, she had considerable positive change in her NOMs. At her enrollment, she had just been released from jail, was unemployed, was not attending school, and had no stable place to live. At disenrollment she was self-supporting, enrolled in college, living with a family friend, and had no additional arrests. She was involved in Narcotics Anonymous and was not using drugs or alcohol. In Dana’s case, the NOMs data showed substantial progress on all four measures. However, Dana was unable to sustain these changes and relapsed about 1 year after completing treatment. When auditors spoke with her in February 2009, she was in prison with 5 more years to serve on a drug-related conviction. Case studies such as these, together with the results of the auditors’ analyses of the four NOMs, illustrate the difficulty in making decisions about how to improve program success. For example, the Division captures NOMs data at various points during treatment, but the data alone does not adequately indicate whether or not treatment has been successful. Auditors’ review of studies, best practices, and other materials showed, however, that improvements in program outcomes are likely to come from focusing on three main strategies: increasing the number of consumers who complete their treatment program, providing a continuum of care that addresses multiple needs, and using practices that have been shown to generate positive outcomes. The sections that follow discuss each of these strategies in turn. Division should increase focus on treatment retention and completion Research and the Department’s own data indicate that consumers who remain longer in treatment experience better outcomes. However, most consumers state-wide do not complete their treatment. As the analysis discussed on page 12 indicated, those who complete treatment experience better outcomes. Several factors, such as individual motivation and the availability of different treatment options, can influence whether or not consumers stay in treatment. The Division can take several steps to increase consumer retention and completion rates, including establishing performance goals, monitoring completion rates, and using financial and/or nonfinancial incentives and other case management techniques. Office of the Auditor General page 17 Longer treatment yields better results—Research has shown that consumers who remain longer in treatment, no matter what type of treatment it is, show the best post-treatment outcomes.1 For example, studies have shown that longer treatment duration resulted in lower readmission rates and lower rates of drug use and relapse. One study found that consumers who received long-term residential or outpatient treatment for 6 months or more showed a lessened likelihood of engagement in illegal activities than those with shorter lengths of stay.2 Another reported that longer lengths of stay were positively associated with post-treatment client earnings across all types of treatment.3 The National Institute on Drug Abuse states that most consumers need to remain in treatment for about 3 months in order to achieve significant improvement, and additional progress toward recovery can be achieved by providing treatment after the 3-month threshold.4 However, a 2003 study suggested that consumers who stay in treatment for longer than 18 months may start to show reduced amounts of improvement.5 Ensuring treatment completion is an important strategy for improving consumers’ outcomes. Auditors’ analysis of division data found that in addition to being more likely to reduce or stop substance use, consumers who completed treatment were more likely to gain employment and avoid criminal activity than those who did not complete treatment. For example, of the consumers who were unemployed when they entered treatment, 14.4 percent of those who completed treatment gained employment, while only 6.2 percent of noncompleters gained employment. Similarly, of the consumers who had been arrested within the 30 days prior to entering treatment, 47.5 percent of those who completed treatment had no new arrests at their update or discharge, but only 27.5 percent of those who did not complete treatment had no new arrests. Most consumers drop out or receive short-term treatment—Despite the importance of retention, auditors’ analysis of division data found that a majority— 58 percent—of the consumers reviewed did not complete their treatment before they were disenrolled. All six geographic services areas had fewer consumers who completed treatment than those who did not complete treatment (see Figure 3, page 19). The most common reason given for disenrolling consumers who did not complete treatment was that providers lost contact with them (38.3 percent of all reviewed consumers), while 9.4 percent of reviewed consumers refused further treatment. According to the Division, without a court order, consumers’ treatment is voluntary and they can withdraw at any time. Other reasons for disenrollment are death, incarceration, and a loss of Title XIX status. Further, in contrast to the 3 months suggested in literature, the Division reported that in fiscal year 2007, its average length of stay for outpatient treatment was 29 days and for long-term residential care the average length of stay was 78 days. In that fiscal year, more than 60,000 consumers were treated in outpatient programs, while fewer than 700 were treated in residential, long-term programs. State of Arizona page 18 1 McLellan, Chalk, & Bartlett, 2007 2 Etheridge, Craddock, Hubbard, & Rounds-Bryant, 1999 as cited in Koenig, Harwood, Sullivan, & Sen, 2000 3 Koenig, Harwood, Sullivan, & Sen, 2000 4 National Institute on Drug Abuse, 2009 5 Zhang, Friedmann, & Gerstein, 2003 Studies show that treatment retention and completion result in better outcomes. Office of the Auditor General page 19 39.4 48.7 37.6 43.3 44.5 42.5 50.8 60.6 51.3 62.4 56.7 55.5 57.5 49.2 0 10 20 30 40 50 60 70 NARBHA (GSA 1) Cenpatico (GSA 2) CPSA (GSA 3) Cenpatico (GSA 4) CPSA (GSA 5) Magellan/VO (GSA 6) Tribes Percent of Consumers Completed Treatment Did Not Complete Treatment Figure 3: Treatment Completion and Noncompletion Rates by RBHA (GSA) Fiscal Years 2006 through 2008 Source: Auditor General staff analysis of data from the Division’s Client Information System (CIS) for adult substance abuse treatment consumers without a serious mental illness enrolled in fiscal years 2006, 2007, or 2008. State of Arizona page 20 Several factors affect retention—Whether a person stays in treatment depends on factors associated with both the individual and program, according to the National Institute on Drug Abuse (Institute). The Institute’s Principles of Drug Addiction Treatment states that “individual factors related to engagement and retention include motivation to change drug-using behavior, degree of support from family and friends, and whether there is pressure to stay in treatment from the criminal justice system, child protective services, employers, or the family. Within a treatment program, successful counselors are able to establish a positive, therapeutic relationship with the patient. These counselors also ensure that a treatment plan is established and followed so that the individual knows what to expect during treatment.”1 Similarly, the Treatment Research Institute (TRI) states that retention can be improved by making treatment attractive, offering options, increasing monitoring and management, and giving consumers feedback.2 The TRI points out that consumers who are not in some form of treatment or monitoring are at increased risk of relapse. Other researchers have also reported that providing incentives to consumers, such as a reward for a clean urinalysis test, can help increase retention. Division can do more to increase retention—The Division does not monitor treatment retention or length of stay, and although it collects consumer disenrollment reasons it does not analyze or monitor the data. Its contracts with the RBHAs do not require them to collect or report information on retention or length of both inpatient and outpatient stay. The contracts require efforts to re-engage consumers in treatment—specifically, the RBHAs must require their providers to attempt re-engagement, and the Division’s provider manuals also establish specific requirements regarding attempting to re-establish contact with consumers—and the Division’s Administrative Review monitors RBHA’s compliance with this standard. However, auditors’ analysis of division data found that 47.7 percent of consumers could not be re-engaged in treatment (38.3 percent could not be contacted by providers and 9.4 percent refused further treatment), and the 2007 Administrative Review showed all four RBHAs needed improvement on the re-engagement measure. One of the auditors’ case examples illustrates this. Dana—the methamphetamine user who relapsed after treatment completion and was in prison when auditors interviewed her in February 2009— had spotty attendance during her treatment, including a period in May 2006 where she attended less than half her group sessions. Her case manager did not address her attendance issue until Dana asked about being released from treatment. Then, she was informed that her probation required her to maintain acceptable attendance in order to be considered for graduation from the program. Some other states have adopted goals and incentives related to retention and treatment completion. For example: 1 National Institute on Drug Abuse, 2009 2 McLellan, 2006 Office of the Auditor General page 21 Colorado—To meet performance goals, providers must maintain or improve the average percentage of clients from the previous 2 fiscal years who remain in outpatient treatment longer than 90 days, and maintain or decrease the percentage of consumers who left treatment against professional advice. Maine—Performance goals stipulate a minimum of 50 percent of outpatient and 85 percent of intensive outpatient consumers stay at least four sessions. Additionally, 30 percent of outpatient consumers must stay in treatment 90 days or longer, and 50 percent of intensive outpatient treatment consumers must complete treatment. Providers who exceed these expectations are awarded incentive bonuses, while underperforming providers are subject to disincentive penalties. Delaware—Contracts stipulate a $100 bonus incentive to providers for each consumer who completes at least 60 days in treatment and achieves the major goals of his or her treatment plan, including submitting four consecutive drug-free urine screenings randomly administered. To help improve treatment outcomes, the Division should establish performance goals and financial or nonfinancial incentives and disincentives related to retention and treatment completion in its RBHA contracts, and collect and monitor data on retention and completion. In establishing benchmarks and incentives, the Division should consider the potential for unintended consequences such as retaining consumers inappropriately and take steps to ensure that the incentives achieve the desired goals. For example, it could follow Delaware’s example of combining retention with other goals in order to determine whether providers qualify for incentives. In addition, the Division should use its existing oversight practices such as its quarterly case reviews to determine whether providers are taking appropriate steps to retain and engage consumers in treatment. Based on the results of these reviews, the Division should work with the RBHAs to address weaknesses through mechanisms such as technical assistance, training, contract requirements, and/or policy and procedural changes. Division should monitor continuum of care A second strategy for improving consumer outcomes is to provide a continuum of care. Under a continuum of care approach, consumers are placed in treatment based on the types of services they need and the phase of recovery they are in. After initial placement, consumers then step up to more intensive treatment or down to less intense treatment as needed. However, as illustrated by case studies, not all consumers have their treatment needs met or receive the services and therapies that may improve the chances of good treatment outcomes. The Division can do more to ensure continuum of care for consumers by taking several steps, such as collecting and monitoring data relevant to assessment and case management, and working with the RBHAs to make improvements when weaknesses are identified. State of Arizona page 22 Continuum of care matches consumers to appropriate treatment based on their problems and stage of recovery—Because no single treatment is appropriate for all individuals, providers need to be able to match treatment settings, interventions, and services to each individual’s particular problems. Continuum of care is a treatment system in which consumers entering treatment are placed in a level of treatment appropriate to their assessed needs. Individuals are first assessed to determine appropriate treatment services. Successful matching of the consumers to the appropriate treatments for them helps to ensure positive outcomes.1 After initial placement, consumers then step up to more intensive treatment or down to less intense treatment as needed. As the consumers recover, and further assessments are made of consumers’ progress, the treatment is phased down and consumers can be moved into alternative community-based services. Some consumers receive continuum of care while others do not— One of the auditors’ case studies illustrates how treatment that is appropriate to a consumer’s needs can yield long-term results: Mandy, 29, was in treatment for nearly a year. When she contacted a provider, she had already had previous experience with treatment, she was pregnant, her husband was in jail, and her two children were living with relatives. She reported that she had used methamphetamines, alcohol, and marijuana within the past several weeks. After completing a month of treatment with her first provider, she was referred to a specialized residential treatment center for women who are pregnant or have children that offers integrated services designed to teach them to be better parents and give them skills they need to get jobs, take care of their children, and stay sober. As a pregnant woman and therefore part of a priority population, Mandy’s treatment was eligible for SAPT funding. During almost 1 year of treatment with this second provider, Mandy made the decision to end her marriage and place her baby with adoptive parents in an open adoption. At the end of treatment, Mandy had a job, was living with her mother and one of her sons, and was working on her relationship with her other son. When auditors interviewed her in April 2009, 2 years after she completed treatment, Mandy stated that long-term treatment that focused on all her issues gave her a solid foundation for a sober life. She said she no longer had any cravings or desire to use drugs. During her year of integrated treatment, she learned to address the issues that led to her addiction. Her previous short-term treatments had not given her the ability to reflect on the information provided during treatment. Mandy now works as a peer support specialist in the substance abuse field, has custody of one of her sons, and as of April 2009 was about to get custody of her other son. In addition, she was living independently and was in contact with her daughter’s adoptive parents. Continuum of care is an important strategy in ensuring treatment is successful. 1 Mee-Lee, n.d. Office of the Auditor General page 23 By contrast, another example, identified through auditors’ analysis of high-cost consumers, illustrates a failure to provide services along the continuum from acute detoxification and stabilization through rehabilitation to continuing recovery support: Rodney, 43, used medical detoxification services 54 times during the period July 2005 through June 2008—approximately every 3 weeks—receiving services valued at a total of more than $82,000. (See Finding 2, pages 31 through 42, for a discussion of service costs). Rodney was homeless and reported using alcohol daily. Although he received some case management and other crisis services besides detoxification, he received very little in the way of other services, such as treatment that would reinforce his sobriety after detoxification and rehabilitation to help him learn to remain sober in the long term. According to a case manager at the detoxification center, Rodney typically stays in the center for only 2 to 3 days and chooses not to complete treatment or receive follow-up care. A consumer like Rodney who undergoes only the first stage of care is likely to return to substance abuse. Further, when auditors discussed this case with the Division’s medical director, he expressed concern that Rodney’s health is jeopardized by this constant cycling between heavy drinking and medical detoxification. Consumers are sometimes placed into services that may not meet their individual needs—contrary to the continuum-of-care concept—or that duplicate services they have already received. Auditors’ case studies included three consumers who received the same treatment program from different providers in different parts of the State: Juan, discussed previously, and two other consumers, Tom and Victoria. All three received alcohol education classes designed for people who have received their first DUI or alcohol-related arrest. These classes are intended to get participants to recognize problematic behaviors and make better choices in the future. For Juan, who had his first arrest for DUI at the age of 44, the class was an effective wake-up call regarding the dangers of impaired driving and how drinking affected his family relationships. He told the auditors that his 9 weeks of classes were very helpful to him and improved his life. Tom and Victoria, however, had long histories of alcoholism and completed the classes after serving prison sentences for drunken driving. Both Tom and Victoria had received similar classes while in prison. Tom told auditors that going through the same treatment after his release was not helpful and did not contribute to his continued sobriety. Instead, Tom and Victoria felt they needed treatment that would have helped them adjust to life after prison and address other issues. Both said their continued sobriety resulted from their own desire to avoid further legal troubles. In addition to auditors’ case studies that found that continuum of care is not consistently and appropriately provided, a 2008 study by one RBHA also found cases where consumers did not receive needed substance abuse services. Auditors’ case studies and a RBHA study revealed that not all consumers’ needs were met. 1 De Leon, Melnick, & Cleland, 2008 2 McLellan, Weinstein, Shen, Kendig, & Levine, 2005b State of Arizona page 24 Specifically, the Community Partnership of Southern Arizona (CPSA) conducted a focus review in August 2008 of a psychiatric health facility in its network and found that there was little evidence to support that the facility provided active treatment to the ten consumers sampled in the review, including three consumers who required substance abuse treatment (For more information on this review, see Finding 2, pages 31 through 42). The review found that none of the three consumers received substance abuse treatment services during their 7- to 10-day inpatient stays at the facility. Continuum of care requires that consumers’ needs are assessed and that they receive treatment that matches their needs. Division can do more to ensure continuum of care—The Division’s contracts with the RBHAs already specify that providers are required to develop treatments and services designed to support long-term recovery and focus on life factors such as employment, ongoing feedback to the recipient, and re-engagement into treatment based on the recipient’s changing needs. However, the Division’s oversight processes have not focused on ensuring that the RBHAs have implemented these requirements for substance abuse consumers (See Finding 2, pages 31 through 42, for recommendations on improving oversight of substance abuse treatment). As the Division moves to ensure that the RBHAs implement a more effective continuum of care, it should adopt two approaches shown by research to improve consumer outcomes. Specifically: Establishing assessment standards—The Division should establish standards for assessing the severity of consumers’ substance abuse problems and referring consumers to appropriate treatment, and monitor implementation of these standards as part of its regular oversight of RBHA performance. A 2008 study found that assessing the severity of a consumer’s substance abuse problem and then assigning more severe consumers to more intensive treatment improves both outcomes and cost-effectiveness of treatment.1 The study reported that appropriate placement does not require a complex matching process where individuals are matched with a specific style of treatment, such as cognitive behavioral therapy versus a 12-step program (see textbox, page 25). Defining case management expectations—The Division should define appropriate expectations for case management of substance abuse consumers, and consider assigning consumers with severe or complex cases to a case manager for their complete course of treatment, taking into consideration the costs of case management. For example, effective clinical case management can reduce inefficient over-use of detoxification. A 2005 study in Philadelphia concluded that multiple episodes of detoxification without follow-up care are inappropriate, ineffective, and inefficient.2 Pairing detoxification with clinical case management resulted in a 55 percent Office of the Auditor General page 25 reduction in detoxification admissions and a 70 percent increase in use of rehabilitation. Clinical case managers were expected to engage consumers early and assess their substance use, health, and social problems; use motivational interviewing designed to help consumers recognize and accept the need for continued care; and play an active, participative role in helping the consumers access and continue with needed medical and social services (See Finding 2, page 40, for information on how the Division can use case management to help minimize costs). When applying these approaches in its management of the program, the Division should also collect and monitor data relevant to assessment and case management and work with the RBHAs to make improvements when its oversight identifies weaknesses. Using appropriate evidence-based practices can improve success rate A third strategy the Division can employ is to increase its attention to providers’ use of evidence-based practices. These practices are validated by observation or experience as improving treatment success. Although the Division requires the RBHAs to use evidence-based practices, RBHAs are not necessarily doing so, and the Division is not ensuring compliance. To improve adherence to evidence-based practices, the Division should enhance its monitoring of the practices and take appropriate steps to address any weaknesses it finds. Evidence-based practices important to success—Evidence-based practices are therapies confirmed by scientific research to have a positive effect in the treatment of substance abuse. They include motivational interviewing, cognitive behavioral therapy, community reinforcement therapy, and 12-step facilitation therapy (see textbox). There is a growing body of literature on the cost-effectiveness of well-defined treatments that have been validated by observation or experience for substance use disorder, according to a 2007 study.1 SAMHSA has developed the National Registry of Evidence-based Programs and Practices, a listing of programs verified by independent researchers for their ability to achieve 1 Olmstead, Sindelar, Easton, & Carroll, 2007 Examples of Evidence-based Practices • Motivational Interviewing—A counseling style designed to help consumers recognize and accept the need for continued care. (www.motivationalinterview.org) • Cognitive behavioral therapy—A treatment that seeks to change the way an individual thinks to feel/act better even if the situation does not change. (www.nacbt.org) • Community reinforcement therapy—Therapy that focuses on events that influence behavior, such as family and social events, and uses the events to support successful abstinence. (www.ncsacw.samhsa.gov) • 12 step facilitation therapy—A brief, structured therapy that supports initial and ongoing participation in a 12- step program such as Alcoholics Anonymous. (www.nrepp.samhsa.gov) • ASAM criteria—Placement guidelines that focus on assessing consumers across six dimensions such as emotional and behavioral conditions and readiness to change and then placing the consumer in a level of care to match the severity of his/her condition. (www.asam.org and www.coce.samhsa.gov) • Stages of change—A model of behavioral change that progresses from examining the risks of change to commitment to action, and then plan, implement, and maintain the change. (www.coce.samhsa.gov) Source: Auditor General staff summary of evidence-based practices as defined on the above Web sites on June 24, 2009. State of Arizona page 26 results when properly implemented. The Division’s review of residential treatment centers supports the use of such evidence-based practices and the results that can be achieved. The Division contracted with a private firm to review nine substance abuse residential treatment centers that provide such services mainly to women to gather baseline data. In that review, all nine centers reported that they used at least two forms of evidence-based best practices. The three most commonly reported practice models across all nine centers included the American Society of Addiction Medicine (ASAM) criteria, motivational interviewing, and stages of change. The report indicated that 97.5 percent of the cases reviewed showed evidence of symptomatic improvement. Evidence-based practices are also used for outpatient treatment. For example, there are three Methamphetamine Treatment Centers of Excellence located throughout the State, which specialize in the treatment of methamphetamine use through techniques such as incentives, community reinforcement, motivational interviewing, and monitoring behavior through urine testing. Requirement for evidence-based practices not necessarily followed or enforced—The Division’s contracts with the RBHAs require the use of evidence-based practices, but the Division can do more to ensure providers are using the most up-to-date research and implementing the practices in accordance with the evidence. The Division has received or conducted three reviews of substance abuse programs. For example, in one study, a 2008 federal grant review that focused on programs for children and adolescents, the Division did a good job of establishing evidence-based practices in some areas, but did not identify such practices across the continuum of care. It could also do more to ensure sustainability of the emphasis on evidence-based practices. In another study, the Division reviewed 11 intensive outpatient programs for youth across the State and found that, although the Division had an internal definition of what constitutes an intensive outpatient program, the definition was not documented or adequately communicated to the providers. Therefore, a number of the providers’ programs did not meet the Division’s definition. One example of implementation of an evidence-based practice for substance abuse treatment is at NARBHA, which received a SAMHSA grant to implement a program called Matrix for adult treatment services. NARBHA uses a tool provided by the Matrix Institute to evaluate providers and ensure the program is implemented properly. The Matrix model incorporates therapist support, group participation, support group participation, relapse prevention and education, family involvement, and an explicit structure giving consumers a clear understanding of treatment. According to staff at one of NARBHA’s providers, the program has helped them get consumers into the structure of recovery. One element of the model, contingency management—where consumers are rewarded for successful participation, for example with prizes such as movie passes—is seen as helpful for keeping consumers engaged in treatment. NARBHA uses an evidence-based practice for substance abuse treatment and monitors it to ensure proper implementation. Office of the Auditor General page 27 The Division may be able to use one evidence-based practice—self-help group participation—more effectively. Research has shown a strong association between self-help group participation and long-term recovery following treatment. Although none of the consumers included in auditors’ case studies continued to participate regularly in self-help groups after treatment, one of them, Dana—the young woman serving a drug-related prison sentence—attributed her return to drug use to when she stopped attending Narcotics Anonymous meetings and continued to associate with friends who used drugs. She indicated that she found it useful to have group leaders who were recovering addicts because their familiarity with addiction makes them good at breaking down addicts’ behavioral defense mechanisms. Division can do more to encourage use of evidence-based practices—Auditors identified two key ways in which the Division can encourage greater use of evidence-based practices: More extensive monitoring—The Division examines a sample of substance abuse consumers’ cases each quarter as part of its overall oversight of the RBHAs in order to ensure consumers obtain services in a timely manner and receive appropriate services. In addition, in order to fulfill federal grant requirements, the Division has conducted a review of substance abuse programs for adolescents and contracted for a review of women’s substance abuse residential treatment facilities. Specifically, during 2007 and 2008, division staff reviewed 11 intensive outpatient programs for adolescents to determine how the programs could be improved. Also, in 2008, the Health Services Advisory Group, an external quality review organization contracted by the Division, conducted a study of women’s residential treatment facilities to gather baseline data on women’s substance abuse treatment programs. However, the Division does not routinely review services to determine whether the RBHAs are applying evidence-based practices or implementing them as they were designed to be used. Similar to NARBHA’s approach with its providers that use the Matrix model, the Division should monitor evidence-based practices either directly or by reviewing the RBHAs’ assessments of providers’ practices. Improved guidance—The Division should also continue and expand its work with RBHAs to ensure that providers have the guidance needed to implement specific evidence-based practices. When the Division implements new treatment models, it provides guidance and other assistance to ensure the RBHAs and providers implement the models correctly. For example, it used this approach to implement a new system of care for children in the early 2000s, as discussed in the Auditor General’s report on implementation of HB2003 programs for children (see Report No. 02-12). The Division also cosponsors an annual summer institute conference for RBHAs and providers. The Division should use the results of its monitoring of evidence-based The Division may be able to use self-help groups more effectively. practices implementation to identify any needed improvements, and then work with RBHAs to provide technical assistance, training, and guidelines as appropriate. Finally, the Division should make better use of self-help programs by developing a method to track and monitor participation and encouraging RBHAs to offer a wider variety of programs for consumers. Because self-help group participation can continue the benefits of treatment long after treatment is completed, the Division should also consider working with the RBHAs to develop procedures to follow up with consumers after treatment is completed to encourage continued participation in such groups. Recommendations: 1.1. To help improve retention, the Division should: a. Collect and monitor data on retention and completion, including length of stay and disenrollment reasons; b. Establish performance goals and financial and/or nonfinancial incentives and disincentives related to retention and treatment completion in its RBHA contracts, taking care to avoid encouraging providers to inappropriately retain consumers in treatment in order to meet the goals; c. Use its existing oversight practices such as its quarterly case reviews to determine whether RBHAs are taking appropriate steps to retain and engage consumers in treatment; and d. Based on the results of these reviews, the Division should work with the RBHAs to address weaknesses through mechanisms such as technical assistance, training, contract requirements, and/or policy and procedural changes. 1.2. To make better use of the continuum of care to improve treatment outcomes, the Division should: a. Establish standards for assessing the severity of consumers’ substance abuse problems and referring them to appropriate treatment; b. Using data, monitor implementation of these standards as part of its regular oversight of RBHA performance; c. Define appropriate expectations for case management of substance abuse consumers, taking into consideration costs of case management and the advantages of monitoring consumers with severe or complex cases; State of Arizona page 28 d. Collect and monitor data relevant to assessment and case management; and e. Work with the RBHAs to make improvements when its oversight identifies weaknesses. 1.3. To better ensure the use of appropriate evidence-based practices, the Division should: a. Monitor compliance with its contractual requirements to use evidence-based practices; b. Work with RBHAs to provide technical assistance, training, and guidelines as appropriate to ensure that providers have the guidance needed to implement specific evidence-based practices such as motivational interviewing, cognitive behavioral therapy, and community reinforcement therapy; c. Develop a method to track and monitor self-help group participation; d. Encourage RBHAs to offer a wider variety of self-help programs for consumers; and e. Consider working with the RBHAs to develop procedures to engage consumers in community and peer support outlets that would reinforce progress made in treatment once consumers are disenrolled. Office of the Auditor General page 29 State of Arizona page 30 Division should improve oversight of substance abuse programs The Department of Health Services, Division of Behavioral Health Services (Division), should take steps to improve its oversight of the substance abuse programs administered by the Regional Behavioral Health Authorities (RBHAs). Current oversight efforts are limited in two key ways. First, although the Division gathers information on treatment outcomes, it does not use this information to monitor programs, focusing instead on process-related information such as whether services are timely. To increase the benefit that treatment participants receive, the Division should continue its efforts to streamline uniform data collection that focuses on consumer outcomes, establish relevant performance goals in contracts with the RBHAs, and encourage the RBHAs to consider mechanisms for awarding financial and/or nonfinancial incentives to providers who meet these goals and penalizing those who do not. Second, oversight efforts currently lack any significant focus on analyzing treatment costs and types. This is particularly true for analyzing high-cost or low-cost cases that may indicate ineffective or inadequate services. To help better ensure that the Division’s scarce resources are used effectively, the Division should implement a systematic approach that includes but is not limited to reviewing high-and low-dollar cost services by individual users and treatment categories and among RBHAs, and considering how to best use assessment and case management to contain costs. Division should monitor outcomes The Division should improve oversight of substance abuse treatment by increasing its focus on consumer outcomes—that is, what the services are accomplishing in terms of keeping consumers from reverting to abuse of alcohol or drugs. The Division focuses primarily on treatment process measures for all behavioral health consumers, including substance abuse consumers, and it is therefore largely unable to determine if its substance abuse treatment programs are achieving positive results or if its resources are being used effectively. Focusing more on outcomes involves taking such steps as streamlining uniform outcome data collection, implementing minimum performance goals, and establishing performance incentives and penalties to help ensure that goals are met. Office of the Auditor General page 31 FINDING 2 Division monitors process, not treatment outcomes—The Division’s oversight of the behavioral health treatment system is limited to a number of process measures and is not focused on treatment results. For example, as required by its Arizona Health Care Cost Containment System (AHCCCS) contract, on a quarterly basis, the Division reviews the RBHAs’ ability to provide timely and appropriate services to consumers and coordination of care with the consumer’s primary care physician. Such measures are required for all Medicaid-covered services and provide information that is relevant to assessing whether consumers receive services, but they do not provide meaningful insight as to whether consumers are reducing their dependence on alcohol or drugs. The Division gathers consumer outcome data but does not appear to use it for monitoring purposes. The Division gathers outcome data such as information on abstinence and participation in employment or education when developing its Annual Report on Substance Abuse Treatment Programs, and when applying for the annual federal Substance Abuse Prevention and Treatment block grant. In addition, the Division reported that it uses outcome data to compare Arizona’s to other states’ outcomes. Division policies state that this information allows for the measurement of behavioral health outcomes, is required of all consumers, and is used to support quality and utilization management activities. However, auditors found no indications that the Division uses this information to evaluate RBHA performance in providing effective substance abuse treatment. The Division also does not use this outcome information to compare performance between RBHAs or between individual providers. As a result, the Division is unable to effectively evaluate system performance and determine which providers’ treatment services are resulting in positive consumer outcomes. Likewise, the Division is unable to identify underperforming providers and require that they make necessary improvements. Similarly, at the RBHA level, some RBHA representatives said that although they are required to gather the outcome data and submit it to the Division, they do not evaluate the outcome information as thoroughly as they could because they need more guidance from the Division on how this should be done. Therefore, the Division should provide more guidance to the RBHAs to help ensure they can thoroughly evaluate the consumer outcome information. Division should continue efforts to streamline uniform outcome data collection—The Division is revising its assessment process, and in doing so it needs to ensure the new process enables it to uniformly collect the data needed to assess consumer outcomes. The Division requires RBHAs to gather consumer outcome data during intake, update, and disenrollment assessments. Historically, according to division officials, providers have used a uniform core assessment tool mandated by the Division, but some have expressed concern about the length of the 32-page tool and the more than 1 hour that it took to fill it out. The tool, which is used for all the Division’s consumers, includes a full behavioral health assessment, not just a substance abuse assessment. During the audit, the Division began developing a new method for collecting data that would streamline the data collection process. The Division gathers consumer outcome data but does not use it for monitoring purposes. State of Arizona page 32 Because A.R.S. §36-2006(A) and (B)(4) require the Department to establish a standardized screening assessment for alcohol and other drug education and treatment programs, and because consistent outcome information is important for monitoring program success, the Division should continue in its efforts to streamline uniform collection of required outcome data. Another state has developed uniform requirements that are short, and the Division may be able to adopt its solution. Maine has developed a one-page form specifically for gathering consumer outcome information during assessments that includes standardized yes-or-no questions or provides a list of the answer options for each question. Adopting a similar form or modifying its existing method would address providers’ concerns about the form’s length while retaining the Division’s ability to effectively and uniformly collect, monitor, and easily validate outcome data as well as allowing it to comply with statute. Division should establish minimum outcome-based performance goals, incentives, and penalties—In 2006, a national policy panel convened to examine how state governments could be most effective in preventing and treating substance use disorders and problems, and reported that states should hold agencies and contracted providers accountable for meeting identified outcome measures and should reward those that meet or exceed outcome targets and penalize those that consistently fail.1 As discussed in Finding 1 (see pages 9 through 29), three states—Maine, Delaware, and Colorado— reported having established performance goals for providers on a variety of measures such as treatment retention, continuation of care, and abstinence. Delaware and Maine have both set a minimum number of consumers that must meet the goals while Colorado requires that providers maintain or improve their performance without specifying an exact number of consumers. However, a Colorado Division of Behavioral Health official reported that it will modify the performance goals in fiscal year 2010 to require that the providers obtain certain percentages rather than perform better than the average of their previous scores for 2 years. In addition, Colorado has established goals for consumer transition from detoxification to some other form of treatment (see textbox). Further, a fourth state, Nevada, is currently conducting a 3-year pilot program that includes performance goals. By contrast, although the Division has some contractual requirements regarding provider performance, it has not established benchmarks for clinical performance, or outcome goals that are specific to substance abuse treatment. 1 Rosenbloom, Leis, Shah, & Ambrogi, 2006 Office of the Auditor General page 33 Examples of Colorado’s performance goals Maintain or improve the percentage of detox clients who are referred to further treatment Maintain or improve the percentage of detox clients who enter any non-detox treatment setting within 30 days of their detox discharge Maintain or improve the percentage of treatment clients who stayed in outpatient treatment longer than 90 days Maintain or improve the average of treatment clients who had a reduction in primary drug use at discharge Source: Colorado’s fiscal years 2006 through 2008 state-wide treatment performance measures for all managed service organizations provided by an official from Colorado’s Department of Human Services, Division of Behavioral Health. In addition to performance goals, two states—Delaware and Maine—have incorporated financial incentives and disincentives into their performance goals that could serve as models for Arizona. For example, Delaware has established incentives for criteria such as patient participation and treatment retention/completion. For patient participation, the provider receives a 1 percent bonus if 50 percent of participating patients attend two sessions per week in the first 30 days of treatment. An incentive payment is also tied to completion, active participation for 60 days, achievement of the major goals of the treatment plan, and 4 consecutive weeks with clean urine samples. Maine also includes incentives and disincentives in its performance-based contract, which stipulates that their providers can collect a percentage of their quarterly payment each quarter they exceed their performance goals. Likewise, its providers can lose a percentage for each underperforming quarter. Division needs to establish benchmarks first, then incentives—The Division is not yet in a position to do what other states have done because it lacks clear benchmarks for measuring performance and incentives to encourage good performance. However, the Division has some contractual requirements regarding provider performance and financial incentives. For example, Magellan, the RBHA that oversees behavioral health services in Maricopa County, is required by contract to implement financial incentives with its service providers to increase performance on the National Outcome Measures by September 2009. According to a Magellan official, the RBHA is working with its providers to implement performance goals and to put a process in place for awarding incentives or levying penalties. Other RBHAs’ contracts state that the RBHA will share incentives originating from the Division with applicable providers. However, before performance incentives, or penalties, can be levied, the Division must first establish performance benchmarks or standards for treatment retention and completion and consumer continuation of treatment from detoxification to some other form of treatment and modify its contracts with the RBHAs to require such measures. According to a division official, the Division is exploring possible benchmarks for performance incentives or penalties, but difficulties in collecting and analyzing the necessary data have hampered efforts in establishing benchmarks. In April 2009, the Division re-organized its data management staff in order to address the data collection and analysis issues. The Division should encourage the RBHAs to consider contractually implementing a financial or nonfinancial method of incentivizing substance abuse treatment providers who exceed the goals established in their contracts and penalizing those providers who continually fail to meet these standards related to consumer outcomes, treatment retention, and treatment completion. Magellan is required by contract to implement financial incentives with its service providers to increase performance by September 2009. State of Arizona page 34 Division should monitor monies spent on treatment The second area in which improved monitoring is needed is more reviews of service costs. Reviews of over- and underutilization of healthcare services are important to help contain costs and manage quality of substance abuse treatment. Although the Division contractually requires the RBHAs to perform such reviews of services, the reviews should be expanded to identify unusually high and low costs of services to individual consumers and high treatment or services costs at each RBHA, and monitor variations in treatment across all RBHAs. The Division should continue and expand on the steps it took during the audit to begin reviewing service costs. In addition, the Division should also consider how to best use assessment and case management to contain costs. Lastly, the Division should continue its efforts to fill vacant positions and should perform follow-up work to ensure that the restructuring it initiated in April 2009 results in improved oversight. Utilization reviews can help contain healthcare costs and maintain quality of care—In a managed care system like Arizona’s, utilization reviews are critical to help control service costs while ensuring that quality of care remains high (see textbox). Because the Division uses a capitated system to pay RBHAs in advance for providing all required services, current-year costs are not reflected in current-year payments as they would be in a fee-for-service system. Rather, each year’s capitation rate is established based on analyzing actual costs of services delivered in previous years, so future payments are affected by current-year service costs. Research has shown that the cost-containment goals of managed care systems can be obtained without sacrificing client outcomes. However, attention must be paid to both cost-containment strategies and quality-of-care issues (such as evidence-based practices) if clients are to benefit from substance abuse treatment. Specifically, 2005 and 2006 studies that compared one California county that used managed care to two other counties that used a more traditional fee-for-service approach to substance abuse treatment found that the managed care approach was more cost-effective—it achieved the same results at a lower cost.1 The managed care county established various best practices to manage both cost and promote better outcomes including clear assessment guidelines, case management guidelines for complex cases, continuum of care requirements, performance-based contracting, and utilization management guidelines. For example, the utilization management guidelines included a standard 45-day length of stay for stabilization in residential treatment, followed by outpatient rehabilitation, and a standard 90-day outpatient treatment with intensity guidelines for the number of sessions. 1 Beattie, Hu, Li, & Bond, 2005; Beattie, McDaniel, & Bond, 2006 Office of the Auditor General page 35 Utilization review—A process to evaluate and approve or deny healthcare services, procedures, or settings based on medical necessity, appropriateness, effectiveness, and efficiency. Source: AHCCCS Medical Policy Manual Chapter 1000—Medical Management/Utilization Management. The Division’s contractual requirements for utilization reviews cover nine dimensions identified in the AHCCCS utilization management policy (see textbox), and RBHAs include all nine dimensions in their utilization management plans. As outlined in these plans, RBHAs review a wide range of issues, including data on readmissions, average length of stay, and pharmacy utilization, such as monitoring for multiple medicines taken to treat the same illness. By monitoring these measures of care, the RBHAs have information that would allow them to help control costs and identify potential issues with quality of care as illustrated in the following example: In August 2008, one RBHA, Community Partnership of Southern Arizona (CPSA), conducted a focus review of a psychiatric health facility. In that 10- case file review, CPSA found that there was little evidence to support that the facility provided active treatment to the consumers, including three consumers who required substance abuse treatment. As a result, CPSA made recommendations regarding treatment planning documentation and group therapy improvement. A CPSA official reported that CPSA will follow up with the facility in August 2009 to ensure that it implemented these recommendations. State of Arizona page 36 AHCCCS medical/utilization management policy requirements • Utilization Data Analysis and Data Management—Collect, monitor, analyze, evaluate, and report utilization data to detect both underutilization and overutilization of services. • Concurrent Review—Review the medical necessity for a planned institutional admission or ongoing institutional care. • Prior Authorization—Make medical decisions regarding the initial authorization of services or requests for continuation of services in specified time frames. • Retrospective Review—Provide policies, procedures on how to conduct reviews to determine medical necessity post delivery of services. • Adoption and Dissemination of Practice Guidelines—Adopt and disseminate practice guidelines that are based on valid and reliable clinical evidence or a consensus of healthcare professionals and include a thorough review of medical journals’ peer-reviewed articles published in the United States when national practice guidelines are not available. • New Medical Technologies and New Uses of Existing Technologies—Develop and implement written policies and procedures for evaluating new technologies and new uses of existing technologies on both an individual basis and systemic basis. • Case Management/Care Coordination—Ensure that a member’s needs are appropriately met through a coordination of cost-effective care involving early identification of health risk factors or special care needs whether in an acute, home, chronic, or alternative setting. • Disease/Chronic Care Management—Implement a program that focuses on members with high risk and/or chronic conditions to increase member self-management and improve providers’ practice patterns and thereby improve members’ healthcare outcome. • Drug Utilization Review—Review the prescribing, dispensing, and use of medications to ensure that the medications are clinically appropriate, safe, and cost-effective drug therapy that improves a member’s health status and quality of care. Source: AHCCCS Medical Policy Manual Chapter 1000—Medical Management/Utilization Management. Expanding existing reviews can help manage costs—Although current utilization reviews contain many elements that could potentially help manage costs, auditors identified several ways to improve their ability to do so. These include the following: Conducting reviews of high- and low-cost substance abuse treatment cases—Auditors’ analysis of division data identified 14 consumers who received substance abuse treatment valued at a total of over $100,000 each during fiscal years 2006 through 2008. The Division was not aware of these cases until auditors brought them to officials’ attention. In one of these cases, presented in Finding 1 as Rodney (see Finding 1, pages 9 through 29), the consumer repeatedly used medical detoxification services valued at a total of over $82,000 in fiscal years 2006 through 2008. Rodney’s stay at a Tucson detoxification center in September 2008 was typical of his pattern. He was brought to a hospital emergency room one evening by emergency medical services. At triage, he had a blood alcohol level of .396. The center provided Ativan, a medication for withdrawal symptoms; fed him; and applied a nicotine patch because the hospital is a nonsmoking campus. A registered nurse monitored him periodically while he slept. Rodney transferred to the detoxification center the following day. When the center conducted an assessment to determine his readiness for treatment to address his addiction, he said, “I’m not sure what I want to do after detox. I’ve never been to treatment, so I can’t say if I would like to go there.” Rodney discharged himself around noon on the third day, against medical advice. Auditors’ analysis of division data found 54 separate division data records for detoxification services for Rodney over the 3-year period analyzed, and a case manager at the center reported that Rodney used detoxification three or four times a month. Cases such as Rodney’s involve high costs, but appear to produce limited outcomes. Altogether, for fiscal years 2006 through 2008, approximately $100 million, or 72 percent of the total encounter value (reported value of services) for substance abuse treatment, was used to provide services to 20 percent of the consumer population, and 45 percent of these consumers did not complete treatment. Reviewing high-cost cases such as Rodney’s could enable the Division to work with the RBHAs and providers to find ways to treat consumers more effectively and break the cycle of substance abuse and detoxification. During the audit, in March 2009, the Division began reviews that focused on service costs for consumers with serious mental illness. Specifically, the Division reviewed Magellan’s fiscal year 2008 healthcare costs for consumers with serious mental illness and identified the top ten service users as Office of the Auditor General page 37 measured by service costs and 49 other clients who were enrolled in fiscal year 2008 but did not receive services that year, most of whom also had not received services in fiscal year 2009. Low-dollar service users could indicate potential quality of care issues or an inadequate disenrollment process. The Division presented the information to Magellan officials, who replied that the top ten service users were very complex cases involving safety concerns. Magellan added that it determined that all services provided to nine of the ten consumers were clinically appropriate and, as a result, would conduct an in-depth review of the clinical appropriateness of the services offered to the one other consumer. Regarding consumers who underutilized services, Magellan reported that they found an opportunity to improve timeliness of closures and follow up after crisis episodes. As a result, Magellan reported that it would implement a claims inactivity report to identify consumers who did not have encounters (service records) for periods of more than 210 days. However, this review was limited to one RBHA and consumers with serious mental illness. To more effectively control costs, the Division needs to continue and expand these efforts. The Division should perform similar reviews for consumers receiving treatment for substance abuse in order to identify, research, and appropriately address all high- and low-dollar service users. In addition, the Division should follow the example of the California managed-care county discussed on page 35 and establish benchmarks for length of stay, and collect and monitor data on length of stay in order to identify consumers who may be over- or under-utilizing certain types of services, which could indicate they need a different service to be treated cost-effectively. Conducting cost-focused reviews of specific treatments or services—In addition to identifying and addressing high- and low-dollar service users, the Division should also focus reviews on managing high-cost treatment or service types. Conducting such reviews would enable the Division to determine not only if the services are necessary, but help identify other effective, potentially less-costly treatments. In addition to reviewing the highest-cost consumers, auditors identified other consumers in various service categories such as transportation, inpatient, and outpatient treatment. Auditors specifically looked for cases with costs significantly higher than the median for that service category. Using division data, auditors identified consumers whose fiscal years 2006 through 2008 treatment values ranged from $50,000 to $100,000, and found that the costs of some of the specific services these consumers received were 15 times more than the median costs of such services. For example: • Ron, 52, has an opiate dependence that requires supervised methadone treatment. According to his treatment center’s clinical director, although Ron lives in Lake Havasu City, his treatment provider transports him twice a week over 180 miles round-trip to Bullhead City, which the clinical State of Arizona page 38 director added is the only authorized methadone treatment center in rural Mohave County. His treatment center’s clinical director also explained that because of federal methadone regulations and licensing requirements, it is very difficult to incentivize other healthcare facilities that may be closer to the consumer to become methadone treatment facilities.1 Therefore, his only option is to transport the client to the Bullhead City Community Medical Services II center at a value over fiscal years 2006 through 2008 of almost $48,000, which represents more than 87 percent of the total value of Ron’s treatment for that time period. This amount greatly exceeds the RBHA’s median transportation value of approximately $88. Examining costs such as this can help the Division identify high-cost practices and examine alternatives for reducing them. Methadone treatment is one area in which the Division has already begun to study alternatives. From fiscal years 2006 to 2008, approximately 2,600 consumers, or about 5 percent of all substance abuse consumers served, received methadone treatment valued at more than $9.5 million. According to a division official, the Division is transitioning to a new treatment, buprenorphine, which the consumer can take home and administer daily. In addition, in an effort to reduce transportation costs for consumers receiving methadone treatment, in February 2009, the Northern Arizona Regional Behavioral Health Authority (NARBHA) approved the purchase of a van to provide such services instead of paying a third-party vendor to do so. To help contain costs, such as transportation costs associated with methadone treatment, the Division and RBHAs should continue their efforts to transition to alternative treatments. In addition, all RBHAs and the Division should identify consumers with higher-than-usual costs for specific services, and alternative methods or treatments that would provide the same quality of care at a reduced cost. Comparing variation in treatment types among RBHAs—Finally, to better monitor ser
Object Description
TITLE | Performance audit, department of health services division of behavioral health services substance abuse treatment programs: report to the Arizona Legislature |
CREATOR | Office of the Auditor General |
SUBJECT | Arizona Department of Health Services--Division of Behavioral Health Services--Auditing |
Browse Topic |
Government and politics |
DESCRIPTION | This title contains one or more publications |
Language | English |
Contributor | Arizona Department of Health Services |
Publisher | Office of the Auditor General |
TYPE |
Text |
Material Collection | State Documents |
Acquisition Note | Report No. 09-07 |
Source Identifier | LG 6.2:R 36 |
Location | o430059294 |
REPOSITORY | Arizona State Library, Archives and Public Records--Law and Research Library |
Description
TITLE | Performance audit, department of health services division of behavioral health services substance abuse treatment programs: report to the Arizona Legislature |
DESCRIPTION | 92 pages (PDF version). File size: 1500 KB |
TYPE |
Text |
RIGHTS MANAGEMENT | Copyright to this resource is held by the creating agency and is provided here for educational purposes only. It may not be downloaded, reproduced or distributed in any format without written permission of the creating agency. Any attempt to circumvent the access controls placed on this file is a violation of United States and international copyright laws, and is subject to criminal prosecution. |
DATE ORIGINAL | 2009-07 |
Time Period |
2000s (2000-2009) |
ORIGINAL FORMAT | Born Digital |
Source Identifier | LG 6.2:R 36 |
Location | o430059294 |
DIGITAL IDENTIFIER | 09-07.pdf |
DIGITAL FORMAT | PDF (Portable Document Format) |
REPOSITORY | Arizona State Library, Archives and Public Records--Law and Research Library. |
File Size | 1535955 Bytes |
Full Text | Debra K. Davenport Auditor General Performance Audit Department of Health Services Division of Behavioral Health Services— Substance Abuse Treatment Programs Performance Audit Division July • 2009 REPORT NO. 09-07 A REPORT TO THE ARIZONA LEGISLATURE The is appointed by the Joint Legislative Audit Committee, a bipartisan committee composed of five senators and five representatives. Her mission is to provide independent and impartial information and specific recommendations to improve the operations of state and local government entities. To this end, she provides financial audits and accounting services to the State and political subdivisions, investigates possible misuse of public monies, and conducts performance audits of school districts, state agencies, and the programs they administer. The Joint Legislative Audit Committee Audit Staff Copies of the Auditor General’s reports are free. You may request them by contacting us at: Office of the Auditor General 2910 N. 44th Street, Suite 410 • Phoenix, AZ 85018 • (602) 553-0333 Additionally, many of our reports can be found in electronic format at: www.azauditor.gov Melanie M. Chesney, Director Shan Hays, Manager and Contact Person Anne Hunter, Team Leader Kerry Howell Karl Kulick Senator Thayer Verschoor, Chair Representative Judy Burges, Vice-Chair Senator Pamela Gorman Representative Tom Boone Senator John Huppenthal Representative Cloves Campbell, Jr. Senator Richard Miranda Representative Rich Crandall Senator Rebecca Rios Representative Kyrsten Sinema Senator Bob Burns (ex-officio) Representative Kirk Adams (ex-officio) 2910 NORTH 44th STREET • SUITE 410 • PHOENIX, ARIZONA 85018 • (602) 553-0333 • FAX (602) 553-0051 WILLIAM THOMSON DEPUTY AUDITOR GENERAL DEBRA K. DAVENPORT, CPA AUDITOR GENERAL STATE OF ARIZONA OFFICE OF THE AUDITOR GENERAL July 30, 2009 Members of the Arizona Legislature The Honorable Janice K. Brewer, Governor Will Humble, Interim Director Department of Health Services Transmitted herewith is a report of the Auditor General, a Performance Audit of the Department of Health Services, Division of Behavioral Health Services—Substance Abuse Treatment Programs. This report is in response to an October 5, 2006, resolution of the Joint Legislative Audit Committee. The performance audit was conducted as part of the sunset review process prescribed in Arizona Revised Statutes §41-2951 et seq. I am also transmitting with this report a copy of the Report Highlights for this audit to provide a quick summary for your convenience. As outlined in its response, the Department of Health Services agrees with all of the findings and plans to implement all of the recommendations. My staff and I will be pleased to discuss or clarify items in the report. This report will be released to the public on July 31, 2009. Sincerely, Debbie Davenport Auditor General Attachment The Office of the Auditor General has conducted a performance audit of the substance abuse treatment programs provided by the Department of Health Services (Department), Division of Behavioral Health Services (Division), pursuant to an October 5, 2006, resolution of the Joint Legislative Audit Committee. This is the second audit in a series of three reports on the Department and was conducted as part of the sunset review process prescribed in Arizona Revised Statutes (A.R.S.) §41-2951 et seq. This audit focuses on substance abuse treatment outcomes and system oversight. The first audit focused on the Division of Licensing Services, and the final report will be an analysis of the 12 statutory sunset factors. According to its Annual Report on Substance Abuse Treatment Programs, the Division spent more than $121 million for substance abuse services in fiscal year 2008. Program participants, whom the Division refers to as “consumers,” numbered more than 63,000 adults and children. Most were enrolled in the Arizona Health Care Cost Containment System, or AHCCCS, the State’s Medicaid program. They received alcohol- and drug-related services ranging from counseling and skills training to crisis intervention and detoxification in hospitals or other inpatient facilities. The Division provides these services through contracts with four regional behavioral health authorities, or RBHAs, and three tribal regional behavioral health authorities, or TRBHAs, which contract with a network of more than 100 substance abuse treatment service providers throughout the State. This performance audit focused on the program’s outcomes—that is, the extent to which services reduced dependency on alcohol and drugs—and on the Division’s oversight of behavioral health authorities and providers. Division should focus on strategies that improve outcomes (see pages 9 through 29) Although substance abuse is difficult to treat, the Division can take steps to improve outcomes for individuals who participate in substance abuse treatment. Auditors analyzed 3 years of data related to four measures commonly used to evaluate program effectiveness—extent of continuing alcohol or drug use, employment, criminal activity, and homelessness. The analysis showed that outcomes related to continued alcohol and drug use were associated with the following two factors: Office of the Auditor General SUMMARY page i Deciding to abstain from using alcohol or drugs before treatment started— More than half of all consumers reported that they were abstinent when their treatment started. Within this group, more than 93 percent reported that they were still abstinent when they left the program. By contrast, most of those who reported using alcohol or drugs when they began treatment were still using these substances at about the same level when they left. About one person in every four who began treatment while still using alcohol or drugs reported diminishing his/her use of alcohol or drugs or stopping it altogether by the time he/she left treatment. Completing treatment—Overall, 58 percent of consumers did not complete their treatment. Providers lost contact with many of them, while others refused treatment or left for other reasons, but rates of continued use of alcohol or drugs varied substantially between those who completed the treatment and those who did not. For example, among consumers who reported using alcohol or drugs when they began treatment, 27 percent of those who completed their treatment reported abstinence when they left. By contrast, among consumers who reported using alcohol or drugs when they began treatment, only 17.6 percent of those consumers who left before completing their treatment reported that their use had diminished. The analysis showed little change across the three remaining performance measures—lack of recent arrests, employment, and stable housing. For example, 21 percent of consumers reported recent arrests upon entering treatment, and 18 percent reported new arrests at the time of their update or disenrollment. Similarly, 38 percent said they were employed when they entered treatment, while 41 percent reported being employed at their annual update or disenrollment. Finally, slightly more than 7 percent were homeless upon entering treatment, and slightly less than 7 percent were homeless at their annual update or disenrollment. Substance abuse is difficult to treat, and auditors’ more detailed case studies of a limited number of consumers showed that the reasons for success or failure are complicated and varied. Nonetheless, research and best practices indicate the best opportunities for increasing success rest in three main strategies: Focusing on treatment retention—Research corroborates what auditors’ analysis showed: consumers who remain longer in treatment experience better outcomes. The Division can take several steps to increase consumer retention, including establishing performance goals, monitoring completion rates, and using incentives and other case management techniques. Other states that auditors reviewed have taken such steps, and the Division may be able to adopt some of these approaches. Ensuring that consumers have access to a full range of services that can potentially be used to address their particular needs—This strategy, called continuum of care, involves incorporating appropriate types of treatment over State of Arizona page ii time and placing the consumer in more or less intensive treatment as needed. Auditors’ case studies showed that while some consumers showed good outcomes and received appropriate services, others did not necessarily receive the services and therapies that might improve the chances of good treatment outcomes. The Division can take several steps to ensure continuum of care, including collecting and monitoring data relevant to assessment, better defining case management, and working with RBHAs to improve the continuum of care when weaknesses are identified. Following practices that have been shown to carry the greatest chance of success—These evidence-based practices have been validated by observation or experience as improving treatment success. Examples include motivational interviews, which is a counseling style designed to help consumers recognize and accept the need for continued care. Although the Division requires the RBHAs to use evidence-based practices, RBHAs are not necessarily doing so, and the Division is not ensuring compliance. According to a 2008 federal grant review that focused on programs for children and adolescents, the Division had done a good job of establishing evidence-based practices in some areas but had not identified such practices across the continuum of care and could do more to ensure sustainability of the emphasis on evidence-based practices. The Division reached similar conclusions in another study of intensive outpatient programs for youth. Steps the Division can take to place greater emphasis on evidence-based practices include encouraging RBHAs to offer a wider variety of programs, monitoring compliance with its contractual requirements to use evidence-based practices, and expanding its work with the RBHAs to ensure that providers have the guidance needed to implement specific evidence-based practices. Division should improve oversight of substance abuse programs (see pages 31 through 42) The Division should take steps to improve its oversight of the substance abuse programs administered by RBHAs. These steps take two main forms: Increasing the use of information about treatment outcomes—Although the Division collects outcome information to complete certain reports, auditors found that oversight efforts focused almost entirely on process-related information, such as the timeliness of services or coordination with a consumer’s primary care physician as required by the Division’s contract with AHCCCS. As a result, the Division is largely unable to determine if its substance abuse treatment programs are achieving positive results or if its resources are being used effectively. Additionally, because the Division does not compare Office of the Auditor General page iii substance abuse outcome measures across RBHAs or providers, it cannot assess which providers’ treatment services are resulting in improved client outcomes or identify underperforming providers. Needed actions include continuing its efforts to streamline uniform outcome data collection, establishing relevant performance goals in contracts with the RBHAs, and encouraging the RBHAs to consider ways to reward providers who meet standards and penalize those who do not. Expanding utilization reviews to focus more on service costs, consumer assessments, and case management—Although its oversight efforts contain many elements that could potentially help manage costs, the Division could implement several actions that could improve its ability to do so. Greater emphasis on cost appears warranted. Auditors’ review of division data from fiscal years 2006 to 2008, for example, identified 14 substance abuse consumers with service costs over $100,000. One incurred $82,000 in medical detoxification costs, during which time he continued to drink and require detoxification three or four times a month. The Division was not aware of these cases until auditors brought them to officials’ attention. Actions needed include (1) regularly reviewing high- and low-cost substance abuse treatment cases, (2) collecting data to identify consumers who may be overutilizing or underutilizing certain types of services, which could indicate a lack of alternative forms of treatment or a need for other changes to improve treatment, (3) comparing variations in the use of types of treatment at each RBHA to see if the use of such services positively affects consumer treatment outcomes and adjusting treatment accordingly, and (4) determining how to best use assessment and case management to contain costs while maintaining quality of care. Further, to improve oversight, the Division should continue its efforts to fill vacant positions in its data systems and analysis and quality management functions, and should perform follow-up work to ensure that the restructuring it initiated in April 2009 has provided management with the information to do so. State of Arizona page iv Office of the Auditor General TABLE OF CONTENTS continued page v Introduction & Background 1 Finding 1: Division should focus on strategies that improve outcomes 9 Substance abuse difficult to treat 9 Most consumers showed little change after treatment 10 Division should increase focus on treatment retention and completion 17 Division should monitor continuum of care 21 Using appropriate evidence-based practices can improve success rate 25 Recommendations 28 Finding 2: Division should improve oversight of substance abuse programs 31 Division should monitor outcomes 31 Division should monitor monies spent on treatment 35 Recommendations 41 Appendix A: Outcome analysis methodology and results a-i Appendix B: Methodology b-i Appendix C: Bibliography c-i Agency Response TABLE OF CONTENTS continued page vi State of Arizona Table: 1 Funding Sources for Substance Abuse Services Expenditures Fiscal Year 2008 (Unaudited) 4 Figures: 1 Regional and Tribal Behavioral Health Authorities’ Geographical Service Areas (GSA) and Fiscal Year 2008 Substance Abuse Enrollment 3 2 State-wide Substance Abuse Treatment Enrollment Fiscal Years 2001 through 2008 6 3 Treatment Completion and Noncompletion Rates by RBHA (GSA) Fiscal Years 2006 through 2008 19 4 Consumer-Reported Abstinence from Substance Use By RBHA (GSA) from Admission to Annual Review or Discharge Fiscal Years 2006 through 2008 a-iv 5 Change in Substance Use Among Consumers Who Reported Substance Use Before Treatment by RBHA (GSA) From Admission to Annual Review or Discharge Fiscal Years 2006 through 2008 a-v 6 Alcohol Abstinence Status of Consumers by State (Western Region) Fiscal Year 2009 SAPT Block Grant Application (Unaudited) a-vi 7 Drug Abstinence Status of Consumers by State (Western Region) Fiscal Year 2009 SAPT Block Grant Application (Unaudited) a-vii Office of the Auditor General TABLE OF CONTENTS continued page vii Figures (continued): 8 Alcohol Abstinence Status of Consumers Arizona, Western Region, and National Averages Fiscal Year 2009 SAPT Block Grant Application (Unaudited) a-viii 9 Drug Abstinence Status of Consumers Arizona, Western Region, and National Averages Fiscal Years 2009 SAPT Block Grant Application (Unaudited) a-ix 10 Employment Status of Consumers by RBHA (GSA) From Admission to Annual Review or Discharge Fiscal Years 2006 through 2008 a-xi 11 Employment Status of Consumers by State (Western Region) Fiscal Year 2009 SAPT Block Grant Application (Unaudited) a-xii 12 Employment Status of Consumers Arizona, Western Region, and National Averages Fiscal Year 2009 SAPT Block Grant Application (Unaudited) a-xiii 13 Arrest-Free Status of Consumers by RBHA (GSA) From Admission to Annual Review or Discharge Fiscal Years 2006 through 2008 a-xv 14 Arrest-Free Status of Consumers by State (Western Region) Fiscal Year 2009 SAPT Block Grant Application (Unaudited) a-xvi 15 Arrest-Free Status of Consumers Arizona, Western Region, and National Averages Fiscal Year 2009 SAPT Block Grant Application (Unaudited) a-xvii State of Arizona TABLE OF CONTENTS concluded page viii Figures (concluded): 16 Homeless Status of Consumers by RBHA (GSA) From Admission to Annual Review or Discharge Fiscal Years 2006 through 2008 a-xix 17 Homeless Status of Consumers by State (Western Region) Fiscal Year 2009 SAPT Block Grant Application (Unaudited) a-xx 18 Homeless Status of Consumers Arizona, Western Region, and National Averages Fiscal Year 2009 SAPT Block Grant Application (Unaudited) a-xxi The Office of the Auditor General has conducted a performance audit of the substance abuse treatment programs provided by the Department of Health Services (Department), Division of Behavioral Health Services (Division), pursuant to an October 5, 2006, resolution of the Joint Legislative Audit Committee. This is the second audit in a series of three reports on the Department and was conducted as part of the sunset review process prescribed in Arizona Revised Statutes (A.R.S.) §41-2951 et seq. This audit focuses on substance abuse treatment outcomes and system oversight. The first audit focused on the Division of Licensing Services, and the final report will be an analysis of the 12 statutory sunset factors. Importance of treating substance abuse Drug and alcohol abuse are associated with some of society’s most serious and expensive problems. For example, according to literature, nation-wide: More than half of all state prison inmates were under the influence of alcohol or drugs when they were arrested.1 Nearly one in six state inmates committed crimes to support a drug addiction.2 About 20 percent of acute care Medicaid expenditures pay for alcohol- or drug-related medical costs.3 Drunk driving is a major expense for the police, courts, and emergency medical systems.4 1 U.S. Department of Justice, Bureau of Justice Statistics, 2005a as cited in Rosenbloom et al., 2006 2 U.S. Department of Justice, Bureau of Justice Statistics, 2005b as cited in Rosenbloom et al., 2006 3 U.S. Department of Health and Human Services, Substance Abuse and Mental Health Services Administration, 2005 as cited in Rosenbloom et al., 2006 4 Miller, Cox, Zaloshnja, & Taylor, 2002 Office of the Auditor General INTRODUCTION & BACKGROUND page 1 In 2003, an estimated 22.5 million people ages 12 and older nation-wide, or about 9.4 percent of the population, had a substance abuse disorder, and an estimated $21 billion was devoted to substance abuse treatment, according to a U.S. Department of Health and Human Services’ Substance Abuse and Mental Health Services Administration (SAMHSA) report dated 2007.1 According to the same report, people with substance use disorders rely on public sources of financing far more than do people with other diseases. The report also states that public sources of funding, including state and local government, Medicaid, and other federal spending, such as block grants, provided 77 percent of total substance abuse spending in 2003 but only 45 percent of all healthcare spending. The nature of addiction, such as its quality as a chronic and recurring problem, and other factors, such as age and socio-economic status, affect treatment success. Arizona, like many other states, struggles with the problem of substance abuse (see textbox). Structure and funding of Arizona’s system for providing substance abuse services The Division provides substance abuse services mainly to consumers enrolled in the Arizona Health Care Cost Containment System, or AHCCCS, which is the State’s Medicaid program. However, others can receive services to the extent funding is available, particularly if they are members of priority populations such as pregnant women. AHCCCS enrollees include adults who meet the requirements of Title XIX (Medicaid) of the Social Security Act (Act), as well as children and their families who meet the requirements of Title XXI (the federal SCHIP program, which is called KidsCare in Arizona) of the Act. Publicly funded substance abuse treatment and prevention services in Arizona are provided primarily through the behavioral health system administered by the Division.2 The Division provides these services through contracts with four regional behavioral health authorities, or RBHAs, and three tribal regional behavioral health 1 Mark et al., 2007 2 Incarcerated individuals may receive publicly funded substance abuse services through the Department of Corrections or the county jails. State of Arizona page 2 Prevalence and impacts of substance abuse in Arizona Altogether, an estimated 466,000 Arizonans ages 12 and older were dependent on or abused alcohol, and an estimated 137,000 were dependent on or abused illicit drugs, according to the 2005 National Survey on Drug Use and Health. Drug- and alcohol-related deaths, including those attributable to motor vehicle and boating fatalities, accounted for nearly 1,700 deaths in Arizona in 2005. There were significant increases in the number of alcohol and drug-induced deaths in Arizona between 2000 and 2005. The rates of drug-induced deaths more than doubled between 2000 and 2005, and the number of alcohol-induced deaths increased by more than one-third for the same time period. A 2008 study of adults arrested in Maricopa County in 2007 found that over one-third of arrestees interviewed for the study tested positive for methamphetamine use, with over 40 percent of female arrestees testing positive. Source: The Substance Abuse Epidemiology Work Group’s 2007 Arizona Statewide Substance Abuse Epidemiology Profile, and Arizona State University, Center for Violence Prevention and Community Safety’s Arizona Arrestee Reporting Information Network Annual Adult Report 2007. authorities, or TRBHAs, which contract with a network of more than 100 substance abuse treatment service providers within six geographical service areas, or GSAs, throughout the State (see Figure 1).1 During fiscal year 2008, over 63,000 adults and children participated in substance abuse treatment programs offered through the RBHAs and TRBHAs and their providers throughout the State. Magellan, the RBHA that serves Maricopa County, served the largest proportion—38 percent—of individuals receiving substance abuse treatment in fiscal year 2008, followed by Community Partnership of Southern Arizona (CPSA) at 32 percent, Northern Arizona Behavioral Health Authority (NARBHA) at 15 percent, and Cenpatico Behavioral Health of Arizona, the three tribal regional behavioral health authorities, and the Navajo Nation serving the remaining approximately 15 percent. The Navajo Nation serves approximately 2 percent of individuals receiving substance abuse treatment. According to the Division’s Annual Report on Substance Abuse Treatment Programs, in fiscal year 2008 approximately $121.2 million, or approximately 11 percent of the Division’s estimated $1.1 billion budget, was expended for substance abuse services.2 The majority of this funding came from Title XIX/XXI monies, with additional funds provided through the federal Substance Abuse Prevention and Treatment (SAPT) block grant, state appropriations, and other funding sources (see Table 1, page 4). Budget reductions in fiscal year 2009 will probably affect substance abuse treatment. In addition, as of July 16, 2009, the Department’s 2010 budget had not been determined. According to the Division, a total of $2.5 million of appropriated funds that are used for substance abuse services was cut from its fiscal year 2009 budget. Office of the Auditor General page 3 1 Magellan replaced ValueOptions as the RBHA for Maricopa County on September 1, 2007. Source: Auditor General staff analysis of the Arizona Department of Health Services, Division of Behavioral Health Services’ Annual Report on Substance Abuse Treatment Programs for fiscal year 2008 and division-provided enrollment data. Figure 1: Regional and Tribal Behavioral Health Authorities’ Geographical Service Areas (GSA) and Fiscal Year 2008 Substance Abuse Enrollment Northern Arizona Regional Behavioral Health Authority Cenpatico Behavioral Health of Arizona Magellan1 Cenpatico Behavioral Health of Arizona Community Partnership of Southern Arizona Community Partnership of Southern Arizona Tribal Regional Behavioral Health Authorities 1 In addition to its three TRBHA contracts, the Division also contracts with two tribal nations—the Navajo Nation and the Colorado River Indian Tribes—to deliver substance abuse treatment services to reservation residents. 2 The total amount of funding allocated to the RBHAs in fiscal year 2008 was $989.4 million, including monies for serving children, adults with serious mental illness, and people with co-occurring mental health and substance abuse disorders, as well as providing general mental health and substance abuse prevention and treatment services. In addition, the Division reported fund sweeps and appropriations reductions totaling almost $1.1 million in substance abuse fund monies. These monies come mainly from fees and fines collected from criminal offenses and are used to provide alcohol and drug screening, education, and treatment for individuals who are court-ordered to attend, but do not have sufficient ability to pay. Because of these reductions, the Division expects that the number of consumers served in fiscal year 2009 will be less than the number served in fiscal year 2008. In addition, to offset some of the budget reductions, the Division reported that it is updating its policy to require co-payments for consumers who are not eligible for Title XIX and whose income is a certain percentage above the federal poverty level. On April 3, 2009, a draft version of the policy regarding co-payments was released for public comments, and as of July 16, 2009, the policy had not been finalized.1 Funding is allocated to the RBHAs through a capitated payment arrangement whereby the RBHA is provided a set amount of funding each month for each consumer in its geographic service area who is enrolled in AHCCCS. In order to allow the Division to monitor services and determine funding needs, the RBHAs submit service data called encounters (see textbox) to the Division. The Division’s contracts with the RBHAs require them to submit encounters whose total value equals at least 85 percent of the total service revenue payments they receive on an annual basis. Substance abuse services provided The Division provides many different types of substance abuse services. In fiscal years 2006 through 2008, the RBHAs reported that they provided services valued at $139.3 million to substance abuse treatment consumers, excluding substance abuse services to other consumers such as adults with serious mental illness, children, and adolescents. The largest service category for those consumers, 1 Specific changes to the policy include: 1) consumers with serious mental illness are no longer excluded from the co-payment requirement, 2) the types of services excluded from the co-payment requirement were modified, and 3) providers are allowed to refuse service as well as terminate services to a consumer for non-payment of a co-payment. State of Arizona page 4 Encounter—Record of a service delivered to a consumer by a provider. Includes specifics about the type and date of service, provider that delivered the service, and dollar value of the service. Funding Source Amount Percentage Title XIX/XXI funding $ 75,941,357 62.66% Substance Abuse Prevention and Treatment Block Grant 23,545,206 19.43 State appropriations 16,518,610 13.63 Intergovernmental agreements (IGA) and interagency service agreement (ISA)1 5,141,081 4.24 Liquor fees2 45,325 0.04 Total $121,191,579 100.00% Table 1: Funding Sources for Substance Abuse Services Expenditures Fiscal Year 2008 (Unaudited) 1 In fiscal year 2008 the Division had IGAs with Maricopa County and the City of Phoenix, and an ISA with the Arizona Department of Corrections, Correctional Officer/Offender Liaison program (the COOL program). Maricopa County and the City of Phoenix provide monies to help operate the Local Alcohol Rehabilitation Center, a detoxification center in Phoenix. The COOL program was eliminated by the Department of Corrections in 2009. However, in 2008 the program provided transitional services for persons with substance abuse issues who were leaving the penal system. The services included assistance in finding housing, employment, and behavioral health services. 2 Liquor fees are fees collected from the issuance of special event liquor licenses. Source: Auditor General staff analysis of the Arizona Department of Health Services, Division of Behavioral Health Services’ Annual Report on Substance Abuse Treatment Programs for fiscal year 2008. treatment services, accounted for approximately 31.5 percent of the total.1 The types of services were as follows: Treatment services ($43.9 million)—Individual and group counseling, therapy, assessment, evaluation, screening, and other professional services. Residential services ($26.9 million)—Twenty-four-hour residential services, including structured treatment, which includes room and board, delivered in residential facilities or supported independent living settings. Support services ($25.3 million)—Case management, peer support services, and transportation. Medical and pharmacy services ($19.0 million)—Medication to assist with alcohol or drug withdrawal or with co-occurring disorders. Inpatient ($9.5 million)—Inpatient detoxification and treatment services delivered in hospitals and other inpatient facilities, including residential treatment centers that provide a structured treatment with 24-hour supervision, an intensive treatment program, and on-site medical services. Crisis intervention services ($5.7 million)—Crisis intervention/stabilization services provided in the community, hospitals, and residential treatment centers. Rehabilitation services ($4.9 million)—Education, coaching, training, and other services, including securing and maintaining employment. Services include living skills training, cognitive rehabilitation, health promotion, and ongoing support to help maintain employment. Day programs ($4.1 million)—Skills training and development, behavioral health prevention/promotion, medication training and support, ongoing support to maintain employment, and self-help/peer services to improve consumers’ ability to function in the community. The cost of services varies by type of treatment. For example, auditors’ analysis of encounter data from fiscal years 2006 through 2008 revealed that the average cost per person, per year, for residential treatment was $1,500 and the average cost of crisis intervention per person, per year, was $240. Because some consumers may receive several different types of treatment or receive treatment for longer periods or at several different times during the year, the total cost per person for all services received varies widely, with a few consumers receiving services valued at more than Office of the Auditor General page 5 1 The figures reported in these bullets reflect auditors’ analysis of services reported for consumers whose primary reason for enrollment was substance abuse. These figures differ from the total substance abuse funding of $121 million shown in Table 1 (see page 4) because these figures cover a 3-year period and do not include substance abuse services provided to other consumers. Further, because the Division uses a capitated system to pay RBHAs in advance for providing all required services, current-year costs are not reflected in current-year payments as they would be in a fee-for- service system. Therefore, the total substance abuse funding is not the same amount as the value of the services provided as reported by the RBHAs for a given time period. $100,000 per year (See Finding 2, pages 31 through 42, for additional information on high-utilization consumers). The median cost of treatment per person for fiscal years 2006 through 2008 was $1,090.1 Consumers served Generally, to obtain substance abuse services through the RBHAs or TRBHAs a person must be enrolled in AHCCCS. According to the Division’s Annual Report on Substance Abuse Treatment Programs, in fiscal year 2008, 74 percent of adults and children receiving substance abuse services, or approximately 47,000 individuals, were AHCCCS eligible. Substance abuse services for non-Title XIX/XXI-eligible individuals and families, also delivered by RBHA- and TRBHA-contracted providers, are provided only if monies are available, and these services are provided first to people in priority populations. The priority populations are spelled out by grant requirements and other funding sources. For example, the federal SAPT block grant designates priority populations as pregnant women, women with dependent children, and intravenous drug users. In fiscal year 2008, 1,139 pregnant women and 6,238 women with children received substance abuse services. In addition, in fiscal year 2008, 4,033 people were tested for human immunodeficiency virus (HIV), with 24 identified as being HIV positive and receiving counseling. From fiscal year 2001 to fiscal year 2008, the number of people receiving some type of state-provided substance abuse treatment increased by nearly 300 percent, from 16,115 to 63,571 (see Figure 2). The Division attributes this growth primarily to expansion in AHCCCS eligibility requirements. For example, in fiscal year 2000, Arizona voters approved Proposition 204, expanding eligibility for AHCCCS starting in 2001. The proposition expanded income eligibility requirements up to 100 percent of the federal poverty level. In fiscal year 2008 nearly all consumers receiving substance abuse treatment services through the RBHAs and TRBHAs and their providers were adults, most were non-Hispanic, and alcohol was the most commonly used substance (see textbox, page 7). That year, according to its Annual State of Arizona page 6 16,115 32,772 41,172 36,375 59,661 60,105 65,696 63,571 0 10,000 20,000 30,000 40,000 50,000 60,000 70,000 2001 2002 2003 2004 2005 2006 2007 2008 Enrollment Fiscal Year Figure 2: State-wide Substance Abuse Treatment Enrollment Fiscal Years 2001 through 2008 Source: Auditor General staff analysis of the Arizona Department of Health Services, Division of Behavioral Health Services’ Annual Report on Substance Abuse Treatment Programs for fiscal years 2007 and 2008. 1 Median cost per person was calculated using demographic and encounter data from the Division’s Client Information System for fiscal years 2006 through 2008 for adult substance abuse consumers without a serious mental illness. Report on Substance Abuse Treatment Programs, the Division provided substance abuse treatment services to 49,751 adults without a serious mental illness (78 percent of substance abuse treatment consumers), 10,071 adults who had serious mental illness with a co-occurring substance abuse disorder (16 percent), and 3,929 children or adolescents (6 percent). Consumer characteristics are important in planning treatment and can affect treatment success. For example, women with children need specialized help or services, such as childcare. Consumers entered treatment through a variety of referral sources, with nearly half seeking treatment on their own or on the advice of a friend or family member. Referral by court order or involvement with a criminal justice agency varies regionally. In rural areas, court or criminal justice referrals are as high as 40 percent of the treatment-seeking population. For example, in fiscal year 2007, in the primarily rural counties of Yuma and La Paz, 41 percent of people enrolled in substance abuse treatment were referred by a court or a probation/parole officer. Office of the Auditor General page 7 Consumer Characteristics Fiscal Year 2008 Age: •94% adults; 6% adolescents •50% of adults were aged 25 to 44 Gender: •56% male; 44% female Diagnosis: •78% were adults with a substance abuse disorder •16% had serious mental illness with a co-occurring substance abuse disorder Race: •86% Caucasian •7% African American •5% Native American •2% other or multi-race Ethnicity: •26% Hispanic/Latino •74% non-Hispanic/Latino Primary •37% alcohol substance: •26% stimulants including methamphetamine and cocaine/crack •24% marijuana •11% narcotics including heroin •2% other Referral •45% self, friend, or family source:1 •23% court order or criminal justice agency •11% other or unknown •10% other behavioral health providers •6% other state agencies •3% community agencies •3% Health Plan or primary care physician 1 Numbers do not total 100 percent because of rounding. Source: Auditor General staff analysis of the Arizona Department of Health Services, Division of Behavioral Health Services’ Annual Report on Substance Abuse Treatment Programs for fiscal year 2008. Substance abuse program oversight The Division is primarily an oversight agency that monitors all aspects of behavioral health services, including substance abuse treatment services, through contract compliance and other mechanisms. For example, the Division’s contract compliance unit conducts administrative reviews, as required by the Division’s contract with AHCCCS, that examine the RBHAs’ compliance with state and federal requirements, program operations, fiscal operations, and financial status. Although these reviews encompass overall behavioral health, including substance abuse, they do not focus on substance abuse. Additionally, various units within the Division are responsible for overseeing different aspects of the substance abuse program, as well as the general mental health program and co-occurring disorders. For example, the grants, clinical practice improvement, interagency coordination units, and office of the medical director all have responsibilities that involve some oversight of general mental health treatment, substance abuse treatment, and treatment for co-occurring disorders. The Division has one full-time equivalent position, Lead Substance Abuse Clinical Advisor, that is dedicated solely to the substance abuse program, and as of April 15, 2009, that position had been vacant for 6 months because of the state hiring freeze. The responsibilities of the Lead Substance Abuse Clinical Advisor include clinical practice protocols and oversight of clinical practice improvement for substance abuse and co-occurring treatment. According to the Division, other staff, such as another clinical advisor and the Interim Director of Clinical Operations, have taken over some of the responsibilities of the Lead Substance Abuse Clinical Advisor position. Scope and objectives This performance audit focused on steps the Division can take to improve substance abuse treatment outcomes and its oversight of the substance abuse program. The audit’s analysis and case studies focused on adult substance abuse consumers and excluded any consumers who had a serious mental illness in addition to a substance abuse diagnosis and consumers who were younger than 18 years old when they entered treatment. This audit was conducted in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. The Auditor General and staff express appreciation to the Department’s Interim Director and staff for their cooperation and assistance throughout the audit. State of Arizona page 8 Division should focus on strategies that improve outcomes Substance abuse is difficult to treat, but the Department of Health Services, Division of Behavioral Health Services (Division), can employ more effective strategies to improve outcomes. The difficulty of treatment is borne out in auditors’ analysis of the available outcome data, which showed little change in consumer outcomes after treatment. This analysis showed that consumers who entered treatment while already abstaining from alcohol or drugs had the greatest success as measured in terms of continued abstinence, whereas those consumers who entered treatment while using alcohol or drugs showed much lower success rates. Among those consumers who entered treatment while still using alcohol or drugs, those who actually completed their treatment tended to do better than those who dropped out. To improve outcomes, the Division should take steps in three main areas that research has shown to be effective: (1) helping ensure that consumers complete their treatment, (2) ensuring that consumers are placed in a level of treatment that is appropriate based on their assessed needs, and (3) following practices that have been shown to carry the greatest chance of success. Substance abuse difficult to treat Arizona, like many other states, struggles with the problem of treating individuals with substance abuse problems. Substance abuse is increasingly recognized as a chronic, relapsing condition that may require multiple episodes of care over many years. According to the National Institute on Drug Abuse of the National Institutes of Health, long-term drug use results in changes in brain function that can result in the compulsion to use drugs despite adverse consequences.1 Research indicates that, in general, approximately 50 to 60 percent of patients begin using alcohol or drugs again within 6 months of treatment cessation, regardless of the type of discharge, patient characteristics, or the particular substance used.2 Successful treatment of substance abuse relies upon an individual’s ability to change his/her behavior, and ability and motivation to integrate techniques for disease management into his/her 1 National Institute on Drug Abuse, 2009 2 Institute of Medicine of the National Academies, 2006 Office of the Auditor General page 9 FINDING 1 lifestyle. The goal is for the individual to manage symptoms independently using techniques learned in treatment. This is a complex process complicated by a variety of personal, social, and cultural factors such as socio-economic, legal, family, and employment situations. Most consumers showed little change after treatment Auditors’ analysis of commonly used outcome measures for substance abuse consumers found that most participants in the Division’s programs showed little change after entering treatment, although some consumers improved. Auditors analyzed four National Outcome Measures (NOMs) developed by the federal government—substance use, criminal activity, employment, and homelessness (see textbox). Outcomes were tied to two key factors. If consumers were already doing well in a measured area before the start of treatment they generally maintained their status, particularly if they completed treatment. Consumers who were not doing well in a measured area before the start of treatment were more likely to improve if they completed treatment, and stayed the same or got worse if they did not complete treatment. NOMs measure program success—The federal Substance Abuse and Mental Health Services Administration (SAMHSA) has developed a set of National Outcome Measures for use in evaluating treatment effectiveness. It requires states that receive federal Substance Abuse Prevention and Treatment (SAPT) block grant monies to report their progress using these measures. According to experts, for the consumer and other stakeholders, effectiveness of addiction treatment is measured by its ability to reduce addiction-related problems.1 Therefore, in addition to substance use, the NOMs measure other outcomes such as criminal activity, employment, and homelessness. Auditors selected those measures that SAMHSA has fully developed and that could be measured using division demographic data.2 Extent of improvement varied—To determine the Division’s success in treating substance abuse consumers, auditors analyzed data from the Division’s Client Information System (CIS) for substance abuse consumers who did not have a serious mental illness and were enrolled with the Division in fiscal years 2006, 2007, or 2008 (see Appendix A, pages a-i through a-xxi).3 The analyses showed 1 McLellan, McKay, Forman, Cacciola, & Kemp, 2005a 2 In addition to the four measures reported in this audit, SAMHSA has identified six other domains for evaluating program success, some of which are still under development. The other domains are social connectedness, access/capacity, retention, perception of care, cost-effectiveness of treatment, and use of evidence-based practices. 3 Auditors’ analysis excludes any consumers who had a serious mental illness in addition to a substance abuse diagnosis, and also excludes any consumers who were younger than 18 years old when they entered treatment. In addition, for each outcome analyzed, auditors included only the consumers who had valid entries in the outcome data field at both times: (1) entering treatment and (2) annual update or disenrollment. As a result, the total number of consumers analyzed for each outcome varies. “Annual update” refers to an annual reassessment each consumer who remains in treatment should receive. “Disenrollment” means the consumer is removed from enrollment, which occurs when the consumer completes treatment, refuses further treatment, or stops treatment for another reason, such as moving out of Arizona. State of Arizona page 10 National Outcome Measures (NOMs) analyzed by auditors • Fewer consumers reporting recent substance use • Fewer consumers with recent criminal activity • More consumers employed • Fewer consumers who are homeless Source: U.S. Department of Health and Human Services Substance Abuse and Mental Health Services Administration. that more than half of the consumers who entered treatment reported abstaining from alcohol or drugs during the 30 days before starting treatment. Nearly all of these consumers reported that they were still abstaining at the time they left the program. However, of those who were not abstaining when they entered treatment, over 70 percent reported no change in their substance use. Overall, the analyses found little change in the areas of employment, criminal activity, and homelessness. Specifically: Slight overall improvement in substance use—Most consumers who auditors analyzed did not change their use of drugs or alcohol from entering treatment to their annual update or discharge. Approximately half of the consumers were abstinent upon entering treatment and were still abstinent at their update or discharge, while approximately one-third of the consumers who were using drugs or alcohol when they entered treatment were still using drugs or alcohol at their update or discharge (see first textbox on this page). Out of the more than 50,000 consumers auditors analyzed, a total of 5,767 came in using drugs or alcohol and either stopped or reduced use. Approximately 2,400 consumers got worse. Almost all consumers abstaining at the start of treatment remained abstinent—Auditors’ analysis of consumers’ self-reported data showed that more than half (54.1 percent) reported they were not using a substance at the beginning of treatment. More than 93 percent of these consumers reported still being abstinent at their update or disenrollment (see textboxes bottom of this page and top of page 12). According to officials at the Division, RBHAs, and providers, several factors may explain why so many consumers report they are already sober upon entering treatment. For example, some consumers may have quit drinking or using drugs but enter treatment when they realize they cannot maintain sobriety without help. Others may misrepresent their substance abuse, either to make themselves look better or because of wishful thinking if they have been trying to stop for a period of time. Still other consumers may have a problem with binge drinking at irregular intervals. Finally, some consumers enter treatment after spending time in forced abstinence in jail or prison and mistakenly report they have been abstinent, although they are supposed to report on their use prior to entering jail or prison. Office of the Auditor General page 11 Usage outcomes for all consumers1 1 Numbers do not total 100 percent because of rounding. Source: Auditor General staff analysis of division data for adult substance abuse consumers without a serious mental illness enrolled during fiscal years 2006, 2007, or 2008. Consumers who reported using alcohol or drugs when treatment started Reduced use 770 1.5% Stopped using 4,997 10.0% Stayed the same 16,504 33.0% Got worse 693 1.4% Consumers who reported being abstinent when treatment started Stayed abstinent 25,376 50.7% Did not stay abstinent 1,729 3.5% Total 50,069 Extent to which consumers reported being abstinent from alcohol or drug usage when treatment started Source: Auditor General staff analysis of division data for adult substance abuse consumers without a serious mental illness enrolled during fiscal years 2006, 2007, or 2008. Abstinent 27,105 54.1% Using alcohol or drugs 22,964 45.9% Total 50,069 Most consumers using alcohol or drugs at the start of treatment continued to do so—Among the consumers who reported that they were using alcohol or drugs in the 30 days prior to treatment, the majority—nearly 72 percent— reported no change in the frequency of use of their primary substance at their update or disenrollment, while 3 percent actually increased their frequency of use (see textbox, middle of this page). The remaining approximately 25 percent reported either reducing their use of alcohol or drugs or stopping use altogether. For consumers who started treatment using drugs or alcohol, completing treatment appears to be a factor in decreasing substance use. Those who completed treatment had greater success in stopping their drug or alcohol use than those who left without completing treatment. More specifically, among this group, only 17.6 percent who did not complete treatment stopped using drugs or alcohol, but 27 percent of those who completed treatment stopped (see textbox, bottom of this page). Percentage of consumers with recent arrests showed little change—Auditors’ analysis of division data found no significant change in criminal activity as measured by recent arrests. Altogether, 21.2 percent of consumers reported arrests within the past 30 days upon entering treatment, while 18.1 percent reported new arrests at the time of their update or disenrollment date. Most consumers reported not having any arrests within the 30 days prior to starting treatment and only a small number of those consumers had a new arrest at the time of their update or disenrollment (see textbox, page 13). By contrast, of the consumers who reporting being arrested within 30 days of entering treatment, although approximately one–third reported no new arrests as of their annual update or enrollment, the majority—68.1 percent—reported at least one more arrest when they had their annual update or disenrollment. NARBHA, the RBHA for Northern Arizona, showed the greatest reduction of criminal activity among consumers who had been arrested within 30 days of entering treatment. Specifically, 41.2 percent of these NARBHA consumers did not have a new arrest when they had their update or disenrollment. State of Arizona page 12 Usage outcomes for consumers who reported using alcohol or drugs when treatment started1 Source: Auditor General staff analysis of division data for adult substance abuse consumers without a serious mental illness enrolled during fiscal years 2006, 2007, or 2008. 1 Numbers do not total 100 percent because of rounding. Usage outcomes for consumers who reported being abstinent when treatment started Source: Auditor General staff analysis of division data for adult substance abuse consumers without a serious mental illness enrolled during fiscal years 2006, 2007, or 2008. Stayed abstinent 25,376 93.6% Did not stay abstinent 1,729 6.4% Total 27,105 Reduced use 770 3.4% Stopped use 4,997 21.8% Stayed the same 16,504 71.9% Increased use 693 3.0% Total 22,964 Usage outcomes for consumers who reported using alcohol or drugs when treatment started and who completed treatment Source: Auditor General staff analysis of division data for adult substance abuse consumers without a serious mental illness enrolled during fiscal years 2006, 2007, or 2008. Excludes consumers who were still in treatment or who disenrolled for administrative reasons such as transfers between RBHAs, and 278 consumers who enrolled as a result of a crisis and then disenrolled. Reduced use 121 1.7% Stopped use 1,894 27.0% Stayed the same 4,855 69.2% Got worse 144 2.1% Total 7,014 Usage outcomes for consumers who reported using alcohol or drugs before treatment and who did not complete treatment Reduced use 432 3.4% Stopped use 2,270 17.6% Stayed the same 9,771 75.8% Got worse 410 3.2% Total 12,883 Employment did not significantly change—The Division’s programs have not led to a significant change in overall consumer employment. Thirty-eight percent of substance abuse treatment consumers, or 16,360, reported that they were already employed when they entered treatment, and 41 percent reported being employed as of their annual update or disenrollment. Of those consumers who were already employed, more than 92 percent reported being employed as of their annual update or disenrollment (see textbox, page 14). Among the almost 27,000 consumers, or 62 percent, who reported not being employed upon entering treatment, nearly 10 percent reported that they had gained employment as of their annual update or disenrollment, although 90 percent were still unemployed. NARBHA showed the most improvement in this outcome area, as 16.1 percent of NARBHA’s unemployed consumers had gained employment as of their update or disenrollment. Homelessness showed little change—Overall, the number of consumers who were homeless did not change from entering treatment to update or disenrollment, with 7.1 percent homeless upon entering treatment and 6.8 percent homeless at update or disenrollment. Most consumers were not homeless upon entering treatment, and those who were homeless had some Office of the Auditor General page 13 Arrest outcomes for all consumers Arrest outcomes for consumers with no recent arrests when treatment started Arrest outcomes for consumers with recent arrests when treatment started Source: Auditor General staff analysis of division data for adult substance abuse consumers without a serious mental illness enrolled during fiscal years 2006, 2007, or 2008. Consumers with no recent arrests when treatment started Had new arrest before leaving treatment 1,907 3.7% Did not have new arrest 38,565 75.1% Had new arrest before leaving treatment 7,408 14.4% Did not have new arrest 3,468 6.8% Total 51,348 Consumers with recent arrests when treatment started Had new arrest before leaving treatment 1,907 4.7% Did not have new arrest 38,565 95.3% Total 40,472 Had new arrest before leaving treatment 7,408 68.1% Did not have new arrest 3,468 31.9% Total 10,876 success moving into stable housing situations. Specifically, almost 46,000, or 93 percent, of substance abuse consumers reported living in a stable housing situation upon entering treatment, and almost 99 percent of these consumers reported being in stable housing at their annual update or disenrollment (see textbox, page 15). Of the more than 3,500 consumers, or 7 percent, who reported not being in stable housing when they entered treatment, approximately 21.7 percent of these consumers reported gaining stable housing by the time of their annual update or disenrollment. Housing services are generally not available to substance abuse treatment consumers. Such services are not covered by Title XIX, and state monies only provide the services for consumers with serious mental illness. Still, increased housing stability is considered an important measure of substance abuse treatment success. Limited comparative information indicates improvements in Arizona’s program may be low—As part of the analysis, auditors compared Arizona’s NOMs with those reported by other states. The Division uses this information to compare Arizona outcomes to other states. The available data, while limited in quality and consistency, suggests that Arizona’s performance is below that of substance abuse programs in other states. The limitations in the State of Arizona page 14 Employment outcomes for all consumers1 Employment outcomes for consumers who reported being employed before starting treatment Employment outcomes for consumers who reported being unemployed before starting treatment 1 Numbers do not total 100 percent because of rounding. Source: Auditor General staff analysis of division data for adult substance abuse consumers without a serious mental illness enrolled during fiscal years 2006, 2007, or 2008. Consumers who were employed before starting treatment Unemployed when treatment ended 1,255 2.9% Employed when treatment ended 15,105 34.9% Unemployed when treatment ended 24,245 56.0% Employed when treatment ended 2,656 6.1% Total 43,261 Consumers who were unemployed before starting treatment Unemployed when treatment ended 1,255 7.7% Employed when treatment ended 15,105 92.3% Total 16,360 Unemployed when treatment ended 24,245 90.1% Employed when treatment ended 2,656 9.9% Total 26,901 quality of the comparative data stem from differences in how state programs are structured and what the states may be reporting. For example, the data that Arizona reports to SAMHSA includes substance abuse treatment for consumers with serious mental illness, which probably contributes to the lower percentages of consumers meeting the measurement. Whether other states similarly report people with serious mental illness as part of their substance abuse cases is unknown. Thus, while the comparison may be useful in providing a general indication of where Arizona stands relative to other states, it should not be taken as authoritative. Even so, however, Arizona’s apparent ranking relative to other states is another reason to examine the program carefully to determine if it can be improved. Auditors examined the national data collected by SAMHSA at the federal level and found that both nationally and in the SAMHSA-designated Western Region, the percentage of Arizona’s substance abuse treatment consumers meeting the measurement criteria is below average in three of the four areas auditors analyzed: abstinence, criminal activity, and employment.1 This is the case both for consumers who were entering treatment and for consumers at the time of their discharge. Further, the rate of change for Arizona’s consumers from admission to Office of the Auditor General page 15 Housing outcomes for all consumers1 Housing outcomes for consumers reporting they were not homeless before starting treatment Housing outcomes for consumers reporting they were homeless before starting treatment Source: Auditor General staff analysis of division data for adult substance abuse consumers without a serious mental illness enrolled during fiscal years 2006, 2007, or 2008. Consumers who were not homeless before starting treatment Homeless at update or discharge 612 1.2% Not homeless at update or discharge 45,282 91.6% Homeless at update or discharge 2,763 5.6% Not homeless at update or discharge 765 1.5% Total 49,422 Consumers who were homeless before starting treatment Homeless at update or discharge 612 1.3% In stable housing at update or discharge 45,282 98.7% Total 45,894 Homeless at update or discharge 2,763 78.3% In stable housing at update or discharge 765 21.7% Total 3,528 1 Numbers do not total 100 percent because of rounding. 1 SAMHSA’s designated western states reporting NOMs data as part of their fiscal year 2009 SAPT block grant application were Alaska, Arizona, California, Hawaii, Idaho, Montana, Nevada, Oregon, Washington, and Wyoming. discharge is below both the national average and the average for western states on all four measures—that is, the substance abuse programs in other states appeared to have a greater effect in raising outcomes in these other states. (See Appendix A, Figures 6 through 9, 11, 12, 14, 15, 17, and 18, pages a-vi through a-xxi, for comparisons of Arizona’s reported NOMs with other states’ reported NOMs, both regionally and nationally.) Case studies help further explain factors affecting treatment—To provide additional perspective on the program, auditors also analyzed a limited set of specific cases to understand consumers’ progress or lack of progress and the factors that affected success. Auditors examined case files from specific treatment episodes and then interviewed the consumers, treatment providers, and others involved in the case to determine long-term outcomes and what consumers thought contributed to their abstinence or use. Two such cases help illustrate the complicated factors that can affect individual long-term outcomes: Juan, 44, a married man with children who lives in southwestern Arizona, entered substance abuse treatment after being arrested for driving with a blood alcohol level of 0.17 and charged with extreme DUI.1 The court suspended his driver’s license and ordered him to undergo treatment. The court referred Juan to a local mental health agency that has a contract with the RBHA to provide substance abuse counseling in his area. The provider placed Juan in a 9-week substance abuse and relapse prevention program consisting of twice-weekly group therapy focused on education and relapse prevention. At Juan’s intake session with the provider, he stated that prior to his arrest he did not drink very often, but would have seven or eight beers when he was out with his friends. Although the reason he went to substance abuse treatment was to clear up his legal troubles related to the DUI, Juan reported to auditors that the program gave him insight into the dangers of drinking and driving and the impact his drinking had on his family. Juan completed his treatment in December 2005, and when auditors interviewed him in February 2009, he reported that he had not had a drink in 3 years. In this case, the consumer’s NOMs outcomes were unchanged from beginning treatment to completion of treatment because he had a high level of achievement before entering treatment. Juan had never been arrested before. He was employed and living in a stable family situation, and he reported no alcohol use within the 30 days before he entered treatment as well as within the 30 days before he was disenrolled. Dana, 19, entered treatment in northern Arizona after being arrested and serving 60 days in jail on charges of drug sale, possession of drug paraphernalia, and shoplifting. Two of her charges were felonies. Her State of Arizona page 16 1 All consumer names have been changed. substance abuse treatment was required as part of her probation. She had been using alcohol since age 13 and methamphetamine since age 16, and had had previous substance abuse treatment and periods of abstinence. At her intake, Dana stated that she needed supervision and did not trust herself to abstain. Her provider recommended a month of residential treatment and then moved Dana into a two-stage intensive outpatient program. Over the 11 months of her treatment, she had considerable positive change in her NOMs. At her enrollment, she had just been released from jail, was unemployed, was not attending school, and had no stable place to live. At disenrollment she was self-supporting, enrolled in college, living with a family friend, and had no additional arrests. She was involved in Narcotics Anonymous and was not using drugs or alcohol. In Dana’s case, the NOMs data showed substantial progress on all four measures. However, Dana was unable to sustain these changes and relapsed about 1 year after completing treatment. When auditors spoke with her in February 2009, she was in prison with 5 more years to serve on a drug-related conviction. Case studies such as these, together with the results of the auditors’ analyses of the four NOMs, illustrate the difficulty in making decisions about how to improve program success. For example, the Division captures NOMs data at various points during treatment, but the data alone does not adequately indicate whether or not treatment has been successful. Auditors’ review of studies, best practices, and other materials showed, however, that improvements in program outcomes are likely to come from focusing on three main strategies: increasing the number of consumers who complete their treatment program, providing a continuum of care that addresses multiple needs, and using practices that have been shown to generate positive outcomes. The sections that follow discuss each of these strategies in turn. Division should increase focus on treatment retention and completion Research and the Department’s own data indicate that consumers who remain longer in treatment experience better outcomes. However, most consumers state-wide do not complete their treatment. As the analysis discussed on page 12 indicated, those who complete treatment experience better outcomes. Several factors, such as individual motivation and the availability of different treatment options, can influence whether or not consumers stay in treatment. The Division can take several steps to increase consumer retention and completion rates, including establishing performance goals, monitoring completion rates, and using financial and/or nonfinancial incentives and other case management techniques. Office of the Auditor General page 17 Longer treatment yields better results—Research has shown that consumers who remain longer in treatment, no matter what type of treatment it is, show the best post-treatment outcomes.1 For example, studies have shown that longer treatment duration resulted in lower readmission rates and lower rates of drug use and relapse. One study found that consumers who received long-term residential or outpatient treatment for 6 months or more showed a lessened likelihood of engagement in illegal activities than those with shorter lengths of stay.2 Another reported that longer lengths of stay were positively associated with post-treatment client earnings across all types of treatment.3 The National Institute on Drug Abuse states that most consumers need to remain in treatment for about 3 months in order to achieve significant improvement, and additional progress toward recovery can be achieved by providing treatment after the 3-month threshold.4 However, a 2003 study suggested that consumers who stay in treatment for longer than 18 months may start to show reduced amounts of improvement.5 Ensuring treatment completion is an important strategy for improving consumers’ outcomes. Auditors’ analysis of division data found that in addition to being more likely to reduce or stop substance use, consumers who completed treatment were more likely to gain employment and avoid criminal activity than those who did not complete treatment. For example, of the consumers who were unemployed when they entered treatment, 14.4 percent of those who completed treatment gained employment, while only 6.2 percent of noncompleters gained employment. Similarly, of the consumers who had been arrested within the 30 days prior to entering treatment, 47.5 percent of those who completed treatment had no new arrests at their update or discharge, but only 27.5 percent of those who did not complete treatment had no new arrests. Most consumers drop out or receive short-term treatment—Despite the importance of retention, auditors’ analysis of division data found that a majority— 58 percent—of the consumers reviewed did not complete their treatment before they were disenrolled. All six geographic services areas had fewer consumers who completed treatment than those who did not complete treatment (see Figure 3, page 19). The most common reason given for disenrolling consumers who did not complete treatment was that providers lost contact with them (38.3 percent of all reviewed consumers), while 9.4 percent of reviewed consumers refused further treatment. According to the Division, without a court order, consumers’ treatment is voluntary and they can withdraw at any time. Other reasons for disenrollment are death, incarceration, and a loss of Title XIX status. Further, in contrast to the 3 months suggested in literature, the Division reported that in fiscal year 2007, its average length of stay for outpatient treatment was 29 days and for long-term residential care the average length of stay was 78 days. In that fiscal year, more than 60,000 consumers were treated in outpatient programs, while fewer than 700 were treated in residential, long-term programs. State of Arizona page 18 1 McLellan, Chalk, & Bartlett, 2007 2 Etheridge, Craddock, Hubbard, & Rounds-Bryant, 1999 as cited in Koenig, Harwood, Sullivan, & Sen, 2000 3 Koenig, Harwood, Sullivan, & Sen, 2000 4 National Institute on Drug Abuse, 2009 5 Zhang, Friedmann, & Gerstein, 2003 Studies show that treatment retention and completion result in better outcomes. Office of the Auditor General page 19 39.4 48.7 37.6 43.3 44.5 42.5 50.8 60.6 51.3 62.4 56.7 55.5 57.5 49.2 0 10 20 30 40 50 60 70 NARBHA (GSA 1) Cenpatico (GSA 2) CPSA (GSA 3) Cenpatico (GSA 4) CPSA (GSA 5) Magellan/VO (GSA 6) Tribes Percent of Consumers Completed Treatment Did Not Complete Treatment Figure 3: Treatment Completion and Noncompletion Rates by RBHA (GSA) Fiscal Years 2006 through 2008 Source: Auditor General staff analysis of data from the Division’s Client Information System (CIS) for adult substance abuse treatment consumers without a serious mental illness enrolled in fiscal years 2006, 2007, or 2008. State of Arizona page 20 Several factors affect retention—Whether a person stays in treatment depends on factors associated with both the individual and program, according to the National Institute on Drug Abuse (Institute). The Institute’s Principles of Drug Addiction Treatment states that “individual factors related to engagement and retention include motivation to change drug-using behavior, degree of support from family and friends, and whether there is pressure to stay in treatment from the criminal justice system, child protective services, employers, or the family. Within a treatment program, successful counselors are able to establish a positive, therapeutic relationship with the patient. These counselors also ensure that a treatment plan is established and followed so that the individual knows what to expect during treatment.”1 Similarly, the Treatment Research Institute (TRI) states that retention can be improved by making treatment attractive, offering options, increasing monitoring and management, and giving consumers feedback.2 The TRI points out that consumers who are not in some form of treatment or monitoring are at increased risk of relapse. Other researchers have also reported that providing incentives to consumers, such as a reward for a clean urinalysis test, can help increase retention. Division can do more to increase retention—The Division does not monitor treatment retention or length of stay, and although it collects consumer disenrollment reasons it does not analyze or monitor the data. Its contracts with the RBHAs do not require them to collect or report information on retention or length of both inpatient and outpatient stay. The contracts require efforts to re-engage consumers in treatment—specifically, the RBHAs must require their providers to attempt re-engagement, and the Division’s provider manuals also establish specific requirements regarding attempting to re-establish contact with consumers—and the Division’s Administrative Review monitors RBHA’s compliance with this standard. However, auditors’ analysis of division data found that 47.7 percent of consumers could not be re-engaged in treatment (38.3 percent could not be contacted by providers and 9.4 percent refused further treatment), and the 2007 Administrative Review showed all four RBHAs needed improvement on the re-engagement measure. One of the auditors’ case examples illustrates this. Dana—the methamphetamine user who relapsed after treatment completion and was in prison when auditors interviewed her in February 2009— had spotty attendance during her treatment, including a period in May 2006 where she attended less than half her group sessions. Her case manager did not address her attendance issue until Dana asked about being released from treatment. Then, she was informed that her probation required her to maintain acceptable attendance in order to be considered for graduation from the program. Some other states have adopted goals and incentives related to retention and treatment completion. For example: 1 National Institute on Drug Abuse, 2009 2 McLellan, 2006 Office of the Auditor General page 21 Colorado—To meet performance goals, providers must maintain or improve the average percentage of clients from the previous 2 fiscal years who remain in outpatient treatment longer than 90 days, and maintain or decrease the percentage of consumers who left treatment against professional advice. Maine—Performance goals stipulate a minimum of 50 percent of outpatient and 85 percent of intensive outpatient consumers stay at least four sessions. Additionally, 30 percent of outpatient consumers must stay in treatment 90 days or longer, and 50 percent of intensive outpatient treatment consumers must complete treatment. Providers who exceed these expectations are awarded incentive bonuses, while underperforming providers are subject to disincentive penalties. Delaware—Contracts stipulate a $100 bonus incentive to providers for each consumer who completes at least 60 days in treatment and achieves the major goals of his or her treatment plan, including submitting four consecutive drug-free urine screenings randomly administered. To help improve treatment outcomes, the Division should establish performance goals and financial or nonfinancial incentives and disincentives related to retention and treatment completion in its RBHA contracts, and collect and monitor data on retention and completion. In establishing benchmarks and incentives, the Division should consider the potential for unintended consequences such as retaining consumers inappropriately and take steps to ensure that the incentives achieve the desired goals. For example, it could follow Delaware’s example of combining retention with other goals in order to determine whether providers qualify for incentives. In addition, the Division should use its existing oversight practices such as its quarterly case reviews to determine whether providers are taking appropriate steps to retain and engage consumers in treatment. Based on the results of these reviews, the Division should work with the RBHAs to address weaknesses through mechanisms such as technical assistance, training, contract requirements, and/or policy and procedural changes. Division should monitor continuum of care A second strategy for improving consumer outcomes is to provide a continuum of care. Under a continuum of care approach, consumers are placed in treatment based on the types of services they need and the phase of recovery they are in. After initial placement, consumers then step up to more intensive treatment or down to less intense treatment as needed. However, as illustrated by case studies, not all consumers have their treatment needs met or receive the services and therapies that may improve the chances of good treatment outcomes. The Division can do more to ensure continuum of care for consumers by taking several steps, such as collecting and monitoring data relevant to assessment and case management, and working with the RBHAs to make improvements when weaknesses are identified. State of Arizona page 22 Continuum of care matches consumers to appropriate treatment based on their problems and stage of recovery—Because no single treatment is appropriate for all individuals, providers need to be able to match treatment settings, interventions, and services to each individual’s particular problems. Continuum of care is a treatment system in which consumers entering treatment are placed in a level of treatment appropriate to their assessed needs. Individuals are first assessed to determine appropriate treatment services. Successful matching of the consumers to the appropriate treatments for them helps to ensure positive outcomes.1 After initial placement, consumers then step up to more intensive treatment or down to less intense treatment as needed. As the consumers recover, and further assessments are made of consumers’ progress, the treatment is phased down and consumers can be moved into alternative community-based services. Some consumers receive continuum of care while others do not— One of the auditors’ case studies illustrates how treatment that is appropriate to a consumer’s needs can yield long-term results: Mandy, 29, was in treatment for nearly a year. When she contacted a provider, she had already had previous experience with treatment, she was pregnant, her husband was in jail, and her two children were living with relatives. She reported that she had used methamphetamines, alcohol, and marijuana within the past several weeks. After completing a month of treatment with her first provider, she was referred to a specialized residential treatment center for women who are pregnant or have children that offers integrated services designed to teach them to be better parents and give them skills they need to get jobs, take care of their children, and stay sober. As a pregnant woman and therefore part of a priority population, Mandy’s treatment was eligible for SAPT funding. During almost 1 year of treatment with this second provider, Mandy made the decision to end her marriage and place her baby with adoptive parents in an open adoption. At the end of treatment, Mandy had a job, was living with her mother and one of her sons, and was working on her relationship with her other son. When auditors interviewed her in April 2009, 2 years after she completed treatment, Mandy stated that long-term treatment that focused on all her issues gave her a solid foundation for a sober life. She said she no longer had any cravings or desire to use drugs. During her year of integrated treatment, she learned to address the issues that led to her addiction. Her previous short-term treatments had not given her the ability to reflect on the information provided during treatment. Mandy now works as a peer support specialist in the substance abuse field, has custody of one of her sons, and as of April 2009 was about to get custody of her other son. In addition, she was living independently and was in contact with her daughter’s adoptive parents. Continuum of care is an important strategy in ensuring treatment is successful. 1 Mee-Lee, n.d. Office of the Auditor General page 23 By contrast, another example, identified through auditors’ analysis of high-cost consumers, illustrates a failure to provide services along the continuum from acute detoxification and stabilization through rehabilitation to continuing recovery support: Rodney, 43, used medical detoxification services 54 times during the period July 2005 through June 2008—approximately every 3 weeks—receiving services valued at a total of more than $82,000. (See Finding 2, pages 31 through 42, for a discussion of service costs). Rodney was homeless and reported using alcohol daily. Although he received some case management and other crisis services besides detoxification, he received very little in the way of other services, such as treatment that would reinforce his sobriety after detoxification and rehabilitation to help him learn to remain sober in the long term. According to a case manager at the detoxification center, Rodney typically stays in the center for only 2 to 3 days and chooses not to complete treatment or receive follow-up care. A consumer like Rodney who undergoes only the first stage of care is likely to return to substance abuse. Further, when auditors discussed this case with the Division’s medical director, he expressed concern that Rodney’s health is jeopardized by this constant cycling between heavy drinking and medical detoxification. Consumers are sometimes placed into services that may not meet their individual needs—contrary to the continuum-of-care concept—or that duplicate services they have already received. Auditors’ case studies included three consumers who received the same treatment program from different providers in different parts of the State: Juan, discussed previously, and two other consumers, Tom and Victoria. All three received alcohol education classes designed for people who have received their first DUI or alcohol-related arrest. These classes are intended to get participants to recognize problematic behaviors and make better choices in the future. For Juan, who had his first arrest for DUI at the age of 44, the class was an effective wake-up call regarding the dangers of impaired driving and how drinking affected his family relationships. He told the auditors that his 9 weeks of classes were very helpful to him and improved his life. Tom and Victoria, however, had long histories of alcoholism and completed the classes after serving prison sentences for drunken driving. Both Tom and Victoria had received similar classes while in prison. Tom told auditors that going through the same treatment after his release was not helpful and did not contribute to his continued sobriety. Instead, Tom and Victoria felt they needed treatment that would have helped them adjust to life after prison and address other issues. Both said their continued sobriety resulted from their own desire to avoid further legal troubles. In addition to auditors’ case studies that found that continuum of care is not consistently and appropriately provided, a 2008 study by one RBHA also found cases where consumers did not receive needed substance abuse services. Auditors’ case studies and a RBHA study revealed that not all consumers’ needs were met. 1 De Leon, Melnick, & Cleland, 2008 2 McLellan, Weinstein, Shen, Kendig, & Levine, 2005b State of Arizona page 24 Specifically, the Community Partnership of Southern Arizona (CPSA) conducted a focus review in August 2008 of a psychiatric health facility in its network and found that there was little evidence to support that the facility provided active treatment to the ten consumers sampled in the review, including three consumers who required substance abuse treatment (For more information on this review, see Finding 2, pages 31 through 42). The review found that none of the three consumers received substance abuse treatment services during their 7- to 10-day inpatient stays at the facility. Continuum of care requires that consumers’ needs are assessed and that they receive treatment that matches their needs. Division can do more to ensure continuum of care—The Division’s contracts with the RBHAs already specify that providers are required to develop treatments and services designed to support long-term recovery and focus on life factors such as employment, ongoing feedback to the recipient, and re-engagement into treatment based on the recipient’s changing needs. However, the Division’s oversight processes have not focused on ensuring that the RBHAs have implemented these requirements for substance abuse consumers (See Finding 2, pages 31 through 42, for recommendations on improving oversight of substance abuse treatment). As the Division moves to ensure that the RBHAs implement a more effective continuum of care, it should adopt two approaches shown by research to improve consumer outcomes. Specifically: Establishing assessment standards—The Division should establish standards for assessing the severity of consumers’ substance abuse problems and referring consumers to appropriate treatment, and monitor implementation of these standards as part of its regular oversight of RBHA performance. A 2008 study found that assessing the severity of a consumer’s substance abuse problem and then assigning more severe consumers to more intensive treatment improves both outcomes and cost-effectiveness of treatment.1 The study reported that appropriate placement does not require a complex matching process where individuals are matched with a specific style of treatment, such as cognitive behavioral therapy versus a 12-step program (see textbox, page 25). Defining case management expectations—The Division should define appropriate expectations for case management of substance abuse consumers, and consider assigning consumers with severe or complex cases to a case manager for their complete course of treatment, taking into consideration the costs of case management. For example, effective clinical case management can reduce inefficient over-use of detoxification. A 2005 study in Philadelphia concluded that multiple episodes of detoxification without follow-up care are inappropriate, ineffective, and inefficient.2 Pairing detoxification with clinical case management resulted in a 55 percent Office of the Auditor General page 25 reduction in detoxification admissions and a 70 percent increase in use of rehabilitation. Clinical case managers were expected to engage consumers early and assess their substance use, health, and social problems; use motivational interviewing designed to help consumers recognize and accept the need for continued care; and play an active, participative role in helping the consumers access and continue with needed medical and social services (See Finding 2, page 40, for information on how the Division can use case management to help minimize costs). When applying these approaches in its management of the program, the Division should also collect and monitor data relevant to assessment and case management and work with the RBHAs to make improvements when its oversight identifies weaknesses. Using appropriate evidence-based practices can improve success rate A third strategy the Division can employ is to increase its attention to providers’ use of evidence-based practices. These practices are validated by observation or experience as improving treatment success. Although the Division requires the RBHAs to use evidence-based practices, RBHAs are not necessarily doing so, and the Division is not ensuring compliance. To improve adherence to evidence-based practices, the Division should enhance its monitoring of the practices and take appropriate steps to address any weaknesses it finds. Evidence-based practices important to success—Evidence-based practices are therapies confirmed by scientific research to have a positive effect in the treatment of substance abuse. They include motivational interviewing, cognitive behavioral therapy, community reinforcement therapy, and 12-step facilitation therapy (see textbox). There is a growing body of literature on the cost-effectiveness of well-defined treatments that have been validated by observation or experience for substance use disorder, according to a 2007 study.1 SAMHSA has developed the National Registry of Evidence-based Programs and Practices, a listing of programs verified by independent researchers for their ability to achieve 1 Olmstead, Sindelar, Easton, & Carroll, 2007 Examples of Evidence-based Practices • Motivational Interviewing—A counseling style designed to help consumers recognize and accept the need for continued care. (www.motivationalinterview.org) • Cognitive behavioral therapy—A treatment that seeks to change the way an individual thinks to feel/act better even if the situation does not change. (www.nacbt.org) • Community reinforcement therapy—Therapy that focuses on events that influence behavior, such as family and social events, and uses the events to support successful abstinence. (www.ncsacw.samhsa.gov) • 12 step facilitation therapy—A brief, structured therapy that supports initial and ongoing participation in a 12- step program such as Alcoholics Anonymous. (www.nrepp.samhsa.gov) • ASAM criteria—Placement guidelines that focus on assessing consumers across six dimensions such as emotional and behavioral conditions and readiness to change and then placing the consumer in a level of care to match the severity of his/her condition. (www.asam.org and www.coce.samhsa.gov) • Stages of change—A model of behavioral change that progresses from examining the risks of change to commitment to action, and then plan, implement, and maintain the change. (www.coce.samhsa.gov) Source: Auditor General staff summary of evidence-based practices as defined on the above Web sites on June 24, 2009. State of Arizona page 26 results when properly implemented. The Division’s review of residential treatment centers supports the use of such evidence-based practices and the results that can be achieved. The Division contracted with a private firm to review nine substance abuse residential treatment centers that provide such services mainly to women to gather baseline data. In that review, all nine centers reported that they used at least two forms of evidence-based best practices. The three most commonly reported practice models across all nine centers included the American Society of Addiction Medicine (ASAM) criteria, motivational interviewing, and stages of change. The report indicated that 97.5 percent of the cases reviewed showed evidence of symptomatic improvement. Evidence-based practices are also used for outpatient treatment. For example, there are three Methamphetamine Treatment Centers of Excellence located throughout the State, which specialize in the treatment of methamphetamine use through techniques such as incentives, community reinforcement, motivational interviewing, and monitoring behavior through urine testing. Requirement for evidence-based practices not necessarily followed or enforced—The Division’s contracts with the RBHAs require the use of evidence-based practices, but the Division can do more to ensure providers are using the most up-to-date research and implementing the practices in accordance with the evidence. The Division has received or conducted three reviews of substance abuse programs. For example, in one study, a 2008 federal grant review that focused on programs for children and adolescents, the Division did a good job of establishing evidence-based practices in some areas, but did not identify such practices across the continuum of care. It could also do more to ensure sustainability of the emphasis on evidence-based practices. In another study, the Division reviewed 11 intensive outpatient programs for youth across the State and found that, although the Division had an internal definition of what constitutes an intensive outpatient program, the definition was not documented or adequately communicated to the providers. Therefore, a number of the providers’ programs did not meet the Division’s definition. One example of implementation of an evidence-based practice for substance abuse treatment is at NARBHA, which received a SAMHSA grant to implement a program called Matrix for adult treatment services. NARBHA uses a tool provided by the Matrix Institute to evaluate providers and ensure the program is implemented properly. The Matrix model incorporates therapist support, group participation, support group participation, relapse prevention and education, family involvement, and an explicit structure giving consumers a clear understanding of treatment. According to staff at one of NARBHA’s providers, the program has helped them get consumers into the structure of recovery. One element of the model, contingency management—where consumers are rewarded for successful participation, for example with prizes such as movie passes—is seen as helpful for keeping consumers engaged in treatment. NARBHA uses an evidence-based practice for substance abuse treatment and monitors it to ensure proper implementation. Office of the Auditor General page 27 The Division may be able to use one evidence-based practice—self-help group participation—more effectively. Research has shown a strong association between self-help group participation and long-term recovery following treatment. Although none of the consumers included in auditors’ case studies continued to participate regularly in self-help groups after treatment, one of them, Dana—the young woman serving a drug-related prison sentence—attributed her return to drug use to when she stopped attending Narcotics Anonymous meetings and continued to associate with friends who used drugs. She indicated that she found it useful to have group leaders who were recovering addicts because their familiarity with addiction makes them good at breaking down addicts’ behavioral defense mechanisms. Division can do more to encourage use of evidence-based practices—Auditors identified two key ways in which the Division can encourage greater use of evidence-based practices: More extensive monitoring—The Division examines a sample of substance abuse consumers’ cases each quarter as part of its overall oversight of the RBHAs in order to ensure consumers obtain services in a timely manner and receive appropriate services. In addition, in order to fulfill federal grant requirements, the Division has conducted a review of substance abuse programs for adolescents and contracted for a review of women’s substance abuse residential treatment facilities. Specifically, during 2007 and 2008, division staff reviewed 11 intensive outpatient programs for adolescents to determine how the programs could be improved. Also, in 2008, the Health Services Advisory Group, an external quality review organization contracted by the Division, conducted a study of women’s residential treatment facilities to gather baseline data on women’s substance abuse treatment programs. However, the Division does not routinely review services to determine whether the RBHAs are applying evidence-based practices or implementing them as they were designed to be used. Similar to NARBHA’s approach with its providers that use the Matrix model, the Division should monitor evidence-based practices either directly or by reviewing the RBHAs’ assessments of providers’ practices. Improved guidance—The Division should also continue and expand its work with RBHAs to ensure that providers have the guidance needed to implement specific evidence-based practices. When the Division implements new treatment models, it provides guidance and other assistance to ensure the RBHAs and providers implement the models correctly. For example, it used this approach to implement a new system of care for children in the early 2000s, as discussed in the Auditor General’s report on implementation of HB2003 programs for children (see Report No. 02-12). The Division also cosponsors an annual summer institute conference for RBHAs and providers. The Division should use the results of its monitoring of evidence-based The Division may be able to use self-help groups more effectively. practices implementation to identify any needed improvements, and then work with RBHAs to provide technical assistance, training, and guidelines as appropriate. Finally, the Division should make better use of self-help programs by developing a method to track and monitor participation and encouraging RBHAs to offer a wider variety of programs for consumers. Because self-help group participation can continue the benefits of treatment long after treatment is completed, the Division should also consider working with the RBHAs to develop procedures to follow up with consumers after treatment is completed to encourage continued participation in such groups. Recommendations: 1.1. To help improve retention, the Division should: a. Collect and monitor data on retention and completion, including length of stay and disenrollment reasons; b. Establish performance goals and financial and/or nonfinancial incentives and disincentives related to retention and treatment completion in its RBHA contracts, taking care to avoid encouraging providers to inappropriately retain consumers in treatment in order to meet the goals; c. Use its existing oversight practices such as its quarterly case reviews to determine whether RBHAs are taking appropriate steps to retain and engage consumers in treatment; and d. Based on the results of these reviews, the Division should work with the RBHAs to address weaknesses through mechanisms such as technical assistance, training, contract requirements, and/or policy and procedural changes. 1.2. To make better use of the continuum of care to improve treatment outcomes, the Division should: a. Establish standards for assessing the severity of consumers’ substance abuse problems and referring them to appropriate treatment; b. Using data, monitor implementation of these standards as part of its regular oversight of RBHA performance; c. Define appropriate expectations for case management of substance abuse consumers, taking into consideration costs of case management and the advantages of monitoring consumers with severe or complex cases; State of Arizona page 28 d. Collect and monitor data relevant to assessment and case management; and e. Work with the RBHAs to make improvements when its oversight identifies weaknesses. 1.3. To better ensure the use of appropriate evidence-based practices, the Division should: a. Monitor compliance with its contractual requirements to use evidence-based practices; b. Work with RBHAs to provide technical assistance, training, and guidelines as appropriate to ensure that providers have the guidance needed to implement specific evidence-based practices such as motivational interviewing, cognitive behavioral therapy, and community reinforcement therapy; c. Develop a method to track and monitor self-help group participation; d. Encourage RBHAs to offer a wider variety of self-help programs for consumers; and e. Consider working with the RBHAs to develop procedures to engage consumers in community and peer support outlets that would reinforce progress made in treatment once consumers are disenrolled. Office of the Auditor General page 29 State of Arizona page 30 Division should improve oversight of substance abuse programs The Department of Health Services, Division of Behavioral Health Services (Division), should take steps to improve its oversight of the substance abuse programs administered by the Regional Behavioral Health Authorities (RBHAs). Current oversight efforts are limited in two key ways. First, although the Division gathers information on treatment outcomes, it does not use this information to monitor programs, focusing instead on process-related information such as whether services are timely. To increase the benefit that treatment participants receive, the Division should continue its efforts to streamline uniform data collection that focuses on consumer outcomes, establish relevant performance goals in contracts with the RBHAs, and encourage the RBHAs to consider mechanisms for awarding financial and/or nonfinancial incentives to providers who meet these goals and penalizing those who do not. Second, oversight efforts currently lack any significant focus on analyzing treatment costs and types. This is particularly true for analyzing high-cost or low-cost cases that may indicate ineffective or inadequate services. To help better ensure that the Division’s scarce resources are used effectively, the Division should implement a systematic approach that includes but is not limited to reviewing high-and low-dollar cost services by individual users and treatment categories and among RBHAs, and considering how to best use assessment and case management to contain costs. Division should monitor outcomes The Division should improve oversight of substance abuse treatment by increasing its focus on consumer outcomes—that is, what the services are accomplishing in terms of keeping consumers from reverting to abuse of alcohol or drugs. The Division focuses primarily on treatment process measures for all behavioral health consumers, including substance abuse consumers, and it is therefore largely unable to determine if its substance abuse treatment programs are achieving positive results or if its resources are being used effectively. Focusing more on outcomes involves taking such steps as streamlining uniform outcome data collection, implementing minimum performance goals, and establishing performance incentives and penalties to help ensure that goals are met. Office of the Auditor General page 31 FINDING 2 Division monitors process, not treatment outcomes—The Division’s oversight of the behavioral health treatment system is limited to a number of process measures and is not focused on treatment results. For example, as required by its Arizona Health Care Cost Containment System (AHCCCS) contract, on a quarterly basis, the Division reviews the RBHAs’ ability to provide timely and appropriate services to consumers and coordination of care with the consumer’s primary care physician. Such measures are required for all Medicaid-covered services and provide information that is relevant to assessing whether consumers receive services, but they do not provide meaningful insight as to whether consumers are reducing their dependence on alcohol or drugs. The Division gathers consumer outcome data but does not appear to use it for monitoring purposes. The Division gathers outcome data such as information on abstinence and participation in employment or education when developing its Annual Report on Substance Abuse Treatment Programs, and when applying for the annual federal Substance Abuse Prevention and Treatment block grant. In addition, the Division reported that it uses outcome data to compare Arizona’s to other states’ outcomes. Division policies state that this information allows for the measurement of behavioral health outcomes, is required of all consumers, and is used to support quality and utilization management activities. However, auditors found no indications that the Division uses this information to evaluate RBHA performance in providing effective substance abuse treatment. The Division also does not use this outcome information to compare performance between RBHAs or between individual providers. As a result, the Division is unable to effectively evaluate system performance and determine which providers’ treatment services are resulting in positive consumer outcomes. Likewise, the Division is unable to identify underperforming providers and require that they make necessary improvements. Similarly, at the RBHA level, some RBHA representatives said that although they are required to gather the outcome data and submit it to the Division, they do not evaluate the outcome information as thoroughly as they could because they need more guidance from the Division on how this should be done. Therefore, the Division should provide more guidance to the RBHAs to help ensure they can thoroughly evaluate the consumer outcome information. Division should continue efforts to streamline uniform outcome data collection—The Division is revising its assessment process, and in doing so it needs to ensure the new process enables it to uniformly collect the data needed to assess consumer outcomes. The Division requires RBHAs to gather consumer outcome data during intake, update, and disenrollment assessments. Historically, according to division officials, providers have used a uniform core assessment tool mandated by the Division, but some have expressed concern about the length of the 32-page tool and the more than 1 hour that it took to fill it out. The tool, which is used for all the Division’s consumers, includes a full behavioral health assessment, not just a substance abuse assessment. During the audit, the Division began developing a new method for collecting data that would streamline the data collection process. The Division gathers consumer outcome data but does not use it for monitoring purposes. State of Arizona page 32 Because A.R.S. §36-2006(A) and (B)(4) require the Department to establish a standardized screening assessment for alcohol and other drug education and treatment programs, and because consistent outcome information is important for monitoring program success, the Division should continue in its efforts to streamline uniform collection of required outcome data. Another state has developed uniform requirements that are short, and the Division may be able to adopt its solution. Maine has developed a one-page form specifically for gathering consumer outcome information during assessments that includes standardized yes-or-no questions or provides a list of the answer options for each question. Adopting a similar form or modifying its existing method would address providers’ concerns about the form’s length while retaining the Division’s ability to effectively and uniformly collect, monitor, and easily validate outcome data as well as allowing it to comply with statute. Division should establish minimum outcome-based performance goals, incentives, and penalties—In 2006, a national policy panel convened to examine how state governments could be most effective in preventing and treating substance use disorders and problems, and reported that states should hold agencies and contracted providers accountable for meeting identified outcome measures and should reward those that meet or exceed outcome targets and penalize those that consistently fail.1 As discussed in Finding 1 (see pages 9 through 29), three states—Maine, Delaware, and Colorado— reported having established performance goals for providers on a variety of measures such as treatment retention, continuation of care, and abstinence. Delaware and Maine have both set a minimum number of consumers that must meet the goals while Colorado requires that providers maintain or improve their performance without specifying an exact number of consumers. However, a Colorado Division of Behavioral Health official reported that it will modify the performance goals in fiscal year 2010 to require that the providers obtain certain percentages rather than perform better than the average of their previous scores for 2 years. In addition, Colorado has established goals for consumer transition from detoxification to some other form of treatment (see textbox). Further, a fourth state, Nevada, is currently conducting a 3-year pilot program that includes performance goals. By contrast, although the Division has some contractual requirements regarding provider performance, it has not established benchmarks for clinical performance, or outcome goals that are specific to substance abuse treatment. 1 Rosenbloom, Leis, Shah, & Ambrogi, 2006 Office of the Auditor General page 33 Examples of Colorado’s performance goals Maintain or improve the percentage of detox clients who are referred to further treatment Maintain or improve the percentage of detox clients who enter any non-detox treatment setting within 30 days of their detox discharge Maintain or improve the percentage of treatment clients who stayed in outpatient treatment longer than 90 days Maintain or improve the average of treatment clients who had a reduction in primary drug use at discharge Source: Colorado’s fiscal years 2006 through 2008 state-wide treatment performance measures for all managed service organizations provided by an official from Colorado’s Department of Human Services, Division of Behavioral Health. In addition to performance goals, two states—Delaware and Maine—have incorporated financial incentives and disincentives into their performance goals that could serve as models for Arizona. For example, Delaware has established incentives for criteria such as patient participation and treatment retention/completion. For patient participation, the provider receives a 1 percent bonus if 50 percent of participating patients attend two sessions per week in the first 30 days of treatment. An incentive payment is also tied to completion, active participation for 60 days, achievement of the major goals of the treatment plan, and 4 consecutive weeks with clean urine samples. Maine also includes incentives and disincentives in its performance-based contract, which stipulates that their providers can collect a percentage of their quarterly payment each quarter they exceed their performance goals. Likewise, its providers can lose a percentage for each underperforming quarter. Division needs to establish benchmarks first, then incentives—The Division is not yet in a position to do what other states have done because it lacks clear benchmarks for measuring performance and incentives to encourage good performance. However, the Division has some contractual requirements regarding provider performance and financial incentives. For example, Magellan, the RBHA that oversees behavioral health services in Maricopa County, is required by contract to implement financial incentives with its service providers to increase performance on the National Outcome Measures by September 2009. According to a Magellan official, the RBHA is working with its providers to implement performance goals and to put a process in place for awarding incentives or levying penalties. Other RBHAs’ contracts state that the RBHA will share incentives originating from the Division with applicable providers. However, before performance incentives, or penalties, can be levied, the Division must first establish performance benchmarks or standards for treatment retention and completion and consumer continuation of treatment from detoxification to some other form of treatment and modify its contracts with the RBHAs to require such measures. According to a division official, the Division is exploring possible benchmarks for performance incentives or penalties, but difficulties in collecting and analyzing the necessary data have hampered efforts in establishing benchmarks. In April 2009, the Division re-organized its data management staff in order to address the data collection and analysis issues. The Division should encourage the RBHAs to consider contractually implementing a financial or nonfinancial method of incentivizing substance abuse treatment providers who exceed the goals established in their contracts and penalizing those providers who continually fail to meet these standards related to consumer outcomes, treatment retention, and treatment completion. Magellan is required by contract to implement financial incentives with its service providers to increase performance by September 2009. State of Arizona page 34 Division should monitor monies spent on treatment The second area in which improved monitoring is needed is more reviews of service costs. Reviews of over- and underutilization of healthcare services are important to help contain costs and manage quality of substance abuse treatment. Although the Division contractually requires the RBHAs to perform such reviews of services, the reviews should be expanded to identify unusually high and low costs of services to individual consumers and high treatment or services costs at each RBHA, and monitor variations in treatment across all RBHAs. The Division should continue and expand on the steps it took during the audit to begin reviewing service costs. In addition, the Division should also consider how to best use assessment and case management to contain costs. Lastly, the Division should continue its efforts to fill vacant positions and should perform follow-up work to ensure that the restructuring it initiated in April 2009 results in improved oversight. Utilization reviews can help contain healthcare costs and maintain quality of care—In a managed care system like Arizona’s, utilization reviews are critical to help control service costs while ensuring that quality of care remains high (see textbox). Because the Division uses a capitated system to pay RBHAs in advance for providing all required services, current-year costs are not reflected in current-year payments as they would be in a fee-for-service system. Rather, each year’s capitation rate is established based on analyzing actual costs of services delivered in previous years, so future payments are affected by current-year service costs. Research has shown that the cost-containment goals of managed care systems can be obtained without sacrificing client outcomes. However, attention must be paid to both cost-containment strategies and quality-of-care issues (such as evidence-based practices) if clients are to benefit from substance abuse treatment. Specifically, 2005 and 2006 studies that compared one California county that used managed care to two other counties that used a more traditional fee-for-service approach to substance abuse treatment found that the managed care approach was more cost-effective—it achieved the same results at a lower cost.1 The managed care county established various best practices to manage both cost and promote better outcomes including clear assessment guidelines, case management guidelines for complex cases, continuum of care requirements, performance-based contracting, and utilization management guidelines. For example, the utilization management guidelines included a standard 45-day length of stay for stabilization in residential treatment, followed by outpatient rehabilitation, and a standard 90-day outpatient treatment with intensity guidelines for the number of sessions. 1 Beattie, Hu, Li, & Bond, 2005; Beattie, McDaniel, & Bond, 2006 Office of the Auditor General page 35 Utilization review—A process to evaluate and approve or deny healthcare services, procedures, or settings based on medical necessity, appropriateness, effectiveness, and efficiency. Source: AHCCCS Medical Policy Manual Chapter 1000—Medical Management/Utilization Management. The Division’s contractual requirements for utilization reviews cover nine dimensions identified in the AHCCCS utilization management policy (see textbox), and RBHAs include all nine dimensions in their utilization management plans. As outlined in these plans, RBHAs review a wide range of issues, including data on readmissions, average length of stay, and pharmacy utilization, such as monitoring for multiple medicines taken to treat the same illness. By monitoring these measures of care, the RBHAs have information that would allow them to help control costs and identify potential issues with quality of care as illustrated in the following example: In August 2008, one RBHA, Community Partnership of Southern Arizona (CPSA), conducted a focus review of a psychiatric health facility. In that 10- case file review, CPSA found that there was little evidence to support that the facility provided active treatment to the consumers, including three consumers who required substance abuse treatment. As a result, CPSA made recommendations regarding treatment planning documentation and group therapy improvement. A CPSA official reported that CPSA will follow up with the facility in August 2009 to ensure that it implemented these recommendations. State of Arizona page 36 AHCCCS medical/utilization management policy requirements • Utilization Data Analysis and Data Management—Collect, monitor, analyze, evaluate, and report utilization data to detect both underutilization and overutilization of services. • Concurrent Review—Review the medical necessity for a planned institutional admission or ongoing institutional care. • Prior Authorization—Make medical decisions regarding the initial authorization of services or requests for continuation of services in specified time frames. • Retrospective Review—Provide policies, procedures on how to conduct reviews to determine medical necessity post delivery of services. • Adoption and Dissemination of Practice Guidelines—Adopt and disseminate practice guidelines that are based on valid and reliable clinical evidence or a consensus of healthcare professionals and include a thorough review of medical journals’ peer-reviewed articles published in the United States when national practice guidelines are not available. • New Medical Technologies and New Uses of Existing Technologies—Develop and implement written policies and procedures for evaluating new technologies and new uses of existing technologies on both an individual basis and systemic basis. • Case Management/Care Coordination—Ensure that a member’s needs are appropriately met through a coordination of cost-effective care involving early identification of health risk factors or special care needs whether in an acute, home, chronic, or alternative setting. • Disease/Chronic Care Management—Implement a program that focuses on members with high risk and/or chronic conditions to increase member self-management and improve providers’ practice patterns and thereby improve members’ healthcare outcome. • Drug Utilization Review—Review the prescribing, dispensing, and use of medications to ensure that the medications are clinically appropriate, safe, and cost-effective drug therapy that improves a member’s health status and quality of care. Source: AHCCCS Medical Policy Manual Chapter 1000—Medical Management/Utilization Management. Expanding existing reviews can help manage costs—Although current utilization reviews contain many elements that could potentially help manage costs, auditors identified several ways to improve their ability to do so. These include the following: Conducting reviews of high- and low-cost substance abuse treatment cases—Auditors’ analysis of division data identified 14 consumers who received substance abuse treatment valued at a total of over $100,000 each during fiscal years 2006 through 2008. The Division was not aware of these cases until auditors brought them to officials’ attention. In one of these cases, presented in Finding 1 as Rodney (see Finding 1, pages 9 through 29), the consumer repeatedly used medical detoxification services valued at a total of over $82,000 in fiscal years 2006 through 2008. Rodney’s stay at a Tucson detoxification center in September 2008 was typical of his pattern. He was brought to a hospital emergency room one evening by emergency medical services. At triage, he had a blood alcohol level of .396. The center provided Ativan, a medication for withdrawal symptoms; fed him; and applied a nicotine patch because the hospital is a nonsmoking campus. A registered nurse monitored him periodically while he slept. Rodney transferred to the detoxification center the following day. When the center conducted an assessment to determine his readiness for treatment to address his addiction, he said, “I’m not sure what I want to do after detox. I’ve never been to treatment, so I can’t say if I would like to go there.” Rodney discharged himself around noon on the third day, against medical advice. Auditors’ analysis of division data found 54 separate division data records for detoxification services for Rodney over the 3-year period analyzed, and a case manager at the center reported that Rodney used detoxification three or four times a month. Cases such as Rodney’s involve high costs, but appear to produce limited outcomes. Altogether, for fiscal years 2006 through 2008, approximately $100 million, or 72 percent of the total encounter value (reported value of services) for substance abuse treatment, was used to provide services to 20 percent of the consumer population, and 45 percent of these consumers did not complete treatment. Reviewing high-cost cases such as Rodney’s could enable the Division to work with the RBHAs and providers to find ways to treat consumers more effectively and break the cycle of substance abuse and detoxification. During the audit, in March 2009, the Division began reviews that focused on service costs for consumers with serious mental illness. Specifically, the Division reviewed Magellan’s fiscal year 2008 healthcare costs for consumers with serious mental illness and identified the top ten service users as Office of the Auditor General page 37 measured by service costs and 49 other clients who were enrolled in fiscal year 2008 but did not receive services that year, most of whom also had not received services in fiscal year 2009. Low-dollar service users could indicate potential quality of care issues or an inadequate disenrollment process. The Division presented the information to Magellan officials, who replied that the top ten service users were very complex cases involving safety concerns. Magellan added that it determined that all services provided to nine of the ten consumers were clinically appropriate and, as a result, would conduct an in-depth review of the clinical appropriateness of the services offered to the one other consumer. Regarding consumers who underutilized services, Magellan reported that they found an opportunity to improve timeliness of closures and follow up after crisis episodes. As a result, Magellan reported that it would implement a claims inactivity report to identify consumers who did not have encounters (service records) for periods of more than 210 days. However, this review was limited to one RBHA and consumers with serious mental illness. To more effectively control costs, the Division needs to continue and expand these efforts. The Division should perform similar reviews for consumers receiving treatment for substance abuse in order to identify, research, and appropriately address all high- and low-dollar service users. In addition, the Division should follow the example of the California managed-care county discussed on page 35 and establish benchmarks for length of stay, and collect and monitor data on length of stay in order to identify consumers who may be over- or under-utilizing certain types of services, which could indicate they need a different service to be treated cost-effectively. Conducting cost-focused reviews of specific treatments or services—In addition to identifying and addressing high- and low-dollar service users, the Division should also focus reviews on managing high-cost treatment or service types. Conducting such reviews would enable the Division to determine not only if the services are necessary, but help identify other effective, potentially less-costly treatments. In addition to reviewing the highest-cost consumers, auditors identified other consumers in various service categories such as transportation, inpatient, and outpatient treatment. Auditors specifically looked for cases with costs significantly higher than the median for that service category. Using division data, auditors identified consumers whose fiscal years 2006 through 2008 treatment values ranged from $50,000 to $100,000, and found that the costs of some of the specific services these consumers received were 15 times more than the median costs of such services. For example: • Ron, 52, has an opiate dependence that requires supervised methadone treatment. According to his treatment center’s clinical director, although Ron lives in Lake Havasu City, his treatment provider transports him twice a week over 180 miles round-trip to Bullhead City, which the clinical State of Arizona page 38 director added is the only authorized methadone treatment center in rural Mohave County. His treatment center’s clinical director also explained that because of federal methadone regulations and licensing requirements, it is very difficult to incentivize other healthcare facilities that may be closer to the consumer to become methadone treatment facilities.1 Therefore, his only option is to transport the client to the Bullhead City Community Medical Services II center at a value over fiscal years 2006 through 2008 of almost $48,000, which represents more than 87 percent of the total value of Ron’s treatment for that time period. This amount greatly exceeds the RBHA’s median transportation value of approximately $88. Examining costs such as this can help the Division identify high-cost practices and examine alternatives for reducing them. Methadone treatment is one area in which the Division has already begun to study alternatives. From fiscal years 2006 to 2008, approximately 2,600 consumers, or about 5 percent of all substance abuse consumers served, received methadone treatment valued at more than $9.5 million. According to a division official, the Division is transitioning to a new treatment, buprenorphine, which the consumer can take home and administer daily. In addition, in an effort to reduce transportation costs for consumers receiving methadone treatment, in February 2009, the Northern Arizona Regional Behavioral Health Authority (NARBHA) approved the purchase of a van to provide such services instead of paying a third-party vendor to do so. To help contain costs, such as transportation costs associated with methadone treatment, the Division and RBHAs should continue their efforts to transition to alternative treatments. In addition, all RBHAs and the Division should identify consumers with higher-than-usual costs for specific services, and alternative methods or treatments that would provide the same quality of care at a reduced cost. Comparing variation in treatment types among RBHAs—Finally, to better monitor ser |