Performance audit, Department of Economic Security, contracting practices

              PERFORMANCE AUDIT
DEPARTMENT OF ECONOMIC SECURITY
CONTRACTING PRACTICES
Report to the Arizona Legislature
By the Auditor General
November 1995
Report # 95- 10
DOUGLAS R. NORTON, CPA
AUOITOT1 GENERAL
STATE OF ARIZONA
OFFICE OF THE
AUDITOR GENERAL
DEBRA K. DAVENPORT, CPA
DEPUTI IUDITOl GENERAL
November 9, 1995
Members of the Arizona Legislature
The Honorable Fife Symington, Governor
Dr. Linda J. Blessing, Director
Department of Economic Security
Transmitted herewith is a report of the Auditor General, A Performance Audit of the
Department of Economic Security's contracting practices. This report is in response to
a May 5, 1993, resolution of the Joint Legislative Audit Committee. The performance
audit was conducted as part of the sunset review set forth in A. R. S. 5541- 2951 through
41- 2957.
This is the final report in a series of six reports issued on the Department of Economic
Security. Our report recommends that DES change the way it contracts with human
service providers. The current approach, in which the Department negotiates
individual rates with each provider, has resulted in wide differences in rates for
similar services. Inequities can occur. For example, low- quality providers may receive
higher rates than high- quality providers. To address this problem, DES should
consider adopting standard, uniform rates for similar services. We also found that the
Agency needs to limit use of consultants to appropriate purposes and manage
consulting contracts more effectively.
My staff and I will be pleased to discuss or clarify items in the report.
This report will be released to the public on November 13, 1995.
Sincerely,
~ d w aRs. Norton
Auditor General
Enclosure
2910 NORTH 44TH STREET 9 SUITE 410 1 PHOENIX, ARIZONA 85018 l( 602) 553- 0333 = FAX ( 602) 553- 0051
SUMMARY
The Office of the Auditor General has conducted an audit of the Department of
Economic Security ( DES), pursuant to a May 5,1993, resolution of the Joint Legislative
Audit Committee. This performance audit, the final in a series of six audits of DES, was
conducted as part of the sunset review set forth in Arizona Revised Statutes ( A. R. S.)
5541- 2951 through 41- 2957. In accordance with the authorizing resolution, this audit
examined an agencywide function within DES. In contrast, prior audits examined
individual DES divisions or programs.
This audit examined DES contracting practices for two specific types of contracts:
contracts with agencies providing human services, and contracts for consultant services.
These two functions are a subset of DES' total contracting responsibilities, which also
include day care services, contracts with foster care families, and DES contracts for
supplies, equipment, and business services.
DES is a major contractor of services. In fact, it spends approximately $ 243 million
annually on human service contracts alone - an amount greater than the budgets of 90
percent of all other state agencies. The effectiveness of DES contracting practices is
important not only because of the significant dollars involved, but also because it
impacts the care received by the many DES populations served. These populations
include some of the most vulnerable groups served by state government; for example,
children under state protective care, as well as persons with developmental disabilities.
The DES processes for setting contractor rates are crucial in human service contracting
because rates are established entirely by DES and the provider. In contrast, rates paid
by DES for medical, dental, and day care services are influenced by ceilings or guidelines
established by the Arizona Health Care Cost Containment System or the federal
government.
Additionally, the audit examined DES' use of consultants and associated DES contracting
practices. In the year ended June 30, 1994, DES spent over $ 4 million for consultant
services.
DES Needs to Adopt a More Equitable
Rate- Setting Methodology
( See pages 7 through 15)
DES' current method of setting human services providers' rates is not working. To set
rates, DES staff attempt to analyze an individual provider's costs for each different
service and then negotiate rates for each service with the provider. Currently, there are
over 10,000 rates in the State, and DES may have over 100 different rates for the same
service. However, negotiating rates on a case- by- case basis produces gross differences
in rates among providers of the same service. For example, DES' Administration of
Children, Youth, and Family has rate ranges from $ 41 per day to $ 118 per day for group
homes supervised by house parents. The DES Division of Developmental Disabilities
( DDD) also has extreme rate ranges for its most widely used service categories. Further,
rate differences are not related to DES' evaluations and ratings of provider attributes. In
some cases there appears to be an inverse relationship between rates and evaluation
scores - providers with higher ratings are often paid less than providers with lower
ratings.
DES should adopt a uniform rate- setting approach agencywide. In a uniform approach
rates are standard and, in general, providers in the same geographic area receive the
same rate for the defined client service and level of care. Other states found this
approach eliminated inequity and thereby improved provider relations and enhanced
competition for services. In fact, uniform rate setting is used or being implemented in
10 of 11 states surveyed and is the trend in human service contracting. DES'
Rehabilitation Services Administration has successfully employed this approach for
years.
DES Needs to Better Manage
Its Use of Consultants
( See pages 17 through 22)
Our review of 14 DES consulting contracts found that consultants were often used to
perform routine tasks or to replace DES management. Consultants have been hired to
write ad hoc reports and develop policy tasks that organizations typically accomplish
internally. For example, one consultant was paid $ 150 an hour to write a procedures
manual for case managers, a task that many DES staff are qualified to perform. DES also
appears to hire consultants as replacements for management. While they perform
essential work, they do so at considerable cost. One consultant, currently billing DES at
$ 90 per hour, managed a unit in the Division of Developmental Disabilities for 6 years.
Additionally, in some cases the consultant, rather than DES, controlled the project. In
one contract, DES paid over $ 1 million but received few deliverables resulting in benefit
to the State.
DES needs to ensure that it adequately assesses the need for a project or consultant
resources. It also can better define projects and ensure that contracts sufficiently specify
project tasks, deliverables, and due dates.
Table of Contents
Page
Introduction and Background . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Finding I: DES Needs to Adopt a More
Equitable Rate- Setting Methodology . . . . . . . . . . . . . . . . . . . 7
DES' Current Rate- Setting
Method Leads to Inequities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
DES Will Benefit from
Uniform Rate Setting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Finding II: DES Needs to Better
Manage Its Use of Consultants . . . . . . . . . . . . . . . . . . . . . . . 17
DES' Use of Consultants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
DES Needs to Better Manage
Its Consulting Contracts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
Agency Response
Tables
Table 1 Summary of DES Human Service Contracts
by Division for the Year Ended June 30, 1994
( Unaudited) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
Table 2 DES Administration for Children, Youth,
and Families Rate Ranges for
Residential Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Table 3 Cost of DES Consulting Contracts
for the Year Ended June 30,1994
( Unaudited) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
iii
INTRODUCTION AND BACKGROUND
The Office of the Auditor General has conducted a performance audit of the Arizona
Department of Economic Security ( DES) pursuant to a May 5, 1993, resolution of the
Joint Legislative Audit Committee. This performance audit, the final in a series of six on
the Department, was conducted as part of the sunset review set forth in Arizona Revised
Statutes ( A. R. S.) 5541- 2951 through 41- 2957. In accordance with the authorizing
resolution, this audit examines an agencywide function within DES. In contrast, prior
audits in the series were specific to single DES divisions or programs.
This audit examined DES contracting practices for two specific types of contracts:
contracts with agencies providing human services, and contracts for consultant services.
These two functions are a subset of DES' total contracting responsibilities, which also
include day care services, contracts with foster care families, and DES contracts for
supplies, equipment, and business services.
DES is a huge contractor of services. In fact, it spends approximately $ 243 million
annually on human service contracts alone - an amount greater than the budgets of 90
percent of all other state agencies. The effectiveness of DES contracting practices is
important not only because of the significant dollars involved, but also because it
impacts the care received by the many DES populations served. These populations
include some of the most vulnerable groups served by state government; for example,
children under state protective care, as well as persons with developmental disabilities.
The DES processes for setting contractor rates are crucial in human service contracting
because rates are established entirely by DES and the provider. In contrast, rates paid
by DES for medical, dental, and day care services are influenced by ceilings or guidelines
established by the Arizona Health Care Cost Containment System or the federal
government.
Table 1 ( see page 2) estimates the DES human service contract expenditures by division
for state fiscal year 1993- 94. Examples of contracted services include residential services
for children in state care; residential, habilitation, and vocational services for persons
with disabilities; and day programs for the aged. The table does not include medical,
dental, day care, or foster care family services.
Table 1
Summary of DES Human Service Contracts by Division
for the Year Ended June 30, 1994
( Unaudited)
Division
Developmental Disabilities
Aging and Community
Services
Children and Family Services
Employment and Rehabilitation
Services
Administrative Support ( b)
Benefits & Medical Eligibility
Child Support Enforcement
DES Total
Number of
Providers Expenditures
Percent of
Total
( a) The number of providers for the Rehabilitation Service Administration within the Division of
Employment and Rehabilitation Services was obtained from the Division's Integrated Rehabilitation
Information System, rather than the Financial and Management Control System.
( b) Administrative Support includes multiple administrative divisions as well as DES advisory councils
and the Director's Office.
Source: DES' Financial and Management Control System.
Organization of DES'
Contracting Function
In general, DES contracting responsibilities are decentralized to the divisions for both
human service and consulting service contracts. However, there are several centralized
contracting functions, presented below.
The Purchasing Section under the Division of Business and Finance has agencywide
responsibility for the purchase of equipment and supplies.
The Contracts Management Section ( CMS) under the Division of Business and
Finance has agencywide responsibility for assuring procurement code compliance and
for training contract personnel.
The Office of Policy, Planning, and Project Control ( OPPPC) under the Director is
responsible for centralized management of consulting projects designated by the
Director as having critical agencywide importance.
Likewise, the divisions retain some contracting functions within division administration
and decentralize other responsibilities to the district organizations across the State. For
example, contracts between the Division of Developmental Disabilities ( DDD) and their
largest human service providers are developed and negotiated centrally, while contracts
with smaller providers are managed by the districts. Similarly, the Administration of
Children, Youth, and Families ( ACYF) centralized the 1994- 95 negotiation of contracts
for residential living and development services that had previously been the
responsibility of the districts. Other ACYF provider contracts are the responsibility of the
district administrations.
Consulting contracts, other than those managed centrally by OPPPC, are initiated and
managed by the divisions. Most division consulting contracts are administered centrally
by division administration rather than by district administration.
As a result of having centralized, division, and district contracting functions, DES
employs contract personnel in all these organizational units.
Audit Scope and Methodology
Our review of DES human service contracting focused on three areas within DES: the
DDD, the ACYF, and the Rehabilitation Services Administration ( RSA). These three DES
organizations together account for almost 80 percent of the total estimated DES
expenditures to agencies providing human services. ( The total excludes medical, dental,
day care, and foster care family services).
To evaluate human service contracting, we employed the following audit methodologies:
Analysis of rates and expenditures for DDD, ACYF, and RSA using data from three
different DES computer systems
Contract file reviews in ACYF and DDD
Review and analysis of data generated in ACYF's contract negotiation for the most
recent two- year contract term beginning July 1, 1994
Extensive surveys of multiple agencies in 11 states
Review of internal DES contracting studies
Interviews with DES management, contracting staff, and providers
In our second audit area, which examined DES' use of consultants, the audit scope was
agencywide. However, we excluded automation contracts. To evaluate the use of
consultants, we conducted an in- depth evaluation of 14 consulting contracts active in
fiscal year 1993- 94.
Audit limitations - We confronted several limitations due to the fact that DES' central
financial information system does not directly link payments to individual contracts.
Additionally, we found certain data essential for comprehensive rate analysis was
inaccurate.
First, DES automated systems do not directly link payments to contracts. We attempted
to obtain contract expenditure data from DES' central financial system and subsystems
for three divisions. With the exception of the information system used by RSA, none of
these systems directly link payments to the associated contracts. Thus, while information
could be obtained on contracted rates and total payments to providers, without manually
matching each payment voucher to the information system, we could not determine the
total dollars paid for a given service or whether the rates paid corresponded to the
contracted rate. Payments can be made from seven different DES systems and
agencywide payment detail is not housed in a single system. The agencywide financial
system ( FMCS) was designed to link payments to contracts, collecting service type, units,
and contract number for each payment; but DES does not use this application and
therefore lacks critical information on individual contracts.
Second, DES cannot produce comprehensive data on how many units it purchases of
each type of service. DES has no central tracking of such data and has not ensured that
the division data is collected or accurate. Of the three divisions we examined, DDD and
RSA have automated data on the rate paid and number of units purchased for each
service category. ACYF, however, has not enforced accurate collection of service codes
and units for its expenditures. While such data is entered into the ACYF data system,
both DES staff and our review of the data raised serious questions regarding data
integrity. The combination of data inaccuracy and data system deficiencies prevented
important audit analyses; for example, we were unable to project the financial impact
of implementing a uniform rate structure as recommended in Finding I.
The audit was conducted in accordance with government auditing standards.
The Auditor General and staff express appreciation to the Director and staff of the
Arizona Department of Economic Security for their cooperation and assistance during
the audit.
FINDING I
DES NEEDS TO ADOPT A MORE
EQUITABLE RATESETTING METHODOLOGY
The rates DES pays its human service providers are often inequitable. Rates vary widely
for similar services and are not tied to quality. DES should abandon its negotiated rate
I approach and adopt a uniform rate structure. States using uniform rate setting have
found it easier to administer and more equitable.
DES' Current Rate- Setting
Method Leads to Inequities
Despite both legislative and provider concern, DES pays providers very different rates
for similar services. Because the current rate- setting method generates many unique rates
for the same service, inequities result. Additionally, the rate- setting method can have
negative impacts on the quality and cost of services, and DES' relations with its
providers.
Rate inequities exist - DES has wide rate ranges for similar services within both DDD
and ACYF. Our analysis found rates are not tied to service quality and DES often pays
higher quality providers less than lower quality providers.
A more equitable basis for funding services is needed. In DDD, for example, for group
homes serving persons with developmental disabilities, rates range from $ 56.70 per day
to $ 123.95 per day even when the highest and lowest rates are excluded.(') There are a
total of 109 rates for this service. Wide rate ranges exist for other DDD services as well.
For example, there are 107 different rates for adult day treatment and training services
ranging from $ 1.96 to $ 16.03 per hour. While client differences account for a portion of
the rate variation, our review suggests many differences among rates may be unjustified.
In 1994, concern over rate differences led the Legislature to mandate that DES establish
a fair rate structure for DDD services. A. R. S. 536- 557. K directs DDD to:
" establish a rate structure that ensures an equitable funding basis for private non- profit
and for profit agencies. In fiscal year 1995- 1996 and each fiscal year thereafter, the
division shall revim and may adjust the rate structure."
(') In order to exclude rates for exceptional cases, we eliminated the highest and lowest 6 percent of the I rates based on the actual units of service purchased by DES.
However, wide rate ranges also exist within ACYF. In fact, for residential services
presented in Table 2, one provider has rates twice the amount of another provider. Table
2 illustrates some of the ACYF services where rates vary widely. For example, rates
range from $ 76 to $ 179 per day for therapeutic group home service.
Table 2
DES Administration for Children, Youth, and Families
Rate Ranqes for Residential Services
Number of
Service Description Unique Rates Lowest Rate Hiqhest Rate
Therapeutic Group Home 26 $ 76.00/ Day $ 179.00/ Day
Group Home House
Parent Model
Group Home Staff Model 37 $ 60.00/ Day $ 149.82/ Day
Source: Auditor General analysis of DES rates for the 22- month contract term ending June 30, 1996.
The wide rate differences cannot be explained by DES' evaluations and ratings of
provider attributes.(') Surprisingly, we found lziglrer rated providers are actzdally paid
less than lower rated providers.( 2) W e analyzed the rates of 40 providers of ACYF group
home staff model services. Of the 40, 8 of the 10 with the highest evaluation scores had
below average rates. At the same time, 6 providers from among the 10 rated lowest of
the 40 had above average rates.
' To evaluate provider quality, we reviewed DES' evaluations of the providers' contract proposals. We
combined the points assigned for methodology and experience on 1994- 95 proposal evaluations scored
by DES staff. The points assigned for experience and methodology include factors such as expertise
of staff, appropriateness of treatment methods, and client outcomes.
We computed correlation coefficients relating the provider's evaluation points to the contracted rate
for the service for five residential service categories. We found a statistically significant inverse
relationship between evaluation scores and rates for the combined total population of residential
services. In specific service categories, we also found a statistically significant inverse relationship
between evaluation scores and rates for the staff model and parent model group homes. For
residential treatment centers, therapeutic group homes, and the independent living model homes, we
found no statistically significant relationship between rate and high evaluation scores.
Similarly, an analysis of ACYF therapeutic group homes also shows higher rated
providers may receive lower rates than lower rated providers, as illustrated in the
following examples from our analysis.
H Seven of ten providers with the highest evaluation scores had below average rates.
H The therapeutic group home provider with the highest evaluation score had below
average rates.
H Another highly rated home received no rate increase during state fiscal year 1995
despite having rates 20 percent below the average for residential treatment services
( the majority of providers received rate increases averaging 18 percent for residential
treatment services).
Meanwhile, other providers with lower evaluation scores received rate increases. The
following case example clearly illustrates that rates are not related to DES' evaluations
of provider attributes.
H During the State's last contracting cycle, data indicated that one ACYF group home
had the fifth lowest rating out of 40 providers. Further, it appears the provider
submitted incorrect information regarding its proposed rates - the costs submitted
by the provider were inflated 82 percent above the costs found in its year- end
financial statements. However, the provider received an increase of nearly 12 percent,
raising its rates 26 percent above the average rate paid to other providers for the
same service.
DES staff and providers alike claim that rate inequities exist. High- level DES
administrators, program managers, and contracting staff expressed the opinion that DES'
current rate- setting methodology generates rate inequities. Additionally, providers we
interviewed claimed that the DES contracting system is inequitable. The following is an
example of the type of inequitable rate decisions that frustrate providers.
In 1994, the Legislature appropriated $ 453,000 for fiscal year 1995 to DDD to increase
the rates of its providers. According to a DES schedule dated June 4, 1995, $ 240,574
was paid to two large Maricopa County providers. The remaining $ 222,514 was
divided among 22 other providers. In a letter to the Legislature explaining its rate
increase decisions, DES reported it used the money to raise rates that were below the
mean. However, the rate of one of the two providers receiving large increases was
already above the mean. Other DDD providers, who had gone years without rate
increases and had rates below the mean, received no monies from the appropriation.
Why current methodology does not work - The current rate- setting method is a
fundamental contributor to rate inequity. The basic assumptions underlying the rate-setting
method are not accurate and, as a result, the process does not work well.
In general, DES sets provider rates after analyzing the provider's proposed cost for each
individual service. These proposed costs and the provider's historical rates serve as the
basis for rate negotiations. DES' current rate- setting method can be best described as a
form of " cost- based, negotiated rate setting." However, the assumptions underlying this
method are:
DES has complete specifications for each desired service,
DES has the time and expertise to analyze and identify the true costs of each
provider's services,
There is adequate competition among providers to: 1) create an incentive to operate
efficiently and at the lowest costs, and 2) allow DES not to contract with a provider
whose costs may be too high.
In reality, these assumptions are invalid for the following reasons.
DES does not have detailed specifications for each service it purchases.
DES staff cannot adequately identify provider costs. Many of the 172 DES staff
involved in the process across the State have limited financial backgrounds and are
not trained in analyzing costs nor provided with guidelines for what constitute
acceptable costs. These staff are involved in negotiating over 10,000 separate rates,
and may have to negotiate with the providers' accountants.
Adequate competition does not exist. The demand for many services often meets or
exceeds the supply and DES sometimes has to contract with undesirable providers
regardless of whether their rates are too high. In addition, under the current method
certain established providers have an advantage over others because of their financial
expertise, control of the market ( especially in rural areas and for highly specialized
services), and long- standing relationships with DES staff. This makes market entry
extremely difficult and further limits competition.
Without adequate competition, providers lack an incentive to decrease costs. In fact,
the incentive for providers involved in the current process is to increase rather than
decrease costs. If providers can prove increased costs they can receive higher rates.
Those providers managing their programs more efficiently cannot justify rates as
high as those of other providers.
Because of these conditions, rates are often set subjectively, on a case- by- case basis, and
inequities occur. While some of these problems could be addressed by better training
and the development of cost guidelines, our research suggests the best way to address
them is to adopt a new rate- setting method.
DES Will Benefit from
Uniform Rate Setting
Many of the problems with the current rate- setting method could be solved by using
uniform rates. Uniform rate setting is the current national trend for human service
contracting. States implementing uniform rate setting have eliminated inequity,
improved relations with providers, and are better able to focus on quality. However, it
will take time to implement and should be done in a manner that avoids significant cost
to the State. It must also be done in a manner that reduces, rather than increases,
differences in rates paid to the same providers by DES and other state agencies.
Under uniform rate setting, programs define client- based levels of care and delineate the
specific services required for each level.(') Programs then determine which costs are
acceptable for each type of service based on cost data from all the providers. Uniform
rates are set at or near the median of the costs for each service.
Unifom rate setting in other states - We surveyed 11 states'" and found that 10 had
either implemented or are in the process of implementing uniform rate setting in ACYF,
DDD, or RSA comparable programs. Uniform rates are used for DDD services in 7 of
the 11 states surveyed. Likewise, 7 of the 11 states use uniform rates for ACYF services.
States use the inherent flexibility of uniform rate setting to design structures specifically
tailored to their needs. For example, states define levels of care and set rates differently.
Some states accounted for geographical differences between their urban and rural areas.
The following case examples display the flexibility of uniform rate setting.
Levels of care - States define levels of care differently. For example, California
developed four levels of care for community care facilities providing DDD services,
while Utah has five tiers of care for most DDD services. Michigan and Missouri
created four and three levels of care, respectively. States have also defined different
For example, if DDD established five different levels of care for its clients, the first level would apply
to DDD's most functional clients, and level five would apply to the most severely disabled clients.
(" We selected six states because of their proximity and similar demographics to Arizona ( Colorado,
Idaho, Nevada, New Mexico, Texas, and Utah). Five additional states were identified by national
experts, such as the U. S. General Accounting Office, the U. S. Department of Health and Human
Services, and the Child Welfare League of America, as having the best human service contracting
practices ( California, Illinois, Michigan, Minnesota, and Missouri).
levels of care for ACYF services. Currently, DES has hundreds of rates for these same
services.
Maximum rates - States set rates differently. For example, California set rates for
DDD services at the 65th percentile of aggregate costs for services. Most other states,
including Utah, Missouri, and Michigan, set rates at or near the median ( 50th
percentile) for ACYF and/ or DDD services.
Regional differences - States established rates for their various geographical
locations. For example, California pays different set rates to Los Angeles County and
San Francisco County. Services cost more in San Francisco County. Utah has a set
rate schedule by county as well. As a result, Utah and California account for the
economic differences between their urban and rural areas. In the DES rate analysis
we found that current rate differences in ACYF are not related to the provider's
geographic location.
Benefits of uniform rates - States using uniform rate setting derived many benefits,
including improved relations with providers, enhanced competition, and a better focus
on quality.
Improved relations with provider community - Prior to uniform rate setting,
providers in other states were upset and, in some cases, undertook legal actions
against the state. In most instances, their legislative bodies mandated that their
human service agencies address rate inequities. Uniform rate setting eliminated many
perennial problems between the provider communities and social service agencies in
other states because the inequities and unfairness associated with their systems
disappeared.
Improved competition - States also reported enhanced competition because
providers understand that " an even playing field" exists for everyone. Providers can
decide whether to go into business based on a known rate. Utah indicated that
competition has increased as a result of uniform rate setting because the approach
diminished control over the market held by larger and well- established providers.
Additionally, in contrast to DES' current rate- setting methodology, uniform rate
setting provides incentives for providers to control their costs and implement
innovative practices. Other states with uniform rates, such as California and Utah,
found that providers manage their programs more efficiently in order to remain
competitive. A director within Utah's Department of Human Services noted that
providers must now concentrate on controlling costs and improving quality rather
than on cultivating political connections.
Promotes focus on quality rather than costs - Uniform rate setting permits other
states to concentrate on provider performance and client outcomes rather than on
provider financial data. For example, California, New Mexico, and Utah all noted that
uniform rate setting allows their staff to spend more time on quality concerns and
less time on paperwork. Likewise, DES could redirect some of the significant
resources it now dedicates to provider rate analysis and negotiation. Some of these
staff could be used to develop systems to measure and reward provider quality.
Provides groundwork for the eventual use of performance- based contracting -
Once DES establishes fair and equitable rates for services and ensures that all
providers are on an even " playing field," the Agency can move toward performance-based
contracting. Performance or incentive- based contracting ties a provider's
compensation to client outcomes rather than solely on the provision of services.
Under performance contracting, DES can define desired client outcomes and design
methods to measure provider achievement of the outcomes. Those providers who
reach or exceed benchmarks can be rewarded with higher rates and more client
referrals. According to the State Procurement Office, performance- based contracting
is already allowed under state law. However, DES would have to work with State
Procurement in developing specific contract provisions.
Implementing unifomz rate setting at DES - Although significant challenges exist, DES
can successfully implement uniform rate setting agencywide. Implementation will take
time, require moderate expenditures, and must be designed to fit Arizona's specific
needs. However, by investing in uniform rate setting, DES will realize long- term
benefits.
Implementing uniform rate setting is a step- by- step process requiring time and effort.
Initially, each DES program will need to define levels of care for their clients and then
establish fair rates for each level. These rates may then need to be adjusted for regional
differences in costs. DES states it may also need to adjust the rates to compensate new
vendors for start- up costs which may exceed the uniform rate. However, not all states
make this adjustment.
This whole process of implementing uniform rates will take longer for DDD than other
DES programs because of the diverse nature of its client population. Programs in other
states took from three to five years to completely implement uniform rate structures.
DES would, however, have some internal guidance available in implementing this
system. DES' Rehabilitation Services Administration has successfully used this method
of rate setting for years.
Costs to establish a uniform rate structure were generally moderate for other states. For
example, Utah and Michigan experienced minimal implementation costs because they
used existing staff resources to define service levels and rates. Utah used four FTEs for
four years. California's costs were more substantial since the State used consultants to
help set rates and hired permanent rate- setting staff. These states noted that costs were
compensated for by long- term reductions in staff hours needed for reviewing providers'
budgets under their old systems. For example, Utah now needs only one FTE to
maintain its set rate structure for human services.
Costs to the State - Uniform rate setting will impact the rates of most providers. The
rates of providers below a set rate will be raised while the rates of providers above a set
rate will be reduced. DES and the Legislature will need to determine whether providers
with rates above the set rate will be reduced to the set rate immediately or will be given
time to prepare for rate decreases.
Other states allowed providers varying lengths of time to prepare for rate cuts. For
example, New Mexico immediately reduced rates while Utah continued to pay the
higher rates to some providers for a year.
Finally, while this system could be implemented so as to be cost neutral to the State,
uniform rate setting is not guaranteed to relieve rate pressures in Arizona. We were
unable to determine whether DES currently has funding to adequately compensate
providers. If DES' budget cannot support adequate rates under the current system,
moving to a uniform rate system will not relieve pressures from existing providers, and
new providers will be unlikely to enter the market.
Impacts on other agencies' rates - Uniform rate setting in DES will impact Arizona's
Single Purchase of Care ( SPOC) effort. Currently, DES, the Department of Health
Services, the Arizona Health Care Cost Containment System, the Department of
Education, and the Department of Youth Treatment and Rehabilitation are working
together to jointly purchase licensed behavioral health services. In 1993, the Governor's
Action Council for Behavioral Health recommended that these agencies develop
interagency agreements in order to avoid duplicating contracting functions for children's
behavioral health services. The first jointly issued request for proposal is expected to be
published in January of 1996 for contracts taking effect in July 1996.
Under SPOC, all five state agencies will generally pay the same rate to a provider. Thus,
any uniform rates adopted by DES for behavioral health services need to be adopted by
the other SPOC agencies. SPOC representatives we spoke with support a more
comparable rate for similar services.
Additionally, there are some residential and other services used by multiple state
agencies that are not currently guided by the SPOC effort. Thus, as DES establishes
uniform rates, continuing multi- agency efforts are needed to ensure that other agencies
do not undermine DES' efforts by offering significantly higher rates to the same
providers for similar services.
RECOMMENDATIONS
1. Although it may take several years to implement, DES should begin now to develop
uniform rate setting for human service contracting. ACYF could be expected to
implement the uniform approach sooner than DDD.
2. After implementing uniform rates, DES should next consider moving to performance-based
contracting.
3. While the new rate structure is being designed, DES should allocate any new
appropriations for provider increases in a more objective, justifiable manner. If
possible, DES should take into account provider quality, the provider's rate in
relation to the average rate for the service, and the provider's length of time without
an increase in comparison to other providers.
FINDING II
DES NEEDS TO BETTER MANAGE
ITS USE OF CONSULTANTS
Because DES frequently uses consultants, it must ensure consulting contracts are well-managed.
We found DES can better manage consultant projects to ensure consultant
expenditures result in benefit to the State.
DES' Use of Consultants
DES contracted over $ 4 million to more than 110 consultants in fiscal year 1993- 94.
However, since many of these contracts span multiple- year terms, the costs are actually
far higher. In fact, DES committed over $ 17 million to these same contracts over multiple
years through June 30, 1994. Table 3 ( see page 18) summarizes DES' financial
commitment to consulting contracts in fiscal year 1993- 94.
Consultants perform a wide variety of tasks throughout DES, ranging from automation
system design to policy manual development and contract negotiation. In fiscal year
1993- 94, the Division of Children and Family Services ( DCFS) and the Division of
Developmental Disabilities ( DDD) used consultant services most extensively. In DDD,
consultants were used to develop new licensing standards, conduct training, and
negotiate provider contracts. ACYF also was engaged in a variety of consulting projects,
one of which is intended to achieve complete redesign of the child welfare system.
Large, multi- year projects include the Federal Revenue Maximization contract, the Early
Prevention, Diagnosis, and Treatment ( EPSDT) program design contract, and the Total
Quality Management contract; all managed by the Division of Administrative Services
with approximately $ 1.5, $ 1.3, and $ 3 million, respectively, committed over multiple
years through June 30, 1994.
Division
Table 3
Cost of DES Consulting Contracts
for the Year Ended June 30,1994
( Unaudited)
Children and Family Services
Developmental Disabilities
Administrative Services
Business and ~ inance(")
Data Administration
Aging and Community Services
Child Support Enforcement
Benefits and Medical Eligibility
Employment and Rehabilitation
Services
DES Totals
Number of Active
Consulting Contracts
Amount
Encumbered
in 1993- 1994
( 3 The Business and Finance Division administers two automation contracts also utilized by the Division
of Data Administration.
Source: Contract listings from the Contract Management Section's database and the Purchasing
Section.
DES Needs to Better Manage
Its Consulting Contracts
While there are many appropriate uses of consultants in DES, state dollars are wasted
when costly consultants perform routine tasks and when expensive projects do not yield
results. Therefore, DES should develop mechanisms to ensure consultants are necessary
and that consultant projects are successful.
We conducted an in- depth evaluation of 14 DES consulting contracts active in fiscal year
1993- 94 that encompassed 5 program areas and an agencywide project. Due to the
complexity of evaluating automation system design, automation contracts were excluded
from our sample. Ten of the 14 were selected because they were costly nonautomation
contracts managed by different DES divisions. The remaining four were selected because
our initial review raised questions regarding the contract's appropriateness. For example,
one of the four appeared to duplicate the work of another consultant.
Poor use of consultants is costly to the State - DES retains high- priced consultants to
perform routine tasks better suited to internal staff. In 5 of the 14 contracts consultants
performed tasks requiring no specialized expertise - tasks that many categories of DES
staff are qualified to perform. In fact, we found consultants writing ad hoc reports and
developing policy, tasks that organizations typically can accomplish internally. For
example, in DDD an actuarial firm ( first hired to perform actuarial work) wrote policy
manuals and procedures for case managers. DES paid the consultant $ 150 an hour to
perform those tasks.
DES also appears to use consultants as replacements for management. While these
persons often perform crucial functions for the Department, they do so at costs that may
far exceed the cost of work performed by DES staff. In 5 of the 14 contracts examined
there was a long- term reliance on consultants who made managerial decisions. Examples
of consultants in managerial roles are presented below.
One consultant managed a DES unit for six years. The consultant was hired by DDD
in 1989 to run the Managed Care Unit and was still managing the unit as of June
1995. The consultant outlasted five different unit directors and appears to be
indispensable for her knowledge of the provider community and her historical
perspective. Currently, the consultant writes policy and procedures, creates and
analyzes statistical reports, advises the managed care director, and develops the
provider network. The services are currently billed at $ 90 per hour. In contrast, the
hourly wage of a comparable DES manager is approximately $ 27. ( DDD use of the
consultant varies from year to year. For example, in fiscal year 1993- 94, DDD paid
the consultant over $ 80,000; while for the first 9 months of fiscal year 1994- 95 DDD
has paid the consultant approximately $ 25,000.)
Another consultant managed DES staff in redesigning licensing rules for providers
serving persons with developmental disabilities. DES hired the consultant because
all other project managers were busy with other assignments. In this case, the
consultant services cost over $ 136,000 - three times the cost of a DES project
manager.
DES managers told us that consultants are used either because staff does not have the
necessary expertise to perform a function, or because there is simply no one in DES who
has the time available to perform the function. Since the scope of our audit was limited
to contracting, we were unable to evaluate whether DES has the managerial capacity to
effectively perform the breadth of functions for which it is responsible. However, using
consultants is a costly alternative to using internal resources.
It is also costly when DES fails to manage projects and implement results. In one
contract we evaluated, no one in DES managed the project or was accountable for its
success or failure. DES spent $ 1.07 million with little apparent result as is detailed below.
ACYF's contract with a " big six accounting firm" - For much of the term of this
$ 1.1 million contract, there was no DES project manager, nor any individual in
management designated with this responsibility. The contract had three primary
projects. Two of the three projects yielded limited benefits to the State.
In one project begun in November 1993, the consultant was assigned to complete a
management control review. However, no significant changes have occurred in ACYF
operations due to the contract. Interviews with DES staff and a memo to DES from
the consultant revealed there was little oversight of the project by DES and DES did
not fulfill its commitment to provide staff for the project. ACYF management
changed three times and division management two times during the course of the
project, contributing to the lack of attention paid to it.
In February 1994, DES engaged the consultant under the same contract to provide
assistance to ACYF in negotiating provider rates for a 22- month contract period
ending June 30, 1996. However, the purpose of this $ 225,000 project was not clearly
specified in the contract file. According to DES management the project was initiated
to centralize negotiations and to train DES employees in negotiating methods. While
the project did centralize negotiations under the direction of the consultants, the
consultants excluded DES staff from some of the critical points of the negotiations,
reducing the training of DES employees that took place. DES executive management
said that when it became aware that DES employees had been excluded from the
negotiations, it intervened to correct the problem. However, much of the opportunity
to train DES employees was lost.
Multiple weaknesses in consultant manugement - Our review revealed significant
weaknesses in DES processes to manage consultants. First, DES has no mechanism to
assess whether a consultant is necessary. Second, contract language is often vague, with
inadequate description of project objectives and deliverables. And third, in some cases,
no one is held accountable for project outcome. While DES has a substantial number of
contract management personnel, they are not utilized fully to develop effective
consultant contracts.
DES does not have policies governing the use of consultants nor formal guidelines to
evaluate project necessity. The absence of such policies and guidelines can lead to high
consultant costs for performance of tasks that DES staff can perform, or the initiation of
projects with marginal value and few benefits. We interviewed management and contract
personnel in 4 DES divisions regarding 14 consultant contracts. In only one case did the
division evaluate DES internal resources prior to hiring a consultant. However, in 5 of
the 14 projects, we were told that a consultant was used because DES did not have
sufficient internal resources.
DES contract language needs to clearly describe project tasks and due dates. In 7 of 14
contracts evaluated we found vague language. Vague language allows projects to drift,
enabling DES to use consultants for routine tasks; and also contributes to DES projects
that do not yield desired results. The Administrator of the State Procurement Office
noted that vague contract language leaves the State indefensible in the event of a breach
of contract. For example, in one DES contract, the consultant was directed to provide
technical assistance " in certain perplexing situations." However, DES failed to define
" perplexing situations." In another contract, the consultant was directed to " upon
request from the division provide any other actuarial or consulting services." Finally, in
the ACYF contract cited earlier, the contract did not clearly specify the purpose of the
project.
DES can better use existing contract staff to strengthen management of consultant
contracts. DES employs 68 contract management personnel who each earn an average
of over $ 29,800 per year. While many of these resources are dedicated to provider
contracts ( discussed in Finding I), most divisions have contract personnel in their central
administrations. Yet the contract specialists responsible for division consulting contracts
are not fully utilized to develop effective contract language. They have no authority to
ensure substantive contract quality. We found contract specialists simply assembling
contracts from boilerplate forms, the requests for proposals, and the accepted proposals.
Strengthening the role of contract management personnel would help establish more
accountability for project management, results, and implementation.
RECOMMENDATIONS
1. DES should develop guidelines governing the use of consultants to help ensure
project necessity, and help determine when to use consultants over existing staff
resources.
2. DES should give contract management specialists responsibility and authority to
ensure well- written contracts that include:
Clearly defined scope of work
w Well- defined consultant and DES tasks
Clearly defined deliverables
Time schedules
Implementation strategy
3. The Department should ensure that project documentation clearly specifies the DES
staff positions that are responsible for project management, results, and implementa-tion.
I Fife Symington
Governor
Linda J. Blessing, DPA
Director
November 3,1995
Mr. Douglas R. Norton, CPA
Auditor General
Office of the Auditor General
2910 North 44th Street, Suite 410
Phoenix, Arizona 85004
Dear Mr. Norton:
Thank you for the opportunity to respond to the Contracting Practices performance audit.
Enclosed is the department's detailed response.
The department concurs, in concept, with the recommendations related to establishing a
uniform rate- setting methodology and performance- based contracting approach.
However, our research indicates that uniform rate- setting may not remedy the issues
associated with the current negotiated rate methodology. Moreover, uniform rate- setting
may not necessarily improve provider relations, enhance competition, or relieve rate
pressures in Arizona.
The department embraces the point made in the report that provider increases should be
equitably allocated. The department disagrees, however, with the report's conclusion that
the fiscal year 1995 provider increase was inequitably allocated. The fiscal year 1995
provider increase was administered in a fashion that ensured continuity of services to
clients. In the future, the department will maintain better documentation of how provider
increases are allocated.
The department disagrees with the report's recommendation that calls for the
establishment of additional guidelines and processing steps to govern the use of
consultants. Currently, Assistant Directors have the authority, responsibility and
accountability for determining the need for consultant services. The establishment of
additional guidelines is unnecessarily burdensome and bureaucratic.
The department is in accord with the recommendation that the contract management
specialist positions should be reviewed to determine their ideal scope of authority and
responsibility. Additionally, the department agrees with the recommendation that project
documentation should routinely specify those positions responsible for project
management.
Finally, the department wishes to express its appreciation to you and your staff for the
time and effort invested in this critically important audit.
Sincerely,
DEPARTMENT OF ECONOMIC SECURITY
RESPONSE
TO THE CONTRACTING PRACTICES PERFORMANCE AUDIT
Finding I: DES Needs to Adopt a More Equitable Rate- Setting Methodology
, The Department supports, in concept, recommendations 1 and 2 of the report. These
recommendations state that the Department should implement a uniform rate- setting approach
for human services and after implementation, the Department should consider moving to
performance- based contracting. Uniform rate setting would stabilize provider rates for a given
service, and in the long run, the benefits listed in the report could be achieved. In fact, the
Division of Developmental Disabilities is currently working on a uniform rate proposal.
However, the Department has several concerns with uniform rate setting that it would like to
bring to the attention of the Legislature. The Department surveyed a number of other states
including: Colorado, Idaho, Nevada, New Mexico, Utah, Illinois and Michigan. The results
of the survey indicated that the audit report over- simplified the standard rate- setting process as
follows: . The report states that DES currently may have hundreds of rates for a given service.
Uniform rate setting as proposed in the audit report may not remedy that situation.
For example, one service could have one rate for each of the five levels of service in
each of the State's six districts. Therefore, the service could have up to thirty different
uniform rates. Since the rate is actually a not- to- exceed cap, providers would still be
required to submit proposals. Each proposal could offer the Department a different
rate; and consequently, the Department could still have over one hundred rates for a
given service.
This situation is similar to that experienced by the Utah Department of Human Services
and the Michigan Office of Foster Care Management. Both of these entities have
established standard rates and can have over one hundred rates for services. . The audit report indicates that DES could expect improved relations with the provider
community as a result of implementing uniform rates.
The Department believes that relationships with the providers would improve for one
segment of the provider community - those providers whose rates are below what will
be the uniform rate. Uniform rate setting will divide providers into two groups: those
whose rates are currently below the uniform rate and those providers whose rates are
currently above the uniform rate. . The audit report indicates that DES could expect improved competition because
providers would understand that an " even playing field" exists for everyone.
A uniform rate could force a number of providers out of the market. The full cost of
doing business may not be reimbursed by the uniform rate and, therefore, it might not
be economically viable to continue to do business with DES. Also, the uniform rate
may not be sufficient to attract new providers. Since demand already exceeds supply,
further reductions in supply would negatively impact the Department. The audit report
could be substantially improved if the report's recommendations included a provision
for providing a stipend, or some other mechanism to finance provider start- up costs.
Without a mechanism to finance provider start- up costs the market could soon be
dominated by those entities who could secure sufficient capital to acquire office and
direct service space and equipment.
DEPARTMENT OF ECONOMIC SECURITY
RESPONSE
TO THE CONTRACTING PRACTICES PERFORMANCE AUDIT
In fact, stipends are used to finance start- up costs for new service delivery facilities at
the Illinois Department of Mental Health and Developmental Disabilities. Illinois is
one of the states the audit report identifies as having the best human service contracting
practices. . The report states that implementation of uniform rates could be accomplished with a
" moderate" level of expenditures with virtually no maintenance costs.
The Department discussed implementation and maintenance costs with a number of
other states. Five states could not even provide the Department with a cost estimate for
the original design and implementation phases of uniform rate setting. Michigan,
California and New Mexico used consultants to establish uniform rates. New Mexico
had the most detailed cost information. For ten treatmentlservice components in it's
Community Based Services programs with a total budget of $ 10 million, New Mexico
spent $ 125,000 developing uniform rates. Since DES provides approximately 140
human services with a budget in excess of $ 300 million, it is reasonable to expect a
substantially larger investment will be required by DES.
Michigan and New Mexico estimate their annual expenditures at $ 100,000 and
$ 344,000 respectively for maintenance of the uniform rate system.
Additionally, the audit report states that a uniform rate- setting system " . . . could be
implemented so as to be cost neutral for the State." This assertion contradicts a
statement earlier in the report that says "... we were unable to project the financial
impact of implementing a uniform rate structure.. . " Accordingly, the report should not
lead readers to believe that standard rate setting is cost neutral when an analysis has not
been performed.
Recommendation 3 of Finding I states that " . . . DES should allocate any new appropriations for
provider increases in a more objective, justifiable manner." The Department concurs, in
concept, with this recommendation and will continue to strive to allocate provider increases
equitably. In the future, the Department will maintain better documentation of how provider
increases are allocated.
However, allocating provider increases as a percentage change in the providers' rates is very
difficult. It cannot be done unless all of the following things are static:
•. the client caseload . the units of service delivered by the provider the mix of services offered by a provider
Oftentimes a provider will contract to deliver a more specialized service that was not provided
in the previous year. Additionally, the client caseload continually increases. Finally, the
amount paid to a given provider is a function of both the provider's and the number of
service y& s delivered to clients. Accordingly, while a provider's rate may have increased by
one percent or more, expenditures to a given provider may have decreased since the number of
service units provided may have decreased. Likewise, expenditures to a given provider may
have increased due to a substantially larger number of service units being delivered.
Therefore, it is virtually impossible to state that a given unit of service, resulting in a certain
dollar expenditure was a part of the one percent provider increase, particularly in light of the
increased demand for services as a result of caseload growth.
DEPARTMENT OF ECONOMIC SECURITY
RESPONSE
TO THE CONTRACTING PRACTICES PERFORMANCE AUDIT
The two contractors who received the reported " large" increases also provide services to large
numbers of clients. In fact, the providers serve well in excess of 500 clients. These providers
also will generate annual total expenditures well in excess of $ 10 million. The $ 240,544 does
not represent a windfall to these providers. The $ 240,574 represents less than 2.5% of these
contractors' total expenditures. Additionally, the report's assertion that only $ 222,514 was
available for the remaining providers is incorrect. The total available for the provider increase
was $ 1,182,000, not $ 453,000 as listed in the report. This amount is easily derived from the
fiscal year 1995 appropriations report.
In allocating funds to these two providers the Division had to, therefore, weigh the numbers of
clients served as well as evidence presented by the providers that they could no longer sustain
the losses being incurred. When combined, these two factors were more than adequate to
justify the increase. Therefore, by relieving some of the financial pressure on these providers,
the Division was able to ensure continuity of services to a large number of clients.
In summary, the Division's allocation of the one percent provider increase should not be
characterized as "... an example of the type of inequitable rate decisions that frustrate
providers. "
Finding 11: DES Needs to Better Manage It's Use of Consultants
The first recommendation for this finding states that " DES should develop guidelines
governing the use of consultants to help ensure project necessity, and help determine when to
use consultants over existing resources. "
The Department does not concur with this recommendation. This recommendation will only
serve to build another bureaucratic process within the Department.
It appears that the basis for this finding is that in 5 of 14 projects the auditors were told that a
consultant was procured because DES did not have sufficient internal resources. It is the
Department's position that no other justification is necessary. The allocation of internal
resources is a management decision. Each Division's Assistant Director is responsible and
held accountable for determining the need for consultants.
The audit report admits that the auditors did not evaluate the Department's managerial capacity
to effectively perform its duties. Since the auditors did not evaluate managerial capacity, it is
safe to assume that they did not evaluate work load, time pressures, or the strengths,
weaknesses and expertise of staff; factors that the Department considers when making
decisions to contract services.
Recommendations 2 and 3 state that the Department should give Contract Management
Specialists responsibility and authority to ensure well written contracts and that the Department
should ensure that project documentation clearly identifies the DES staff positions that are
responsible for project management. The Department agrees that the role of the Contract
Management Specialist could be examined to determine if the positions should be given
additional responsibility and authority. The Department also agrees that those positions
responsible for project management should be clearly defined.
DEPARTMENT OF ECONOMIC SECURITY
RESPONSE
TO THE CONTRACTING PRACTICES PERFORMANCE AUDIT
The Department has noted several weaknesses in Finding 11: . The report states that DES contracted for over $ 4 million with consultants in fiscal year
1994. While the amount of money spent on consultants is significant, the report should
indicate that consultant activity is very small in relation to total DES activity. In fiscal
year 1994, the $ 4 million contracted to consultants represents .2 % of all DES activity
totalling over $ 1.7 billion.
The report states that one consultant managed a DES unit for six years. This assertion
is simply not true. The Division of Developmental Disabilities has continually reduced
the billable hours for the consultant in question. In fact, the billable hours for this
consultant were reduced to about 890 hours in fiscal year 1994 and were further
reduced to less than 400 hours for fiscal year 1995. These hours are not sufficient to
manage a DES unit as the report states.
The report states that a consultant was used to redesign licensing rules at a cost three
times greater than that of a DES project manager. What the report fails to state is that
this project was very difficult and very successful. The auditors underestimated the .
difficulty of achieving consensus among advocates, providers and families. The report
also does not state that all deliverables were properly completed.
The report states that consultants are costly alternatives to using internal resources.
The audit report does not recognize several factors. First, the addition of permanent
positions will be more costly in the long run. Also, the Department is appropriated,
professional and outside services funding for various purposes. Finally, the Governor,
the Department and the Legislature are all interested in privatization. This was evident
in the fiscal year 1996 appropriation whereby the Department was appropriated funding
to acquire a consultant for welfare reform evaluation. This appropriation is $ 50,000
and is substantially in excess of the cost for a permanent full- time position.
The report indicates that the ACYF contract with a big six accounting firm yielded
limited benefits, and that part of project required the consultant to train Department
employees in negotiating methods. The report goes on to state that Department staff
were excluded from some of the critical points of the negotiations and therefore much
of the opportunity to train DES employees was lost.
The Department disagrees with these assertions. First, the deliverables from the
contract are included in the design of the Department's CHILDS project and the related
processes. Second, the report should indicate that Department staff were only excluded
from the initial provider negotiations. Department staff participated in all of the effort
to prepare for negotiations and all subsequent negotiation sessions, after the initial
sessions, with providers. Finally, the Department's staff utilized the negotiation
training during fiscal year 1996 contract negotiations.