Performance audit, Department of Revenue, Business Reengineering/Integrated Tax System (BRITS)

              Performance Audit
Department of
Revenue–
Business Reengineering/Integrated
Tax System (BRITS)
Performance Audit Division
Debra K. Davenport
Auditor General
OCTOBER • 2005
REPORT NO. 05 – 15
A REPORT
TO THE
ARIZONA LEGISLATURE
The Auditor General is appointed by the Joint Legislative Audit Committee, a bipartisan committee composed of five senators
and five representatives. Her mission is to provide independent and impartial information and specific recommendations to
improve the operations of state and local government entities. To this end, she provides financial audits and accounting services
to the State and political subdivisions, investigates possible misuse of public monies, and conducts performance audits of
school districts, state agencies, and the programs they administer.
The Joint Legislative Audit Committee
Senator Robert Blendu, Chair Representative Laura Knaperek, Vice Chair
Senator Carolyn Allen Representative Tom Boone
Senator Gabrielle Giffords Representative Ted Downing
Senator John Huppenthal Representative Pete Rios
Senator Harry Mitchell Representative Steve Yarbrough
Senator Ken Bennett (ex-officio) Representative Jim Weiers (ex-officio)
Audit Staff
Melanie Chesney, Director
Dot Reinhard, Manager and Contact Person
Lori Babbitt
Jason Taylor
Copies of the Auditor General’s reports are free.
You may request them by contacting us at:
Office of the Auditor General
2910 N. 44th Street, Suite 410 • Phoenix, AZ 85018 • (602) 553-0333
Additionally, many of our reports can be found in electronic format at:
www.auditorgen.state.az.us
2910 NORTH 44th STREET • SUITE 410 • PHOENIX, ARIZONA 85018 • (602) 553-0333 • FAX (602) 553-0051
DEBRA K. DAVENPORT, CPA
AUDITOR GENERAL
STATE OF ARIZONA
OFFICE OF THE
AUDITOR GENERAL
WILLIAM THOMSON
DEPUTY AUDITOR GENERAL
October 13, 2005
Members of the Arizona Legislature
The Honorable Janet Napolitano, Governor
Mr. Gale Garriott, Director
Department of Revenue
Transmitted herewith is a report of the Auditor General, A Performance Audit of the Department of
Revenue’s Business Reengineering/Integrated Tax System (BRITS). This report is in response to
a November 20, 2002, resolution of the Joint Legislative Audit Committee. The performance audit
was conducted as part of the sunset review process prescribed in Arizona Revised Statutes §41-
2951 et seq. I am also transmitting with this report a copy of the Report Highlights for this audit to
provide a quick summary for your convenience.
As outlined in its response, the Department agrees with all of the findings and plans to implement
all of the recommendations.
My staff and I will be pleased to discuss or clarify items in the report.
This report will be released to the public on October 14, 2005.
Sincerely,
Debbie Davenport
Auditor General
Enclosure
The Office of the Auditor General has conducted a performance audit of the
Department of Revenue’s Business Reengineering/Integrated Tax System (BRITS)
project pursuant to a November 20, 2002, resolution of the Joint Legislative Audit
Committee. This audit was conducted as part of the sunset review process
prescribed in Arizona Revised Statutes (A.R.S.) §41-2951 et seq and is one of a
series of four reports on the Department. The other reports focus on the Audit
Division, the Collections Division, and an analysis of the 12 statutory sunset factors.
Background
The Department of Revenue (Department) is responsible for licensing, processing,
and collecting over $9.8 billion in taxes for the State of Arizona. The Department is
heavily dependent on information technology to perform its functions, but its
technology was largely outdated and consisted of many different stand-alone
systems. For example, the Department’s system for transaction privilege taxes
(better known as sales taxes) was built in 1982 and had reached its technical limits.
In addition, numerous stand-alone databases were developed in isolation, resulting
in the need to input taxpayer information separately and making it difficult for
department staff to quickly view all aspects of a taxpayer’s business with the
Department.
In August 2002, the State entered into a contract with Accenture, a global
management consulting firm, to reengineer its core business processes,
organizational structure, and implement an integrated tax system—BRITS. With
BRITS, taxpayer data will be entered in one place, and all taxpayer information will be
available through this one integrated system. The project is being implemented in
three phases. The first phase involved designing and implementing the main
database where all tax processing activities will occur; developing other components
to improve the Department’s processes, such as tools to help find taxpayers who
have not filed their taxes; and converting transaction privilege and withholding tax
data from the Department’s old system to BRITS. This phase has been implemented
since October 2004. The second phase primarily includes converting corporate
income tax data and is currently in progress. However, according to department
management, the expected completion date has been revised from July 2005 to July
Office of the Auditor General
SUMMARY
page i
2006 to ensure that sufficient time is taken to fully define system requirements,
complete user testing, and minimize operational impacts during peak processing
times. The final phase primarily involves converting individual income tax data.
According to the Department, the scope of this phase has not
been fully defined at this time; therefore, a target completion date
has not been set. As a part of the BRITS project, the Department
received approximately 1,100 desktop and 60 laptop computers
and, to facilitate field collections, 50 handheld computers. The
contract also includes “additional backfill support,” which
appears to be for hiring temporary employees to help maintain
the Department’s productivity during BRITS’ implementation and
for ordering any needed additional technical support from
Accenture staff during implementation. The State’s central
procurement office, Enterprise Procurement Services (EPS), has
overall authority for the contract, but has delegated contract
administration responsibility to the Department.
The contract uses a benefits-sharing funding approach. Under
this approach, Accenture is financing the contract’s up-front
costs and will be paid, with interest, out of increased
enforcement revenue the system is expected to generate.
Enforcement revenue is money the Department receives as a
result of its enforcement functions, such as audit, accounts
receivable, and collections. BRITS is supposed to improve these
activities, thereby generating three specific types of enforcement
revenue (see text box). As provided for by the contract, the
Department uses 85 percent of the additional enforcement
revenue to pay Accenture’s invoices, and retains the other 15
percent. This allocation continues until the total cost of BRITS,
$122.65 million plus an estimated $9-$13 million in interest, is
paid. However, if BRITS has not generated enough additional
enforcement revenue to pay the contract costs by the end of the
contract’s 10-year term, the Department will no longer be liable
for the unpaid costs and will keep 100 percent of the additional
enforcement revenue generated by the system after that time.
Department needs to better manage BRITS project (see
pages 9 through 18)
The Department could do more to ensure that it effectively manages BRITS
implementation and that BRITS will meet its needs. According to the Department, by
January 2004, demographic information for the transaction privilege (commonly
known as sales tax) and withholding tax types had been converted to BRITS.
Information that was converted included taxpayers’ names and registration numbers,
State of Arizona
page ii
BRITS-generated Enforcement Revenue
􀁺􇩄 Discovery revenue—Money the Department
collects from individuals identified by BRITS who
have not filed taxes, or from taxpayer liabilities the
Department was unable to collect before
implementing BRITS. Each dollar of discovery
revenue can be traced to a specific taxpayer and
BRITS’ discovery program.
􀁺􇩌 License compliance baseline revenue—Money
the Department collects because BRITS has
improved the efficiency of its license compliance
function, which involves staff finding businesses
that are unlicensed. License compliance baseline
revenue is measured against a baseline dollar
amount, which is intended to represent license
compliance revenue the Department would have
collected without BRITS.
􀁺􇩅 Efficiency revenue—Basically all enforcement
revenue, other than discovery revenue and license
compliance baseline revenue, that the Department
collects because BRITS has improved its
enforcement activities’ efficiency (i.e., its audit,
accounts receivable, and collections functions).
Efficiency revenue is measured against a baseline
dollar amount, which is intended to represent
enforcement revenue the Department would have
collected without BRITS.
and some accompanying financial information for transaction privilege tax accounts,
such as the amount taxpayers owed to the State. However, in some cases the data
for transaction privilege tax accounts did not convert correctly. Further, due to BRITS’
implementation problems, the Department had to manually review and correct its
transaction privilege tax bills before sending them to the taxpayers, in turn causing a
backlog of bills to be mailed. For example, as of June 30, 2004, approximately 6
months after beginning to process transaction privilege tax returns in BRITS, bills had
not been sent for over 17,000 taxpayer liabilities totaling nearly $45 million.
BRITS’ implementation issues may have been minimized if the Department had hired
an outside advisor as outlined in the BRITS contract. The BRITS contract contains a
clause indicating the Department intended to hire an oversight advisor who was not
affiliated with the project to report on BRITS’ progress, risks, and budget and make
advisory recommendations to the director. Department officials indicated they were
not certain why an outside advisor was not hired, but cost may have been a factor.
However, hiring an outside advisor may have prevented or at least helped identify
and mitigate the BRITS implementation problems. For example, in 2003, Hawaii’s
Department of Taxation hired an outside advisor for approximately $200,000 a year
to provide quality assurance consultant services for its integrated tax system project.
A Hawaii Department of Taxation official who auditors spoke with indicated that using
an outside advisor was helpful and that a benefit of such an advisor is that they can
alert the agency to things that should or should not be occurring on the project. A
BRITS project outside advisor would have been beneficial to the Department.
Interviews with the Department indicated that it did not involve a sufficient level of the
most knowledgeable department staff, such as IT staff familiar with the previous
systems, and the BRITS project has been managed by four different department
project managers since the project began. Even though the Department is taking
steps to more effectively manage the project, it should still consider hiring an outside
advisor to help oversee BRITS’ remaining phases. To do so, the Department would
have to examine possible funding methods such as allocating monies from its
current budget or seeking a separate appropriation from the Legislature.
Whether or not an outside advisor is hired, the Department needs to continue
improving its project management. The Department has taken some steps to
conform with best practices for IT project management. For example, for BRITS’
second phase, the Department is involving a greater number of knowledgeable staff
and working with Accenture to define in more detail what functions the BRITS system
must be able to do. Auditors found, however, that the Department can make
additional improvements in a number of areas, ranging from better planning and
training to more thorough testing of whether functions actually work. For example,
one best practice is to have an overall project plan to help guide the project
management. Although Accenture created an overall project plan to guide its efforts,
the Department has not created or followed any overall plan to guide its own
management of the BRITS implementation.
Office of the Auditor General
page iii
BRITS not generating additional revenue as expected
(see pages 19 through 24)
BRITS has not generated increased revenue as expected, resulting in negative
financial implications for the State. BRITS appears to be generating two of the three
types of enforcement revenue that are measured—discovery revenue and license
compliance baseline revenue. However, BRITS has not been able to consistently
produce the other type of enforcement revenue, efficiency revenue, primarily
because of BRITS’ inability to generate accurate bills for taxpayers with unpaid tax
liabilities. Based on department information, efficiency revenue has been as much as
$27.3 million less than the Department expected to collect even without BRITS, and
was approximately $9.3 million less than expected as of March 2005.1
BRITS’ inability to increase enforcement revenue carries negative financial
implications for the State. Specifically, because contract payments to Accenture are
supposed to be made from increased enforcement revenue, BRITS’ failure to
generate efficiency revenue is hindering the State’s ability to make payments. Since
Accenture charges interest on unpaid contract costs, this translates to additional
interest costs for the State.
The Department and Accenture are taking steps to improve enforcement revenue.
For example, the Department has developed a special team of department and
Accenture staff that is charged with increasing enforcement revenue by improving
existing BRITS discovery programs and creating and implementing new ones. The
Department needs to continue these efforts. The Department should also continue to
include information about low efficiency revenue in its monthly project status report
to the Government Information Technology Agency, which it only began doing in June
2005 in response to auditor recommendations.
Department should better ensure contract changes are
appropriate (see pages 25 through 30)
In an effort to administer the contract under its indefinite-quantity terms, the
Department has made several changes to the contract that may impact its overall
scope. In all, the Department has added nearly $7 million of additional services to the
contract, including $6.4 million for Accenture to provide offsite data center services.
Initially, to keep the total contract price at $122.65 million, the Department decreased
the amount of monies available for other tasks. In particular, the Department primarily
reduced the amount of monies available for a broadly defined task known as
State of Arizona
page iv
1 As of September 8, 2005, the Department was in the process of finalizing efficiency revenue calculations for April through
June 2005. However, department officials indicate that, based on preliminary calculations, they expect to realize enough
efficiency revenue from those months to eliminate the cumulative negative efficiency amount.
“additional backfill support,” which appears to be for the hiring of temporary
employees to help maintain the Department’s productivity during BRITS’
implementation and for ordering additional technical support from Accenture.
Although the Department is trying to control the changes’ impact, its method of doing
so may increase the contract’s price or require the Department to forego products or
services.
Considering the contract changes’ potential impact, it is important that the
Department consult department and state contracting officials. Consulting these
procurement experts is especially important for the BRITS contract because EPS, not
the Department, has overall authority for the contract. However, for the changes the
Department has already made, communication was lacking. For example, one of the
Department’s project managers signed a memorandum from Accenture indicating
she agreed that 5 of 534 original system requirements are not within the contract’s
original, overall scope, but did not discuss the agreement with the Department’s
procurement office or EPS.
It is also important for the Department to thoroughly document contract changes.
This will help ensure that changes that affect the contract’s overall scope are
identified and incorporated into the contract. However, documentation for many
contract changes either did not exist or did not fully explain how the change affected
the scope of the tasks involved.
To its credit, the Department has begun taking steps to improve its communication
and documentation of contract changes, and should continue its efforts. For
example, on May 17, 2005, the Department sent a memorandum to EPS
documenting the contract changes that had already been made and requesting that
EPS increase the overall contract price by approximately $6.4 million, to a new total
of approximately $129 million, to include the data center costs.1
1 If EPS increases the contract price to include the data center as requested by the Department, then the Department would
no longer have to pay for the data center by reducing the amount of monies available for other contract tasks.
Office of the Auditor General
page v
State of Arizona
page vi
Office of the Auditor General
TABLE OF CONTENTS
continued
page vii
1
9
9
11
12
18
19
19
20
23
24
25
25
28
30
Introduction & Background
Finding 1: Department needs to better manage BRITS
project
BRITS’ problems affecting Department’s ability to perform some
basic functions
Department did not initially involve enough key people
Department can do more to follow best practices
Recommendations
Finding 2: BRITS not generating additional revenue as
expected
BRITS increasing some enforcement revenue types
BRITS not consistently generating one revenue type
Department should continue to provide detail on enforcement
revenue
Recommendations
Finding 3: Department should better ensure contract
changes are appropriate
Contract changes may affect BRITS’ overall scope
Better disclosure and documentation of contract changes needed
Recommendations
Agency Response
State of Arizona
TABLE OF CONTENTS
concluded
page viii
Table:
1 BRITS Components and Implementation Time Frames
as of August 2005
Figure:
1 BRITS Efficiency Revenue (Negative Efficiency Amounts)
March 2003 through March 2005
(Unaudited)
3
21
The Office of the Auditor General has conducted a performance audit of the
Department of Revenue’s (Department) Business Reengineering/Integrated Tax
System (BRITS) project pursuant to a November 20, 2002, resolution of the Joint
Legislative Audit Committee. This audit was conducted as part of the sunset review
process prescribed in Arizona Revised Statutes (A.R.S.) §41-2951 et seq and is one
of a series of four reports on the Department. The
other reports focus on the Audit Division, the
Collections Division, and an analysis of the 12
statutory sunset factors.
Department administers state
taxes
The Department was established in 1973 to
provide an integrated, coordinated, and uniform
system of state tax revenue administration and
collection. The Department is responsible for
licensing, processing, and collecting most taxes
for the State. The Department’s activities include
issuing licenses or registration numbers for
transaction privilege and withholding taxes;
processing tax revenues paid by businesses,
corporations, and individuals; and issuing
taxpayer refunds. In addition, the Department
helps ensure compliance with state tax laws
through its enforcement functions, such as
auditing taxpayer returns, identifying taxpayers
who should have filed tax returns but did not,
and collecting from taxpayers who did not pay or
underpaid their taxes. In fiscal year 2004, the
Department was appropriated approximately
$63 million and 1,134 full-time equivalent
positions. Further, according to the
Department’s 2004 annual report, total tax
Office of the Auditor General
INTRODUCTION
& BACKGROUND
page 1
Arizona’s Primary Tax Revenue Sources and
Amounts (Fiscal Year 2004)
􀁺􇩔 Transaction privilege, use, and severance taxes ($5.6
billion)—A transaction privilege tax is imposed on the seller
for doing business in the State. Because it is usually passed
on to the customer, this tax is commonly referred to as a
sales tax. Use tax is imposed on businesses or individuals
for purchasing personal property from out-of-state vendors
for which sales tax was not paid, and severance tax is
imposed on the business of mining metalliferous minerals
and severing timber. Some of these taxes are collected for
and distributed to Arizona’s cities and counties.
􀁺􇩃 Corporate, individual, and withholding income taxes ($3.8
billion)—Income taxes are imposed on individuals and
corporations earning income in Arizona. In addition,
businesses or individuals who hire employees must
withhold income tax from their employees’ compensation
and remit it to the Department. Income tax revenues are
shared with Arizona cities and towns.
􀁺􇩌 Luxury tax ($338 million)—These taxes apply to liquor and
tobacco. The Department is responsible for issuing tobacco
licenses and stamps and collecting taxes on tobacco
products and liquor. The majority of these monies are
distributed to special funds, such as the Drug Treatment and
Education Fund.
revenue collected by the Department during fiscal year 2004 exceeded $9.8 billion
(see text box, page 1).
BRITS designed to improve service and generate new
revenue
Under the BRITS project, the Department is reengineering its core business
processes and implementing an integrated tax system that is designed to make it
easier for taxpayers to file returns and pay taxes, and the Department to conduct their
tax functions. The Department’s numerous tax administration and enforcement
functions are heavily dependent on information technology. However, the
Department’s information technology, in large measure, was outdated, and over the
years, many separate independent systems were developed. With BRITS, taxpayer
data would be entered in one place, and all taxpayer information would be available
through this one integrated system. This multi-year project is being funded by
increased tax revenue that the system is expected to generate.
BRITS grew out of concerns about existing processes and
systems—BRITS’ beginnings go back more than 5 years. In fiscal year 2000, the
Legislature appropriated $500,000 to the Department to hire a consultant to study its
business processes and evaluate its information technology options. This study
found that the Department’s information technology infrastructure was old and
inefficient, and associated business processes did not support the Department’s
objective to effectively and efficiently serve Arizona’s taxpayers. For example, the
Department’s transaction privilege tax system was built in 1982 and had reached its
technical limits, meaning that it could not be modified to meet the Department’s
needs. In addition, numerous stand-alone databases were developed in isolation,
resulting in the need to input taxpayer information multiple times and making it
difficult for department staff to quickly view all aspects of a taxpayer’s business with
the Department.
Following this study, the Department worked with Arizona’s State Procurement Office,
currently referred to as Enterprise Procurement Services (EPS) to develop a request
for proposal for improving the Department’s business processes, organizational
structure, and information technology. The request for proposal was issued in March
2001, and in August 2002, a state contract was signed with Accenture. As indicated
on its Web site, Accenture is a global management consulting, technology services,
and outsourcing company that has more than 110 offices in 48 countries.
BRITS being phased in and offers several potential advantages—
BRITS, which is being implemented in phases, will provide the Department with
several potential advantages. As shown in Table 1 (see page 3), the first phase of the
project has been implemented. The completion date for the second phase of the
State of Arizona
page 2
Office of the Auditor General
page 3
Table 1: BRITS Components and Implementation Time Frames
as of August 2005
Phase/Component
Description
Implementation Time Frame1
and Status
Phase 1—Transaction Privilege
and Withholding Taxes
September 2002–October 2004
Status: Implemented
Develop core database Develop a core database to perform the Department’s basic tax
functions, such as processing taxpayer licenses, tax returns, and
payments; issuing bills and refunds; and performing collections
and audit processes.
Convert tax data Develop the necessary computer programming to convert
taxpayer data from the Department’s old computer systems into
BRITS.
Create Web-based processing
system—AZTaxes.gov
Create a Web-based processing system business taxpayers can
use for filing transaction privilege and withholding tax returns and
payments.
Develop a system for creating
management reports
Develop a computer application to help department officials
create management and ad hoc reports using BRITS data.
Provide other tools to support audit
and collection activities
Provide tools to help find taxpayers who have not filed their
taxes, identify taxpayers’ correct addresses, determine the
probability of collecting back taxes, and determine which
collection activities are likely to get the best results.
Phase 2—Corporate Income Tax August 2004–July 2006
Status: In process
Enhance core database Enhance the core database to perform corporate income tax
functions such as processing tax returns, payments, and refunds.
Convert tax data Develop the necessary computer programming to convert
taxpayer data from the Department’s old computer systems into
BRITS.
Provide an automated audit
application (ESKORT)
Provide an audit application to automate and enhance audit
selection activities, case management and tracking, and auditing
resources such as standardized templates for Excel
spreadsheets and Word documents.
Phase 3—Individual Income Tax Not yet established
Status: Not started
Enhance core database Enhance the core database to perform individual income tax
functions such as processing returns, payments, and refunds.
Convert tax data Develop the necessary computer programming to convert
taxpayer data from the Department’s old computer systems into
BRITS.
Provide data reading capabilities The specific activities for this area have not yet been defined, but
this area involves the electronic imaging of paper items such as
tax returns and correspondence.
Provide tools to manage relationships
with taxpayers
The specific activities for this area have not yet been defined, but
this area involves improving customer service.
1 The implementation time frame includes all activities related to each phase, including planning, defining system requirements, testing
programs, and converting data. It does not include additional support that may be required after implementation.
Source: Auditor General staff analysis of the BRITS contract and project documents, the May 2005 BRITS Program Monthly Status Report, and
interviews with department staff.
State of Arizona
page 4
project has been revised from July 2005 to July 2006 to ensure that sufficient time is
taken to fully define system requirements, complete testing, and minimize
operational impacts. According to the Department, the scope of the final phase,
which involves converting individual income tax, has not been fully defined at this
time; therefore, target dates have not been set. Some of BRITS’ features include:
􀁺􇩃 Consolidation of disparate systems—BRITS is expected to provide a single
system where all tax processing is done rather than several separate systems
where only specific parts of the process occur. The BRITS project’s three main
phases revolve around developing specific components of BRITS and
converting data from the Department’s old computer systems. For example, as
shown in Table 1 (see page 3), the first phase involved designing and
implementing the main database where all tax processing activities will occur;
developing other BRITS components designed to improve the Department’s
processes, such as providing tools to help find taxpayers who have not filed their
taxes; and converting transaction privilege and withholding tax data from the
Department’s old system to BRITS. This phase has been implemented as of
October 2004. In addition, according to the Department, as a part of the BRITS
project, it has received approximately 1,100 desktop and 60 laptop computers.
􀁺􇨠 Better service to taxpayers—BRITS, when fully implemented, is expected to
provide the public with the ability to do new things. For example, during the first
phase, the Department’s Web site was expanded to include AZTaxes.gov, which
allows business taxpayers to file transaction privilege and use, as well as
withholding tax returns, and remit payments on-line.
􀁺􇨠 Improved tools for tax collection—BRITS also includes tools that will score its
collection cases based on the probability of collecting, and determine the
collection activities that will most likely result in the Department’s collecting the
tax money owed. In addition, the BRITS project also provided the Department’s
field collectors (i.e., those collectors who meet with taxpayers at their business
locations) with 50 handheld computers. The handheld computers contain
specific information on taxpayers’ accounts, including amounts owed,
payments made, and prior collections history. This device essentially provides
collectors with the ability to perform major job functions, such as initiating
payment plans and logging case activity, while in the field.
Department has major responsibility for shaping and staffing
BRITS—BRITS is an “indefinite-quantity contract.” Such contracts are typically used
in situations where, due to a project’s size and complexity, it is not possible to know
the exact project components up front. Although the general scope of BRITS was
known at the time the contract was awarded, according to an EPS official, the
specific components of the project—such as products and services and timing of
work—could only be estimated. Therefore, the project components are actually
ordered and purchased in pieces as implementation nears and the necessary
components can be better defined. The BRITS project refers to each component as
a task. EPS officials delegated day-to-day administration to the Department.
However, changes to any services or products that affect the overall contract scope
cannot be authorized without EPS amending the contract.
BRITS involves many Accenture and department staff resources. According to the
Department, Accenture has approximately 70 staff and subcontractors working on
BRITS, including a project manager who provides the overall guidance and direction;
programmers who program, develop, and execute test plans; support technicians
who monitor system performance and provide technical guidance to programmers;
and staff who design, develop, and deliver training. The Department indicated that it
is also contributing about 65-70 full- or part-time staff to BRITS, including a project
manager, information technology staff, and experienced employees who understand
the Department’s processes and can help articulate system requirements and
participate in testing.
Contract features tie payment to enhanced revenues—The contract
uses a benefits-sharing funding approach. Under this approach, the vendor finances
the contract costs up front and is repaid based on some measurable benefit, such
as cost savings or increased revenue.1 For BRITS, the measured benefit is increased
enforcement revenue. Enforcement revenue is tax money the Department receives
as the result of its enforcement functions, such as audits, accounts receivable, and
collections. According to the Department’s annual report, it collected approximately
$490 million in fiscal year 2004. BRITS is intended to generate additional enforcement
revenue by making the Department’s audit, accounts receivable, and collections
processes more effective and efficient. For example, the BRITS project includes
implementing programs to identify taxpayers who have not paid taxes, but should
have.
The contract provides that Accenture will be paid 85 percent of the additional
enforcement revenue generated by BRITS, and the State receives the remaining 15
percent. Specifically, as Accenture incurs costs to implement BRITS, it provides the
Department with invoices for the costs. If BRITS has generated additional
enforcement revenue, the Department uses 85 percent of that money to pay
Accenture’s invoices. If BRITS has not generated enough additional enforcement
revenue for the Department to pay the invoices, Accenture charges the Department
6 percent annualized interest on the unpaid balance. This process continues until the
total cost of BRITS, $122.65 million plus an estimated $9-$13 million in interest, is
paid. However, if BRITS has not generated enough additional enforcement revenue
to pay the contract costs by the end of the 10-year contract term, the Department will
no longer be liable for the unpaid costs. In other words, the Department will no longer
pay Accenture and will keep 100 percent of the additional enforcement revenue
generated by the system after that time.
1 A.R.S. §41-2559 authorizes the use of benefits-sharing funding methods.
Office of the Auditor General
page 5
Auditors interviewed or reviewed materials from several other states with benefits-sharing
contracts and found that the general benefits-sharing terms of those
contracts and BRITS’ were relatively similar, and like Arizona, many used the method
because they lacked the resources to pay contract costs up front.1 For example,
representatives from Kansas’ Department of Revenue and Virginia’s Department of
Taxation indicated that the states entered into contracts for automated tax systems
priced at approximately $63 million and $166 million, respectively, and used benefits-sharing
payment methods due to budgetary constraints.
Audit scope and methodology
This audit focused on whether the Department was effectively managing BRITS’
implementation and also assessed the enforcement revenue that is subject to the
contract’s benefit-sharing requirements and the potential impact of changes being
made to the BRITS contract. The audit did not include an assessment of BRITS’
vendor or the vendor’s efforts to manage the BRITS project. This audit report
presents three findings and associated recommendations:
􀁺􇩔 The Department could do more to ensure it effectively manages BRITS’
implementation and that BRITS will meet its needs. The first phase of the BRITS
implementation was problematic and hindered the Department’s ability to
perform some basic functions, including billing taxpayers for taxes owed. The
Department may have minimized these problems if it had hired an outside
advisor as it had originally outlined in the contract and had assigned additional
knowledgeable staff to the project. The Department has taken steps to improve
its project management, but could better ensure success by more closely
following IT project management best practices.
􀁺􇩐 Problems with BRITS, in part, have resulted in the Department‘s generating
approximately $9.3 million less then expected for one type of enforcement
revenue, based on department information. Lower enforcement revenue is
hindering the Department’s ability to pay for the contract in a timely manner
resulting in the State’s paying additional interest. The Department needs to
continue working with Accenture to improve BRITS-generated revenue and,
although not required, should also continue to include information about this
enforcement revenue type in its monthly status report.
􀁺􇨠 The Department made changes to the BRITS contract that could affect the
overall contract scope. Despite this, the Department did not consult
procurement experts to ensure the changes were in the State’s best interest and
1 Auditors interviewed representatives and reviewed various contract-related materials from California, Hawaii, Kansas,
Massachusetts, Missouri, North Carolina, and Virginia; and reviewed an audit report from Florida. For most of these
states’ contracts, either the state was charged interest or, if interest was not charged, there was no dollar limit on the total
amount of benefits to be paid to the vendor. Of the four states to be charged interest, Virginia and Missouri were charged
10 percent and 8 percent annual interest rates, respectively. Auditors were not able to determine the annual interest rate
for California and Hawaii.
State of Arizona
page 6
within legal requirements, such as contract terms and procurement rules, and
did not adequately document the changes. The Department should improve its
communication and documentation of contract changes to help ensure the
changes are appropriate.
Various methods were used to study the issues addressed in this audit. These
methods included interviews with legislative staff, including Joint Legislative Budget
Committee staff; department management and staff; Accenture representatives; and
staff from the Government Information Technology Agency (GITA). Auditors also
reviewed Arizona Revised Statutes, the BRITS Project and Investment Justification
and monthly status reports that the Department submitted to GITA, and the BRITS
contract, including the Request for Proposal and Accenture’s proposal and best and
final offer.
Auditors also used the following specific methods:
􀁺􇨠 To determine whether the Department was appropriately managing BRITS’
implementation and project activities were consistent with best practices,
auditors interviewed more than 30 department staff involved in BRITS’
implementation, reviewed project documents, and also obtained and reviewed
best practice materials related to information technology project management.1
Auditors then compared the best-practice information to the contract
requirements, Accenture documents, and department information about what it
was doing to manage BRITS implementation. In addition, auditors reviewed
contract materials and interviewed officials from Hawaii’s Department of Taxation
because it used quality assurance contractors to oversee information
technology contracts. Auditors also observed staff using the BRITS system to
identify specific functions staff could not perform due to BRITS’ implementation
problems.
􀁺􇨠 To assess enforcement revenue that is subject to the contract’s benefits-sharing
requirements, auditors obtained and reviewed documentation from the
beginning of the project through May 2005, such as spreadsheets related to the
Department’s calculation of enforcement revenues generated by BRITS.
Specifically, auditors reviewed this material to determine the amount of revenue
BRITS had generated and whether the process for calculating the revenue was
within the contract’s requirements. To evaluate the Department’s method of
tracking BRITS’ costs, auditors reviewed the Department’s cost-tracking
1 Best practice materials used included: (1) Charvat, Jason. Project Management Nation: Tools, Techniques, and Goals for
the New and Practicing IT Project Manager. New York: John Wiley and Sons, Inc., 2002; (2) IT Governance Institute. 2004.
“Control Objectives for Information and Related Technology (COBIT) Online, v3.1.” www.itgi.org (Nov. 19, 2004); (3) North
Carolina State Government, Information Resource Management Commission. Office of Information Technology Services.
Implementation Framework for Statewide Technology Projects: Best Practices and Standards. North Carolina State
Government, April 2004; (4) Project Management Institute, Inc. A Guide to the Project Management Body of Knowledge,
3rd ed. Pennsylvania: Project Management Institute, Inc., 2004; (5) State of Arizona Government Information Technology
Agency (GITA). “Best Practice Checklist.” www.azgita.gov/downloads/ (November 2004); (6) Taylor, James. Managing
Information Technology Projects: Applying Project Management Strategies to Software, Hardware, and Integration
Initiatives. New York: AMACOM, 2004; and (7) U.S. Department of Labor. IT Project Management. Washington, D.C.: DOL,
Jan. 2003.
Office of the Auditor General
page 7
spreadsheets and selected a sample of nine invoices from the period ranging
from project inception through October 2004. Finally, auditors obtained and
reviewed various sources of literature about benefits-sharing contracts, and
interviewed representatives and/or reviewed materials from eight other states
that have used benefits-sharing contract funding methods.1
Auditors’ efforts in this area focused primarily on the processes the Department
uses for calculating BRITS-generated enforcement revenue. Specifically,
auditors reviewed those processes to ensure that they met the contract’s
benefits-sharing requirements. For example, auditors ensured that the
Department was measuring the appropriate types of enforcement revenue,
adjusting baselines for unusual events, and taking steps to ensure enforcement
revenue was not double-counted. Auditors did not attempt to validate the
accuracy of the Department’s calculations and did not assess the
appropriateness of the contract’s benefits-sharing terms or whether those terms
are favorable to the State.
􀁺􇩔 To assess the changes made to the BRITS’ contract and the impact the changes
would have on the contract’s price, auditors analyzed the contract terms
regarding price and how changes or amendments to the contract should be
handled. In addition, auditors obtained and reviewed department information
from the beginning of the contract through March 2005 regarding the price and
the scope of the contract’s tasks that had been started and any changes that
had occurred. This information was used to assess the effect of any changes on
the contract’s price and requirements and to determine whether the
documentation supporting the changes was adequate. Auditors also obtained
and reviewed general information about indefinite-quantity contracts, and
interviewed management and staff from the Arizona Department of
Administration, Financial Services Division, Enterprise Procurement Services,
about specific terms and requirements of the BRITS contract.
􀁺􇩔 To develop the Introduction and Background section, auditors compiled
information from state laws, unaudited information from the Department’s 2004
Annual Report, its Web site, and other department-prepared documents; the
BRITS contract; the fiscal year 2000 State of Arizona Appropriations Report; a
2001 Gartner Consulting report presented to the Department; and Accenture
project documentation and information from its Web site.
The audit was conducted in accordance with government auditing standards.
The Auditor General and staff express appreciation to the director and staff of the
Department of Revenue for their cooperation and assistance throughout the audit.
1 As noted previously auditors interviewed representatives and reviewed various contract-related materials from California,
Hawaii, Kansas, Massachusetts, Missouri, North Carolina, and Virginia; and reviewed an audit report from Florida.
State of Arizona
page 8
Department needs to better manage BRITS
project
The Department could do more to ensure that it effectively manages BRITS’
implementation and that BRITS will meet its needs. The first phase of BRITS’
implementation was problematic and hindered the Department’s ability to perform
some basic functions, including billing taxpayers for taxes owed. These problems
may have been minimized if the Department had hired an outside advisor as it had
originally outlined in the contract and had assigned additional knowledgeable staff to
the project. The Department is taking steps to improve project management, but
could better ensure success by more closely following IT project-management best
practices.
BRITS’ problems affecting Department’s ability to perform
some basic functions
Problems that occurred during the initial phase of the BRITS implementation have
resulted in BRITS not functioning as intended in some areas. These problems have
impacted the Department’s ability to collect revenue or to perform functions that were
working properly before BRITS was implemented.
Problems experienced in converting data—According to the Department,
by January 2004, demographic information for the transaction privilege (commonly
known as sales tax) and withholding tax types had been converted to BRITS.
Information that was converted included business taxpayer names and identification
numbers, and some accompanying financial information for the transaction privilege
tax type such as the amount taxpayers owed to the State. However, the data did not
convert correctly in some cases. For example, over 7,500 (amounting to more than
$28 million) of the approximately 87,000 transaction privilege tax accounts-receivable
items did not correctly convert. Following the conversion, the Department spent
several months investigating these accounts and was able to correct all but
approximately $2 million of this receivable balance by June 30, 2004.
Office of the Auditor General
page 9
FINDING 1
State of Arizona
page 10
Problems experienced in issuing bills and refunds—Some billings were
incorrect, which also affected some refunds. Due to problems with implementing the
transaction privilege tax (TPT, commonly known as sales tax) portion of BRITS, the
Department had to manually review and correct TPT bills before sending them to the
taxpayers, in turn causing a backlog of bills to be mailed. Specifically, approximately
6 months after the Department began processing transaction privilege tax returns in
BRITS (June 30, 2004), bills had not been sent for over 17,000 taxpayer liabilities
totaling nearly $45 million.
Similarly, according to the Department, bills for withholding tax accounts were also
backlogged, and although the demographic information had already been
converted, the financial information for withholding tax accounts was not converted
to BRITS until October 2004. The bills for converted withholding tax accounts as well
as those associated with the first quarter of calendar year 2004 were not processed
until January 2005, and those associated with the second quarter 2004 were not
processed until June 2005. The Department is taking steps to address these issues
by working with Accenture to make necessary programming changes to BRITS.
Additionally, according to the Department, as of August 2005 it was completing the
processing of third and fourth quarter 2004 withholding returns and issuing the bills
associated with these time frames. However, in part because of BRITS’ billing
problems, the Department could not initially collect the tax monies owed to the State
that were associated with these bills. Additionally, problems with billing in some
cases also affected the Department’s ability to issue refunds. According to a
department official, the Department had to check taxpayer accounts to determine
whether a refund was actually owed to the taxpayer, or whether it needed to be
applied to a previously underpaid period.
System could not process one-time-only license numbers—Following
BRITS’ initial implementation, the Department was able to collect the revenue for one-time-
only transactions, but it could not create the one-time-only license numbers and
process the returns associated with these transactions until January 2005. A license
must be established with the Department to file and pay transaction privilege (sales)
taxes, and sometimes a license number is needed for one transaction only. For
example, if an Arizona resident buys a vehicle out-of-state that will be used in Arizona,
the Department needs to establish a one-time-only license number for the individual
so that the Department can appropriately record the tax that must be paid in Arizona.
According to the Department, these license numbers could not be created in BRITS
because the system programming was not done.
Transaction privilege
and withholding tax
bills were
backlogged due to
BRITS problems.
Department did not initially involve enough key people
When the Department began the BRITS project, it underestimated the level of key
staff required, and although its project management has improved, the Department
can do more. The Department did not hire an outside advisor to help it manage the
project, nor did it initially involve some of the most knowledgeable department IT staff
who had experience with the previous computer systems. The Department has taken
some steps to involve a greater number of more knowledgeable staff, but it could still
benefit from outside assistance.
Outside advisor not hired—BRITS’ implementation issues may have been
minimized if the Department had hired an outside advisor as outlined in the BRITS
contract. The BRITS contract contains a clause indicating that the Department
intended to hire an “oversight advisor” who was not affiliated with the project. The
advisor was to report on BRITS’ progress, risks, and budget to department executive
management and the Government Information Technology Agency and to, among
other things, audit the contractor’s performance and make advisory
recommendations to the director. Department officials indicated they were not certain
why an outside advisor was not hired, but indicated that cost may have been a factor.
Hiring an outside advisor may have prevented or at least helped identify and mitigate
the BRITS implementation problems. For example, in 2003, Hawaii’s Department of
Taxation hired an outside advisor for approximately $200,000 a year to provide quality
assurance consultant services for its integrated tax system project. The contract
called for the consultant to, among other things, review project work plans for
sufficiency, identify project risks and recommend preventive measures, evaluate
project staffing levels, determine whether the project design is compatible with the
agency’s requirements, and evaluate the readiness of business and technical users
who have received training. A Hawaii Department of Taxation official who auditors
spoke with indicated that using an outside advisor was helpful and that a benefit of
such an advisor is that they can alert the agency to things that should or should not
be occurring on the project. For example, the official mentioned that the agency had
the quality assurance contractor look into whether testing was being done to
specifications.
Shortage of key staff involvement and project management
changed—The Department indicated that it did not involve a sufficient level of the
most knowledgeable department staff and that there was turnover in the project
manager position. For example, some department managers indicated that an
insufficient number of department IT staff who were experienced with the previous
computer systems were involved in the BRITS project. In addition, the BRITS
implementation has been managed by four different department project managers
Office of the Auditor General
page 11
An outside advisor may
have helped to mitigate
BRITS implementation
problems.
State of Arizona
page 12
and two different Accenture project managers since it began. The Department’s first
project manager was experienced in business activities, but did not have IT project
management experience, and the Department’s second project manager had IT
experience, but only managed the project for 7 months.
Department has some improvements under way—Although the
Department did not hire an outside advisor for BRITS, it has taken steps to better
control the project. The Department’s third and fourth project managers have
extensive IT project management experience. In addition, the Department formed an
internal oversight committee consisting of the Department’s deputy director and
division assistant directors, and the Department’s and Accenture’s project
managers. This committee now oversees the work being done to correct problems
from the initial phases of the BRITS implementation and make decisions concerning
the future phases of the project. For example, according to department
management, this committee participates in the decisions to adjust the timeline to
allow more time to outline the specific functions the system needs to perform, and
has the authority to recommend postponing the next conversion if it is not satisfied
the Department is ready. Further, according to department management, its IT
division staff have become more involved and have taken on some BRITS’ support
functions. For example, department IT staff are responsible for supporting the
segment of BRITS that is used to create reports by performing tasks such as
providing user training and creating requested reports. Additionally, department IT
staff are involved in the decision-making process to determine when BRITS system
problems will be corrected.
Hiring outside advisor may still be beneficial—Even though two tax types
have already been converted to BRITS, and the Department is taking steps to more
effectively manage the project, it should still consider hiring an outside advisor to help
oversee the remaining implementations. To do so, the Department would have to
examine possible funding methods, such as allocating monies from its current
budget, or seeking a separate appropriation from the Legislature. Based on the
Hawaii contract, it may cost approximately $200,000 a year. According to the
Department, its IT staff are experienced with the systems that BRITS is replacing;
however, it does not currently have the level of expertise in terms of software,
requirements gathering, system design, and programming skills that Accenture has.
In addition, even though the current department project manager has IT project
experience, the assistance of an outside advisor with significant IT project
management experience could still help the department better manage the project
by providing impartial observation and by pointing out possible improvements or
issues that the Department may not recognize.
Department can do more to follow best practices
Whether or not an outside advisor is hired, the Department will need to continue
improving its project management. Auditors’ comparisons between the
Office of the Auditor General
page 13
Department’s efforts and IT project management best practices showed that while
the Department is making progress, it could follow these practices more closely to
further improve its management and increase the chances of project success:
􀁺􇩐 Planning the project—Although Accenture did create a project master planning
document for the BRITS project, the Department did not create planning
documents to guide its own management of the project in order to ensure the
BRITS contract terms are fulfilled and the project is successful. Best practices
indicate that a project plan should be developed containing, among other
things, a statement of the project’s scope, details of project deliverables, and
estimates of required resources and responsibilities. Accenture developed a
project plan for BRITS as its overall guiding document for the project. Although
the plan appears to be consistent with best practices and contains elements
such as a project description, staff roles and responsibilities, and a project
timeline, it is mainly a guide for Accenture’s management of the project, not the
Department’s. In light of the problems experienced with the first phase of the
BRITS implementation, the Department should develop and periodically update
a project management plan that outlines items such as the methods it will use
to ensure that Accenture is meeting requirements, the criteria and process it will
use to create and approve the deliverables, and the resources the Department
will commit to the project along with staff responsibilities.
􀁺􇨠 Defining requirements—The Department worked with Accenture prior to the first
implementation phase to determine what functions it needed the system to
perform, but it did not ensure that these system requirements were adequately
defined. Best practices indicate that there should be clear, concise requirements
written and communicated to the vendor, the vendor should ensure that they are
precisely interpreted, and the vendor and customer should be in complete
agreement about the project intent and desired results.1 Because BRITS is an
indefinite-quantity contract, the specific requirements were expected to be
defined during implementation. Department management acknowledged that
an insufficient number of both system users and key department staff were
involved in articulating the Department’s requirements. In addition, department
management also acknowledged that for BRITS’ initial phases, the
Department’s requirements were not clearly defined and detailed enough.
The Department is taking steps to ensure that its requirements are better defined
for BRITS’ next phase. Specifically, the Department is ensuring that more
detailed processing flowcharts are created, that the defined processes are
reviewed by knowledgeable staff from more than one division, and that these
requirements are approved by the Department’s division assistant directors.
When auditors reviewed a requirements document created for the next tax type
to be converted, corporate income tax, they found it to be more detailed than
1 Taylor, James. Managing Information Technology Projects: Applying Project Management Strategies to Software,
Hardware, and Integration Initiatives. New York: American Management Association, 2004.
The Department should
develop its own BRITS
project management
plan.
State of Arizona
page 14
those created for the first tax type, transaction privilege tax. To prevent some of
the issues that occurred in BRITS’ initial phases, the Department should
continue its efforts to include the most knowledgeable staff in better defining the
Department’s requirements of the BRITS system and ensuring that both the
Department and Accenture agree on what those requirements are until the
BRITS project is complete.
􀁺􇩁 Assessing risks—The Department appears to have risk management
processes in place that conform to best practices. Best practices indicate that
a formalized, documented set of guidelines and procedures should be
implemented for risk management. Accenture developed a Risk Management
Plan that provides guidance on how to identify, analyze, and mitigate risks.
Further, Accenture works with the Department to
produce a risk assessment document that contains
information about project risks and how they should be
mitigated. Risks may come from internal or external
sources and can affect project costs, schedules, and
quality. For example, possible external risks to the
BRITS project include last-minute changes in
legislation that can affect the project schedule, or the
inexperience of some Accenture staff that can affect
project quality. Internally, there is a risk to the BRITS
project if certain outstanding issues from the first phase
are not resolved when converting the next tax type. The Department plans to
mitigate this risk by identifying the priority issues that can affect the next
conversion and adding new programmers to work on them. According to the
Department, a monthly meeting is held to reevaluate the risks and mitigation
measures. To remain aware of and plan for project risks, the Department should
continue its efforts to identify, document, and mitigate risks until the project is
complete.
􀁺􇩔 Testing whether functions work—The Department could better ensure that
BRITS is adequately tested prior to implementation. Although testing plans were
written for the initial phase of BRITS, testing may have been inadequate because
users were not involved soon enough before implementation. Best practice
standards indicate that testing plans are important. Specifically, best practices
indicate that complete and detailed test plans should be developed, and that
system and program designs must be tested to ensure that they will work as
intended. In line with best practice standards, Accenture’s test plan contains
information describing the objectives and strategy for testing and provides
information about the different types of tests to be performed, such as user
testing in which department staff will test the system using their business and
operational procedures.
Even though Accenture developed testing plans, several department
management and staff members had concerns about the amount of testing that
occurred during the first phase of the project. Best practices indicate that an
Internal and External
Project Risks
Internal
􀁺􇩔 Technical Failures
􀁺􇩄 Delays in Scheduled Work
􀁺􇩐 Project Staff Errors
External
􀁺􇩌 Legislative Changes
􀁺􇩖 Vendor Staff Inexperience
Office of the Auditor General
page 15
adequate amount of thorough testing is essential to project success and that
user-testing should be performed. One of the Department’s project managers
indicated that, prior to the transaction privilege tax implementation, there was not
enough time for users to fully test the system. Specifically, the users did not get
involved with testing until about 6 weeks before the conversion occurred.
According to the project manager, system users should have been testing for
about 3 months prior to implementation in order to get through all the necessary
test scenarios and retest the things that did not initially work right. Therefore, the
Department should ensure that system users are involved with testing as early
as possible, and that an adequate amount of time is taken prior to conversion
to perform testing, and correct and retest the problems that are found.
􀁺􇨠 Training users and IT staff—Despite following some best practices for training,
initial user and IT staff training was inadequate and could impact the
Department’s ability to effectively use, operate, or maintain BRITS. Training best
practices suggest that training should be planned for both system users and the
IT staff expected to maintain it. Further, best practices indicate that a training
plan should be developed that includes information such as who will be trained,
what they will be taught, and when training will occur. Therefore, training must
address the needs of users and IT staff, and they should be adequately trained
prior to implementation. Accenture developed training plans that provided an
overview of its training strategy and outlined some of the classes to be given to
users and IT staff. In addition, according to Accenture training staff, department
staff approved the classes’ content.
Despite Accenture’s preparations, BRITS training during the initial phase was not
adequate according to department staff. For example, some staff indicated that
the initial BRITS training was not detailed enough for the specific job functions
they had to perform, and that sometimes during training, computer errors
occurred that prevented them from training on certain functions. For example,
one staff member indicated that the system they used for BRITS training did not
work, and when BRITS was implemented, staff did not know how to perform
some functions such as filing liens.1 In addition, a department project manager
indicated that formal, technical training for IT staff occurred several months after
implementation in 2004, and it should have been done much sooner. A best
practice source indicates that before implementing a project, management
should ensure that IT operational support staff is adequately trained since they
will be responsible for maintaining the system once it is handed over.2
The Department’s training issues have been communicated to Accenture, and
Accenture has updated its overall training plan. The updated plan indicates that
Training should be
sufficient to ensure that
users can use the
system and IT staff can
maintain it.
1 To protect the State’s financial interests, the Department has statutory authority to file liens whenever taxes are owed to
the State. For example, the Department can file property liens with each of Arizona’s 15 county recorders that prevent the
taxpayer who owes state taxes from buying, selling, or borrowing against the property until the lien is resolved.
2 Charvat, Jason. Project Management Nation: Tools, Techniques, and Goals for the New and Practicing IT Project Manager.
New York: John Wiley and Sons, Inc., 2002.
the training classes will focus on each group of users’ specific needs, there will
be more opportunity for “hands-on practice,” and the training will be given to
users as close in time as possible to when the next phase of BRITS is
implemented. Further, according to Accenture, the computer training
environment that users will practice on has been redesigned and appears to be
working well. Finally, according to the Department, IT staff training has been
occurring since September 2004 and is ongoing. The Department should
continue to ensure that future training will address previous training problems
and will allow the Department to effectively use, operate, and maintain BRITS.
􀁺􇩁 Assessing system security—Though BRITS is a new system and not fully
implemented, because of the sensitive nature of the BRITS data, BRITS
information security policies and procedures should be
put in place, and according to best practices, security
assessments should be performed to ensure
compliance with those policies and procedures. Some
security elements that should be covered by policies
and procedures include system access controls, audit
trail creation and analysis, data encryption
management, virus protection, and intrusion detection
and prevention (see text box). The Department has
begun developing information security policies and
procedures, and according to the Department, it has
already addressed some of the security elements they
should contain. For example, the Department contracts
for a yearly independent security assessment that
covers intrusion prevention and detection. In addition,
according to the Department, it performs some
assessment procedures over user system access by
creating an annual system access report that must be
reviewed for accuracy by the unit managers within each
division. However, the Department’s policies and
procedures have not yet been completed, reviewed,
and adopted as department-wide standards. Therefore,
to ensure the BRITS system is secure and sensitive
data is protected, the Department should complete and
implement its information security policies and
procedures as soon as possible. In addition, to conform
to best practices, the Department should periodically
perform security assessments to determine if its policies
and procedures are being followed and that they adequately ensure system
security.
􀁺􇩅 Ensuring requirements are met—Even though the contract requires it, the
Department does not have a formal plan for how it will ensure that the BRITS
State of Arizona
page 16
Best Practice Security Elements Defined
System Access Control—The process that limits and controls
access to a computer system; a logical or physical control
designed to protect against unauthorized entry or use.
Audit Trail Creation and Analysis—A visible trail of evidence
enabling one to trace information contained in statements or
reports back to the original input source.
Data Encryption—The process of taking an unencrypted
message, applying a mathematical function to it, and producing
an encrypted message.
Virus Protection—A method of detecting and removing an
unauthorized program or code buried within an existing program
that will cause some unexpected and undesirable event when the
program is executed.
Intrusion Detection and Prevention—The process of monitoring
the events occurring in a computer system or network to detect
signs of security problems.
Source: Definitions obtained by Auditor General staff from Information Systems Audit and
Control Association, COBIT Online Glossary, and the Arizona Government Information
Technology Agency Web site.
Office of the Auditor General
page 17
project requirements are being met. Best practices indicate that a post-implementation
review plan should be created and that post-implementation
reviews should be conducted in a timely manner. These reviews would include
activities such as assessing whether expected benefits have been achieved;
whether the project met its schedule, budget, and user expectations; and
determining improvements that could be made on future projects. While these
reviews may typically be done at the end of a project, because the BRITS project
is so large and complex, it would be useful to perform reviews as the project
unfolds. The Department has done some things that would partially fulfill a post-implementation
review. For example, the Department is in the process of
comparing the BRITS contract requirements to what was delivered during the first
phase to identify requirements that have not been met. However, to help with the
project’s planning and execution of future phases, the Department should
develop and follow a formal post-implementation review plan, as stated in the
BRITS contract, that outlines how and when it will perform the reviews.
􀁺􇩉 Improving the project along the way—Post-implementation review also involves
determining how to improve along the way. For example, best practices indicate
that meetings should be held to identify project successes and failures and make
recommendations for improving future performance. After the first phase of the
BRITS project was implemented, according to an Accenture official, the
Department and Accenture held a “lessons learned” session to solicit feedback
from department staff. Lessons-learned comments included Accenture
developers not adequately listening to staff to learn about what the Department’s
requirements were, department staff not knowing what they needed to look for
when testing, and BRITS’ outdated and unstable training environment. According
to the Department and Accenture, some of the lessons-learned issues are now
being addressed in the project’s next phase. For example, as indicated above,
experienced department staff and Accenture are working together to gather in-depth
requirements to be used when designing the next phase of BRITS. These
requirements are being approved by the Division’s assistant directors. Until the
project is complete, the Department should continue to hold lessons-learned
sessions at appropriate points during the project and should document and use
the results of these meetings to improve BRITS’ implementation.
Post-implementation
reviews assess issues
such as whether
expected benefits
have been achieved,
and determine what
can be improved on
future projects.
Recommendations
1. To take advantage of expert project management advice, the Department
should consider hiring an outside advisor to oversee the remaining
implementation of BRITS and fulfill the outside advisor‘s requirements that are
described in the BRITS contract. To do so, the Department would have to
examine possible funding methods, such as allocating monies from its current
budget, or seeking a separate appropriation from the Legislature
2. To better ensure success and to ensure that the terms of the BRITS contract are
fulfilled, the Department should develop and follow its own overall plan to
manage the project.
3. To prevent some of the issues that occurred in the initial phases of BRITS, the
Department should continue its efforts to include key stakeholders in better
defining the Department’s requirements for the BRITS system and ensuring that
both the Department and Accenture agree on what those requirements are until
the BRITS project is complete.
4. To remain aware of and plan for project risks, the Department should continue
its efforts to identify, document, and mitigate risks until the BRITS project is
complete.
5. To make certain that all necessary functions of the system are working prior to
conversion, the Department should ensure BRITS testing is adequate, including:
a. Getting system users involved with testing as early as possible, and
b. Making sure that an adequate amount of time is taken to perform testing,
and correct and retest any problems.
6. To ensure that all department staff can perform their jobs using BRITS, the
Department should continue to ensure that future training will address previous
training problems and will allow the Department to effectively use, operate, and
maintain BRITS.
7. To ensure the BRITS system is secure and sensitive data is protected, the
Department should:
a. Complete and implement policies and procedures that govern system
access controls, audit trail creation and analysis, data encryption
management, virus protection, and intrusion detection and prevention; and,
b. Perform security assessments periodically to determine if its policies and
procedures are followed and adequately ensure system security.
8. To help with the planning and execution of future phases of the project, the
Department should develop and follow a formal post-implementation review
plan, as stated in the BRITS contract, that outlines how and when during the
project it will perform the reviews.
9. To increase the chances of success, until the project is complete, the
Department should continue to hold lessons-learned sessions at appropriate
points during the project and should document and use the results of these
meetings to improve BRITS’ implementation.
State of Arizona
page 18
Office of the Auditor General
page 19
BRITS not generating additional revenue as
expected
Although the contract is funded through increased enforcement revenue, BRITS has
not yet increased revenue as expected. BRITS appears to be generating two of the
three types of enforcement revenue that are measured—discovery revenue and
license compliance baseline revenue. However, BRITS has not been consistent in
generating the third enforcement revenue type, efficiency revenue, which appears to
be the result of problems with the system’s implementation. Because this poses
negative financial implications to the State, the Department should continue its efforts
to improve enforcement revenue. The Department should also continue to clearly
report, in its monthly status report, information about whether BRITS is generating
additional enforcement revenue.
BRITS increasing some enforcement revenue types
The BRITS contract is paid from additional enforcement revenue deemed to have
been generated by the new system. Specifically, the contract provides that Accenture
will be paid 85 percent of the additional enforcement revenue and the State receives
the remaining 15 percent until the total cost of the BRITS project, $122.65 million plus
interest, is paid, or the 10-year contract term expires. Based on department
information, BRITS appears to be generating two types of enforcement revenue,
which are discovery revenue and license compliance baseline revenue:
􀁺􇩄 Discovery revenue—Discovery revenue is the money that the Department
collects from nonfilers identified by BRITS, or from taxpayer liabilities the
Department was unable to collect before implementing BRITS. For example,
BRITS includes discovery programs that match department data with various
types of data from cities, counties, and other state agencies to identify
individuals and businesses that are not paying taxes, but should be. Identifying
the amount of BRITS-generated discovery revenue is relatively simple, as each
dollar can be traced to a specific taxpayer and BRITS discovery program.
According to department information, as of May 2005, BRITS had generated
FINDING 2
State of Arizona
page 20
1 The receivable scoring system BRITS provided was developed by the Fair Isaac Corporation and uses various analytical
models to calculate the probability of collecting tax debts. The Fair Isaac Corporation provides decision analytics
products and services that enable businesses to automate and improve the decision-making process.
approximately $9.1 million in discovery revenue, 85 percent of which was paid
to Accenture (approximately $7.7 million), as outlined in the contract.
􀁺􇩌 License compliance baseline revenue—License compliance baseline revenue is
the money that the Department collects because BRITS has improved the
efficiency of its license compliance function, which involves staff finding
businesses that are unlicensed. For example, BRITS automated the data
matching process the Department uses to identify unlicensed businesses. To
determine the amount of license compliance baseline revenue, the Department
compares its monthly license compliance revenue to a baseline dollar amount,
which is intended to represent license compliance revenue the Department
would have collected without BRITS. Any revenue above the baseline amount is
considered BRITS–generated license compliance baseline revenue. The
Department began measuring license compliance baseline revenue in July
2004. According to department information, as of March 2005, BRITS had
generated approximately $6.6 million in license compliance baseline revenue,
85 percent of which was paid to Accenture (approximately $5.6 million), as
outlined in the contract.
BRITS not consistently generating one revenue type
BRITS has not consistently generated the other type of enforcement revenue,
efficiency revenue. For several months, the Department actually collected less
enforcement revenue than it expected to collect without BRITS. This appears to have
been caused by BRITS’ inability to generate accurate bills for taxpayers’ unpaid
liabilities. Because this type of enforcement revenue has been lower than expected,
payments to Accenture have been delayed, and the State may ultimately incur more
interest charges. However, the Department is making an effort to improve
enforcement revenue, and should continue to do so.
BRITS not generating efficiency revenue—BRITS has been less successful
in generating the third type of enforcement revenue that is used to pay Accenture—
efficiency revenue. Efficiency revenue is basically all enforcement revenue other than
discovery revenue and license compliance baseline revenue that the Department
receives because BRITS has improved its enforcement activities’ efficiency (i.e., its
audit, accounts receivable, and collections functions). For example, BRITS includes
a receivable scoring system that allows collectors to focus on those liabilities that are
more likely to be paid.1
Determining the amount of efficiency revenue generated by BRITS is a fairly
complicated process. Specifically, similar to license compliance baseline revenue,
the Department determines its efficiency revenue by comparing its monthly
enforcement revenue (excluding discovery revenue and license compliance revenue)
Office of the Auditor General
page 21
1 According to the Department’s 2003 Arizona Tax Amnesty Program Final Report, published in December 2004, the
Department collected over $72 million in enforcement revenue under the Tax Amnesty Program.
2 As of September 8, 2005, the Department was in the process of finalizing efficiency revenue calculations for April through
June 2005. However, department officials indicate that, based on preliminary calculations, they expect to realize enough
efficiency revenue from those months to eliminate the cumulative negative efficiency amount.
to a baseline dollar amount, which is intended to represent enforcement revenue the
Department would have collected without BRITS. Although the contract does not
require the baseline to be adjusted for population growth or economy changes, it
does require the baseline to be adjusted when significant events occur that affect
enforcement revenue. For example, the Department had to adjust the baseline for the
amnesty program it implemented from September 1, 2003, to October 31, 2003. For
the amnesty program, the Department waived civil penalties and reduced interest on
taxpayer liabilities to encourage payment, which in turn temporarily increased
enforcement revenue.1 Further complicating the process of determining efficiency
revenue is the fact that, because the system is not yet fully implemented and has
some programming problems, the enforcement revenue that is compared to the
baseline is recorded in several different systems and must be manually calculated.
Based on department information, it appears that BRITS has actually generated less
than the baseline amount for several months. When the Department compares the
monthly enforcement revenue to the baseline, any revenue above the baseline
amount is considered BRITS efficiency revenue. As with the other enforcement
revenue types, 85 percent of efficiency
revenue is used to pay Accenture for the
cost of BRITS. Based on the
Department’s calculations, BRITS
generated nearly $23.2 million in
efficiency revenue earlier in the project
(see Figure 1). However, if enforcement
revenue is below the baseline, then no
BRITS efficiency revenue was
generated, and the amount below the
baseline can be considered a negative
efficiency amount. For example, if the
Department’s monthly baseline amount
was $1.5 million and monthly revenue
was only $1 million, then there would be
a negative $500,000 efficiency amount.
Further, the negative efficiency amounts
are added together. Specifically, the
enforcement revenue that the
Department collects each month was below the baseline amount for each month
between June 2004 and December 2004, resulting in a cumulative negative
efficiency amount of approximately $27.3 million, as of December 2004 (see Figure
1). As of March 2005, the Department’s cumulative negative efficiency amount had
improved, but was still $9.3 million. 2
2003 2004
Source: Auditor General staff analysis of BRITS efficiency revenue calculations obtained from the Department of
Revenue for the months of March 2003 through March 2005.
2005
Figure 1: BRITS Efficiency Revenue (Negative Efficiency Amounts)
March 2003 through March 2005
(Unaudited)
(30)
(25)
(20)
(15)
(10)
(5)
0
5
10
15
20
Mar. Apr. May June July Aug. Sept. Oct. Nov. Dec. Jan. Feb. Mar. Apr. May June July Aug. Sept. Oct. Nov. Dec. Jan. Feb. Mar.
Dollars (in millions)
Monthly Efficiency Revenue Cumulative Negative Efficiency Amount
State of Arizona
page 22
1 The nature of BRITS’ implementation involves a lag between the time Accenture incurs and invoices BRITS costs, and
the time BRITS generates enforcement revenue to pay the invoices. Therefore, even if revenue was being generated at
the expected rate, the Department would accrue some interest charges. It is not yet possible to determine if the additional
interest charges from the payment delays will cause the Department to exceed the $9 to $13 million estimate.
Billing problems affecting efficiency revenue—The primary cause of the
negative efficiency amounts appears to be related to BRITS’ inability to generate
accurate bills for taxpayers with unpaid tax liabilities. Due to problems with
implementing the transaction privilege tax portions of BRITS, the Department had to
manually review and correct the bills before sending them to the taxpayers, in turn
causing a backlog of bills to be mailed. Specifically, approximately 6 months after
beginning to process transaction privilege tax returns in BRITS (June 30, 2004), bills
had not been sent for over 17,000 taxpayer liabilities totaling nearly $45 million.
Similarly, according to the Department, bills for withholding tax accounts were also
backlogged. Specifically, bills associated with the first quarter of calendar year 2004
were not processed until January 2005, and those associated with the second
quarter of 2004 were not processed until June 2005.
Lack of enforcement revenue negatively impacts the State—In addition
to the State receiving less enforcement revenue, the drop in efficiency revenue also
means that contract payments to Accenture are being delayed, which has potential
financial implications for the State. Specifically, the longer enforcement revenue is low
and Accenture invoices remain unpaid, the more interest the State accrues for the
project. Under the contract, it was estimated that Accenture would incur costs before
additional enforcement revenue was generated, and the State would pay between $9
and $13 million in interest. According to department information, as of March 2005,
the Department had incurred approximately $4.5 million in interest and had nearly
$43 million in outstanding invoices.1 Although Accenture is being paid with discovery
and license compliance baseline revenue, it currently is not being paid with efficiency
revenue. Although the Department’s figures suggest that BRITS is again starting to
generate efficiency revenue, the BRITS contract provides that before Accenture can
be paid from efficiency revenue, any negative efficiency revenue amounts must be
earned back. In other words, the Department will need to collect enough money
above the baseline to cover the $9.3 million cumulative negative efficiency amount
and continue to collect revenue above the baseline before additional efficiency
revenue can be paid to Accenture. This requirement is consistent with the contract’s
performance-based intent because it limits payments to Accenture until BRITS
improves the Department’s enforcement functions’ efficiency.
Efforts being made to increase enforcement revenue—Although BRITS
appears to be generating efficiency revenue again (see Figure 1, page 21), the
Department and Accenture are taking steps to further improve overall enforcement
revenue. Specifically, department officials indicate that the Department has
dedicated staff to review bills and work with Accenture to resolve BRITS problems
that cause billing inaccuracies. In the meantime, the staff also develops temporary
solutions to allow bills to be sent to taxpayers.
The State accrues
interest when
enforcement revenues
are low and
Accenture invoices
remain unpaid.
Office of the Auditor General
page 23
1 This status report was added as a requirement by the Information Technology Authorization Committee (ITAC) during its
review of the BRITS Project Investment Justification (PIJ) document. ITAC is composed of representatives from the
Legislature, private industry, local and federal government, state agencies, and the courts, and is responsible for
reviewing and approving state agencies’ information technology projects costing more than $1 million. GITA and ITAC
require agencies to prepare a PIJ for projects under review, which is a business case that reports the project’s business
and technical requirements, value to the public, costs, scope, risks, and information on the Department’s management
and technical skills.
2 The BRITS Oversight Committee meets periodically to discuss the progress of all active BRITS projects. Committee
members include an economist and representatives from the Department, Accenture, GITA, various state agencies, the
Arizona League of Cities and Towns, and Pinal County.
The Department has also developed the BRITS Discovery Team, charged with
increasing enforcement revenue. This team, staffed by department and Accenture
employees, focuses on improving existing BRITS discovery programs and creating
and implementing new ones. According to the team’s administrators, as of June
2005, the team was in the process of implementing two new discovery programs.
The team’s administrators indicate that the team’s ultimate goal is to consistently
generate $750,000 of additional enforcement revenue per month.
Department should continue to provide detail on
enforcement revenue
Until recently, the Department was not reporting adequate detail about BRITS’
enforcement revenue. The Department is required to prepare and submit a monthly
project status report to the Government Information Technology Agency (GITA), and
the Department also distributes the report to its Joint Legislative Budget Committee
analyst and members of the BRITS Oversight Committee.1,2 This report includes
information on project costs and payments to Accenture. Although no requirements
were established for the information that the report should include, prior to June
2005, the report combined the amounts of each revenue type into one amount and
compared that figure to the amount of revenue BRITS was expected to earn on a
quarterly basis. The report did not include information about negative efficiency
amounts that the Department incurred and the fact that the cumulative negative
efficiency amount must be earned back before Accenture can be paid any future
efficiency revenue. Therefore, the Department was not providing a complete picture
of BRITS-generated enforcement revenue. To its credit, the Department began
reporting enforcement revenue types separately and disclosing information about
negative efficiencies in its June 2005 report in response to auditor recommendations.
In June 2005, the
Department improved
its reporting of
enforcement revenues
and negative
efficiencies.
Recommendations
1. The Department should continue to work toward improving BRITS-generated
revenue for the remainder of the contract term. Specifically, the Department
should continue to work with Accenture to:
a. Resolve BRITS’ inability to generate accurate collections notices, thereby
increasing efficiency revenue, and
b. Improve existing BRITS discovery programs and identify and implement
new programs.
2. The Department should continue to report the following information about BRITS
enforcement revenue in its monthly Project Status Report to GITA:
a. The amounts of discovery revenue, license compliance baseline revenue,
and efficiency revenue separately,
b. Any negative efficiency amounts incurred, and
c. Any cumulative negative efficiency amount and the fact that the amount
must be earned back before Accenture can be paid from future efficiency
revenue.
State of Arizona
page 24
Office of the Auditor General
page 25
Department should better ensure contract
changes are appropriate
In an attempt to administer the contract under its indefinite-quantity terms, the
Department has made several changes to the contract that may affect its overall
scope. For example, to keep overall contract costs from increasing when it decided
to add approximately $7 million in services, the Department decreased the monies
available for certain other tasks. To help ensure that these changes were in the
State’s best interest and within legal requirements, such as contract terms and
procurement rules, department personnel in charge of the project should have
discussed the changes with department and state contracting officials. However,
they did not do this or adequately document the changes until auditors brought the
need to do so to their attention. The Department should improve its communication
and documentation of contract changes to help ensure the changes are
appropriate.
Contract changes may affect BRITS’ overall scope
The State’s central procurement authority, Enterprise Procurement Services (EPS),
has the overall authority for the BRITS contract. However, in an effort to administer
the contract, the Department has made several changes to the contract by ordering
additional services and eliminating others. These changes may affect the contracts’
overall scope.
EPS has overall contract authority—Although the Department plays a key role
in shaping the BRITS project, the actual BRITS contract is between Accenture and
EPS. Therefore, any amendments to the overall contract scope must be made by
EPS. However, EPS has delegated the contract’s day-to-day administration to the
Department, including the role of determining and ordering the specific products
and services necessary to meet the system requirements.
BRITS is an indefinite-quantity contract, meaning the project’s specific
components—such as products and services, and timing of work—could only be
FINDING 3
Changes to any
services or products
that affect the overall
contract scope cannot
be authorized without
EPS amending the
contract.
State of Arizona
page 26
estimated when the contract was awarded. The contract was designed to meet the
numerous system requirements outlined in the State’s request for proposals (RFP) or
added during the procurement process. Accenture organized these requirements
into 21 tasks, plus various “optional” tasks the State may choose to order.1 Accenture
also provided an estimate of the products and services that would be needed to
implement the tasks, with the understanding that the specific products and services
needed would be finalized when they could be better defined. To that end, BRITS’
indefinite-quantity terms give the State the flexibility to determine the specific
components of each task as implementation nears. However, changes to any
services or products that affect the overall contract scope cannot be authorized
without EPS amending the contract.
Department has made changes to the contract—In an effort to administer
the contract under its indefinite-quantity terms, the Department has ordered some
additional services related to the contract and eliminated others. In all, the
Department had added approximately $7 million of services to the contract as of
March 2005. Specifically, the Department added one new task with an estimated
price of approximately $6.4 million for Accenture to provide the off-site hosting,
configuration, and maintenance of the BRITS server and network hardware (i.e., data
center), which are the components needed to process BRITS’ data.2 Although a data
center and related services are necessary for BRITS to operate, using Accenture to
perform the offsite hosting and services was an optional task under the BRITS
contract and was not included in the $122.65 million contract price.3 The Department
also ordered additional services within some of the existing tasks. For example, it
added technical support and training to the Compliance Data Warehouse task,
thereby increasing the task’s estimated price by $250,000.4
Initially, in an effort to maintain a total contract price of $122.65 million despite these
additions, the Department decreased the amount of monies available for other
tasks. Specifically, the Department planned to pay for most of the $7 million worth
of additions by reducing the amount of monies available for one broadly defined
The Department added
$7 million worth of
services to the contract
as of March 2005.
1 Optional tasks are not part of the $122.65 million contract price. The tasks include products or services offered to
enhance the core system or assist with operating and maintaining the system, and may be ordered based on a joint
agreement between the State and Accenture.
2 The Department executed a 4-year data center agreement with Accenture in December 2003. As of May 2005, the
Department had researched the costs and feasibility of terminating the agreement early and transferring the data center
to the Department of Administration. According to a department official, it would not be in the State’s best interest to
terminate the agreement at this time because of early-termination penalties and because the Departments of Revenue
and Administration do not yet have the capability to support the data center. However, the official indicated that the
Department intends to continue to evaluate its data center options over the remaining life of the current agreement.
3 Accenture originally proposed that it be responsible for the offsite data center hosting and related services. However, at
the time of the contract’s award, the State had not determined whether it would use Accenture for those services.
Therefore, the data center task was included as an optional task within the BRITS contract.
4 The Compliance Data Warehouse will be a repository for data from various sources, internal and external to the
Department, to be used in the Department’s audit and collection efforts.
Office of the Auditor General
page 27
task, known as “Additional Backfill Support.” Although not specifically defined in the
contract, the scope of the Additional Backfill Support task appears to include hiring
temporary employees to help maintain the Department’s productivity during BRITS’
implementation (such as processing backlogged tax returns), and ordering
additional technical support from Accenture during implementation if needed (such
as fine-tuning databases). Other tasks were also reduced by smaller amounts, such
as Skip Tracing, which identifies the most accurate address of taxpayers with
liabilities and Management Support, which is essentially Accenture’s project
management responsibilities.
One of the Department’s project managers also signed a memorandum from
Accenture on December 10, 2004, indicating she agreed that 5 of the original 534
system requirements were not covered under the contract’s $122.65 million target
price. These requirements were (1) Optical/Intelligent Character Recognition to help
improve data input; (2) ability for taxpayers to input filing closures and address
changes via phone; (3) Spanish-language option for the Integrated Voice Response
system for customer inquiries; (4) Web-based chat and video-conferencing for
customer education; and (5) automatic phone dialer system to assist with
collections. The Department’s procurement office was not made aware of this
agreement until auditors brought it to the office’s attention. When the office sought
clarification from Accenture, it was informed that the purpose of the agreement was
to clarify that the first four requirements were optional in the contract (i.e., not part of
the $122.65 million price), and the Department chose not to implement the fifth
requirement.
Changes may affect the contract’s overall scope—Although the contract’s
indefinite-quantity terms allow some flexibility in determining the specific products
and services used in the contract, certain changes may impact the contract’s overall
scope, and therefore BRITS’ price and/or functionality. The contract price is based on
the estimated scope of each task. In other words, if the actual scope of each task
remains consistent with Accenture’s original, overall estimate, then the contract price
should not exceed $122.65 million, plus interest. However, if changes are made that
increase a task’s overall scope or if new tasks are added, then the contract price may
increase as well.
As indicated above, the Department was initially attempting to maintain the contract
price by deceasing the amount of monies available for some tasks when services or
tasks were added. While this method may keep the contract price within $122.65
million, reducing tasks may also result in the State’s having to: (a) forego the related
products or services, possibly resulting in a lesser or incomplete system, or (b)
amend the contract and/or procure any remaining products or services needed,
ultimately increasing the contract’s price. For example, according to auditor analysis
of department information, as of March 2005, the Department had used
State of Arizona
page 28
approximately $6 million of the $14 million available for the Additional Backfill Support
task to pay for most of the contract changes discussed above, and had used the
remaining amount for actual backfill support. As a result, should the State require
backfill support in the future, it will have to amend the contract and/or procure the
amount needed.1 The changes made so far will not likely affect the system’s overall
quality or performance. However, if the Department subsequently reduced the
amount of monies available for core tasks, such as developing the main database to
handle individual income tax functions, then the system’s quality or performance
could be impacted.
Better disclosure and documentation of contract changes
needed
As the Department performs its contract administration role, it should better ensure
that contract changes are appropriate by clearly communicating and documenting
contract changes. Specifically, to ensure changes are within legal requirements and
in the State’s best interest, the Department should discuss proposed changes with
its procurement office and EPS. In addition, to help ensure the contract reflects the
true scope of the project, the Department should improve its documentation of
changes to include a description of how each change affects the scope of the tasks
involved.
Department should discuss proposed changes with procurement
experts—Since the contract changes could affect BRITS’ overall scope, the
Department should consult procurement experts before making changes. As
discussed above, one of the Department’s project managers signed an agreement
with Accenture that some of the original system requirements in the contract are not
within the contract’s original, overall scope. However, the Department’s procurement
office was not made aware of the agreement until auditors brought it to their attention.
In addition, although a department official indicated that the Department had
informed EPS of the contract changes discussed above, the Department did not
maintain documentation related to the discussions, and EPS was unable to verify the
official’s assertion.
It is important to consult procurement experts regarding contract changes to ensure
changes are in the State’s best interest and within legal requirements, such as
contract terms and procurement rules. Specifically, EPS indicated that it should be
involved with changes, such as those already made, that could potentially affect the
1 As indicated on page 29, the Department recently requested that EPS increase the contract price by $6.4 million to
include the data center. If EPS increases the price, the Department could again have monies available for additional
backfill support. Specifically, the Department would no longer have to pay for the data center by reducing the amount of
monies available for the Additional Backfill Support task from which the majority of the data center was originally going
to be paid.
Office of the Auditor General
page 29
contract’s overall scope to consider whether the changes are necessary, ensure that
any changes are within legal requirements, and determine whether the additional
items the Department wants should be competitively bid. Therefore, the Department
should more consistently consult its procurement office and EPS about proposed
changes and document the related discussions. If the Department believes that
proposed changes clearly will not affect the overall contract scope and therefore do
not warrant EPS’ involvement, the Department should at least document the
reasoning behind its determination.
To its credit, the Department has already begun taking steps to improve in this area
since the audit. For example, on May 17, 2005, the Department sent a memorandum
to EPS outlining most of the contract changes that had already been made and
requesting that EPS increase the overall contract price by approximately $6.4 million
to include the data center costs. If made, this change would increase the overall
contract price to approximately $129 million. Additionally, a representative from the
Department’s procurement office began attending meetings of the internal BRITS
Oversight Committee in March 2005. Executive-level staff from the Department and
Accenture attend these on-going meetings that consist of updates and discussions
of BRITS’ project-management issues.
Department should improve documentation of changes made—The
Department should also improve its documentation of changes that are actually
made to the BRITS contract. For the $7 million worth of changes already made, the
Department had not adequately documented the changes. Specifically,
documentation for each change either did not exist or did not fully explain how the
change affected the tasks involved. For example, on December 11, 2003, the
Department increased the scope of the Compliance Data Warehouse task by
reducing the price of the Skip Tracing task by $250,000.1 However, the initial
documentation did not indicate how the price reduction changed the level of Skip
Tracing services to be provided.
To ensure changes affecting the project’s scope are identified and incorporated into
the contract, the Department should improve its documentation of contract changes,
as it did in its May 17, 2005, memorandum discussed above. Best practices indicate
that changes to such areas of the contract as scope, price, and implementation
schedules, even if those areas are only estimates, should be thoroughly
documented.2 Such documentation should explain the change being made, why it is
being made, and whether the change will impact the scope of the tasks involved or
the overall project.
1 Under the Skip Tracing task, Accenture uses an Internet-based tool to determine the most accurate mailing address for
taxpayers with outstanding liabilities, but for whom previously mailed bills have been returned.
2 (1) IT Governance Institute. 2004. “Control Objectives for Information and Related Technology (COBIT) Online, v3.1.”
www.itgi.org (Nov. 19, 2004) and (2) Project Management Institute, Inc. A Guide to the Project Management Body of
Knowledge, 3rd ed. Pennsylvania: Project Management Institute, Inc., 2004.
In May 2005, the
Department requested
that EPS increase the
overall contract price by
approximately $6.4
million to include data
center costs.
Recommendations
1. To ensure that contract changes are in the State’s best interest and within the
legal requirements, such as contract terms and procurement rules, the
Department should:
a. Consult its procurement office and EPS about proposed changes and
document the related discussions, or
b. If the Department believes proposed changes clearly will not affect the
overall contract scope and therefore do not warrant EPS’ involvement,
document the reasoning behind its determination.
2. To ensure that changes affecting the project’s scope are identified and
incorporated into the contract, the Department should prepare documentation
for contract changes that clearly explains the change being made, why it is
being made, and whether the change will impact the scope of the tasks involved
or the overall project.
State of Arizona
page 30
Office of the Auditor General
AGENCY RESPONSE
State of Arizona
Page 1 of 8
STATE OF ARIZONA
Department of Revenue
Office of the Director
(602) 716-6090
Janet Napolitano
Governor
Gale Garriott
Director
1600 West Monroe Street, Phoenix AZ 85007-2650 www.azdor.gov
October 07, 2005
Debbie Davenport, Auditor General
Office of the Auditor General
2910 North 44th Street, Suite 410
Phoenix, Arizona 85018
Dear Mrs. Davenport:
The Department of Revenue (Department) has reviewed the September 21, 2005 Business
Reengineering/Integrated Tax System (BRITS) performance audit report. The Department
commends and thanks your staff for their understanding and professionalism throughout
the performance audit process.
The following comments are provided as the Department’s response to the findings and
recommendations.
Finding 1 – Department needs to better manage BRITS project
Recommendation 1 – To take advantage of expert project management advice, the
Department should consider hiring an outside advisor to oversee the remaining
implementation of BRITS and fulfill the outside advisor’s requirements that are described in
the BRITS contract. To do so, the Department would have to examine possible funding
methods, such as allocating monies from its current budget, or seeking a separate
appropriation from the Legislature.
The finding of the Auditor General is agreed to and the audit recommendation will be
implemented.
The Department will seek a supplemental appropriation from the Legislature for the costs
associated with selecting and employing an outside advisor. In addition to the expertise an
outside advisor may provide, the Department would note that the current Chief Information
Officer and BRITS Project Manager, both of whom are relatively recent hires, have over 30
years of combined experience in IT Project Management and were both recently certified
as Project Management Professionals (PMP) by the Project Management Institute (PMI).
Recommendation 2 – To better ensure success and to ensure that the terms of the BRITS
contract are fulfilled, the Department should develop and follow its own overall plan to
manage the project.
The finding of the Auditor General is agreed to and the audit recommendation will be
implemented.
The Department began drafting an internal plan for management of the BRITS Program in
June 2005. The plan will comply with project management best practices. To allow for
Page 2 of 8
adequate review and comment from the Department Leadership team, the target date for
sign off on the completed plan is October 31, 2005.
Recommendation 3 – To prevent some of the issues that occurred in the initial phases of
BRITS, the Department should continue its efforts to include key stakeholders in better
defining the Department’s requirements for the BRITS system and ensuring that both the
Department and Accenture agree on what those requirements are until the BRITS project is
complete.
The finding of the Auditor General is agreed to and the audit recommendation will be
implemented.
The Department has already implemented a number of changes to ensure the system
requirements are adequately defined. The changes implemented include:
a. Improved Requirement Definition - Between March 2005 and June 2005, the
Department Leadership team initiated a series of meetings with Accenture to review all
534 requirements from the original BRITS proposal. The meetings resulted in a
common understanding of the requested functionality and an identified program phase
for delivery of each requirement.
b. Requirement Sign Off - System requirements are signed off by all key Department
stakeholders. In addition, more detailed process flows and user specifications are being
developed. For example, sample screen designs and report layouts are provided to the
users allowing them to visually inspect the proposed changes to the system.
c. Requirement Traceability - A detailed Requirement Traceability Matrix (RTM) for
tracking requirements through all phases of the project and into production is in place.
d. Requirement Impact Analysis - Cross-functional teams comprised of subject matter
experts from across the Department review project deliverables to ensure the requested
changes are appropriate.
e. Key Deliverable Approval - The Steering Committee - comprised of the Department
Director, Deputy Director, and the Assistant Directors with operational responsibility -
sign off on all key project deliverables.
f. Acceptance Criteria - The Project Manager will be working with key stakeholders to
define specific acceptance criteria for each remaining phase of the program.
g. Operational Readiness Assessment - A Readiness Plan has been developed and
approved. The plan calls for active Steering Committee involvement in the
communication and readiness assessment activities.
Recommendation 4 – To remain aware of and plan for project risks, the Department
should continue its efforts to identify, document and mitigate risks until the BRITS project is
complete.
The finding of the Auditor General is agreed to and the audit recommendation will be
implemented.
The Department understands the importance of risk management and intends to continue
with its current practice of reviewing program risks monthly. Further, an additional step has
been added to the process to further ensure risks/impacts are identified and mitigation
Page 3 of 8
plans developed as early as possible in the project. During the functional design activities
for the Corporate Tax phase of the BRITS program, the project team identified and
captured potential operational impacts. The impacts identified (resources, training, budget,
etc.) were summarized and presented to the Steering Committee members on August 9,
2005 and planning for those impacts has begun.
Recommendation 5 – To make certain that all necessary functions of the system are
working prior to conversion, the Department should ensure BRITS testing is adequate,
including:
a. Getting system users involved with testing as early as possible, and
b. Making sure that an adequate amount of time is taken to perform testing, and correct
and retest any problems.
The finding of the Auditor General is agreed to and the audit recommendation will be
implemented.
In the first phase of the BRITS program, issues with system stability delayed release of the
system to users for testing and resulted in compressed testing schedules. The BRITS
Corporate and Individual Income tax data conversions will include active user involvement
in the testing cycles to identify issues as early as possible.
In addition, a full scale system test will be conducted by the project team prior to user
testing. A system test is intended to identify critical system issues that may impact stability
prior to delivery of the software to end users. The Corporate Income tax timeline was
specifically developed to reduce overlap between system and user testing and to maximize
testing and repair time.
In order to apply lessons learned from the TPT and Withholding tax conversions, the
Corporate and Individual Income tax conversions will incorporate the following additional
controls:
a. Acceptance Criteria - Pre-defined acceptance criteria will be established to control and
measure the data conversion process. These criteria will include elements to evaluate
the success rate of data conversion from Legacy systems, and will set a minimum
quality standard that must be met before production data conversion will be authorized.
b. Monitoring of Conversion Issues - Conversion issues, including known issues with
Legacy data will be identified and managed so that alternatives can be evaluated, and
actions taken to eliminate or minimize the risk associated with the conversion issues.
c. Go/No Go Decision Point - Recognizing that adequate test time is critical to the success
of a conversion, if the minimum acceptance criteria have not been satisfactorily
achieved, the conversion will be delayed and testing will continue until such time as the
criteria have been met. The BRITS Steering Committee will approve or disapprove the
final conversion based on the acceptance criteria and the list of outstanding conversion
issues.
Recommendation 6 – To ensure that all Department staff can perform their jobs using
BRITS, the Department should continue to ensure that future training will address previous
training problems and will allow the Department to effectively use, operate, and maintain
BRITS.
Page 4 of 8
The finding of the Auditor General is agreed to and the audit recommendation will be
implemented.
Several changes aimed at improving the overall quality of current and future phases of the
BRITS program will also increase the quality of the training activities. The relevant changes
to date include:
a. Improved Requirement Definition – The better the requirement documentation is on a
project, the easier it is to develop training that clearly outlines the impact of the
changes.
b. Approval to Proceed – The BRITS Steering Committee will review relevant project
artifacts – deliverables, issues, etc. to determine whether or not to proceed to the next
step of the current phase. If the system is not stable, approval will not be given to
proceed to User Training until necessary repairs are complete.
c. Manual Procedure Development – In addition to completing detailed system
documentation, the project team is also producing detailed manual procedures for each
Division. These procedures form a strong basis for development of detailed training
curriculum. In addition, all manual procedures will be loaded into the system on-line
Help function for future reference after the training has been completed.
d. Go/No Go Decision Point - The BRITS Steering Committee will approve or disapprove
the conversion for each of the remaining tax type conversions (i.e., Corporate and
Individual Income tax) based upon predefined acceptance criteria. User training will be
a key element of the acceptance criteria. If the minimum acceptance criteria have not
been satisfactorily achieved, the conversion will be delayed.
Recommendation 7 – To ensure the BRITS system is secure and sensitive data is
protected, the Department should:
a. Complete and implement policies and procedures that govern system access controls;
audit trail creation and analysis; data encryption management, virus protection, and
intrusion detection and prevention; and,
b. Perform security assessments periodically to determine if its policies and procedures
are followed and adequately ensure system security.
The finding of the Auditor General is agreed to and the audit recommendation will be
implemented.
The Department understands the importance of properly controlling access to computer
resources and is in the process of assembling a Security Manual which will contain all of
the key processes and procedures related to data security. The manual will also include the
method and timing of security assessments to ensure policies and procedures are adhered
to and systems are adequately protected from unauthorized access.
Historically, the Department has conducted an annual review of data access privileges
across all systems and users. The new applications implemented for the BRITS project will
be included in the annual review scheduled for October 2005. In addition, the Department
engages Cisco Systems, Inc. to conduct an annual Security Posture Assessment on the
network. This assessment tests for network vulnerabilities. In the most recent assessment,
Page 5 of 8
conducted in June 2005, there were no high risk vulnerabilities identified and all
vulnerabilities identified in the previous year’s report had been resolved.
In October 2004, the Department recognized the need for a centralized data security role in
the newly implemented BRITS environment and established a Security Officer position.
Over the period of October 2004 to January 2005, the Department’s IT Division assumed
responsibility for BRITS data security from various members of the BRITS project team.
Since January 2005, the IT Division has conducted two Security Summit meetings to
review and evaluate existing security processes and procedures. As a result of these
meetings, we have updated, improved, and streamlined our data security processes and
tools. In addition, the Department continues to review and refine user access privileges
across all Divisions.
Recommendation 8 – To help with the planning and execution of future phases of the
project, the Department should develop and follow a formal post-implementation review
plan, as stated in the BRITS contract, that outlines how and when during the project it will
perform reviews.
The finding of the Auditor General is agreed to and the audit recommendation will be
implemented.
The Department will work with Accenture to develop a formal post-implementation review
plan. The plan will document the review activities already in place from the first phase of
the program as well as those additional activities planned for future phases. The reviews
planned for the Corporate and Individual Income tax phases are as follows:
a. Approval to Proceed - A “Gate” review meeting will be conducted with Steering
Committee members at predefined intervals for the remaining phases of the BRITS
program. The purpose of a “gate” meeting is to gain formal approval for the project team
to proceed to the next step of the current phase; i.e. functional design to technical
design. Following are examples of discussion topics for the gate meetings:
a) The completion status of deliverables for the current phase.
b) The status of open issues. The intent is to discuss critical issues that may
jeopardize future project phases.
c) Outstanding project risks and associated mitigation plans.
b. Go/No Go Decision Point - The BRITS Steering Committee will approve or disapprove
the conversion for each of the remaining tax type conversions (i.e. Corporate and
Individual Income tax) based upon the acceptance criteria and the list of outstanding
conversion issues. If the minimum acceptance criteria have not been satisfactorily
achieved, the conversion will be delayed and testing will continue until such time as
they have been achieved.
c. Lessons Learned - A lessons-learned meeting will be conducted after each of the major
BRITS program phases.
Recommendation 9 – To increase the chances of success, until the project is complete,
the Department should continue to hold lessons-learned sessions at appropriate points
during the project and should document and use the results of these meetings to improve
BRITS’ implementation.
Page 6 of 8
The finding of the Auditor General is agreed to and the audit recommendation will be
implemented.
A lessons learned meeting was conducted after the initial phase of the BRITS program.
The results of that session were presented to the BRITS Steering Committee for review
and consideration. As described in the Department response to Recommendation 3 above,
several improvements have already been implemented in the current phase of the
program. As outlined in the Department response to Recommendation 8 above, a lessons-learned
meeting will be conducted after each of the major BRITS program phases.
Finding 2 – BRITS not generating additional revenue as expected
Recommendation 1 – The Department should continue to work toward improving BRITS-generated
revenue for the remainder of the contract term. Specifically, the Department
should continue to work with Accenture to:
a. Resolve BRITS’ inability to generate accurate collections notices thereby increasing
efficiency revenue, and
b. Improve existing BRITS discovery programs and identify and implement new programs.
The finding of the Auditor General is agreed to and the audit recommendation will be
implemented.
With regard to the generation of collections notices, or more appropriately, billings, the
Department acknowledges that there was a significant delay in this process after the initial
conversion to the BRITS system, and that a backlog did develop. This delay affected
processes further down the line, such as staging bills to Collections. The delay was
primarily caused first by programming delays and then by the need to manually review the
data to validate the accuracy of the billings.
Accenture and DOR met in March 2005 and agreed to dedicate a team of developers to
focus on billing and collections related issues in order to improve the collection of efficiency
revenue. In addition, the Department dedicated a team of users to review a sample of bills
from weekly billing runs to identify system errors.
As of August 15, 2005, the development team had made over 90 improvements in the
areas of Taxpayer Accounting and Collections. In addition, more rigorous issue
prioritization, release management, and testing processes implemented in March 2005
have helped further stabilize the production system. As incremental programming
improvements move to the production environment, fewer bills need manual review and
backlogs are reduced. The Department acknowledges that improvements to this process
have and will continue to improve the flow of Enforcement Revenue, and, as a result,
Efficiency Revenue.
Currently in BRITS, the Department is issuing billings for Transaction Privilege Tax on a
weekly basis. Billings for Withholding are being issued but a backlog remains with a goal of
being current by December 31, 2005. In conjunction with the billing, receivables are
automatically staging to Collections for further collections activities in accordance with
Department policies. Although BRITS does not currently issue delinquency notices in
mass, delinquencies are being referred to Collections for further collections activities on a
monthly basis and the specific collector can manually issue a delinquency notice when
Page 7 of 8
working the case if he or she chooses. Work continues on automated issuance of
delinquency notices, and it is expected to be completed before the end of calendar year
2005.
With regard to the improvement of existing BRITS Discovery programs and the
implementation of new programs, the Department acknowledges that the BRITS system
and the tools that have been provided with it offer new opportunities for discovering non-filers
of all tax types. With this in mind the Department established a BRITS Discovery
Team, as a part of the audit staff, with the sole focus of initiating and managing Discovery
programs.
The Department acknowledges that there are potentially many methods that can be used
to find non-filers and bring them into compliance. As the Corporate and Individual Income
taxes come into the BRITS system, and more integrated data becomes available to the
Department, the opportunities for Discovery will improve. Discovery Benefits are up
considerably over the last several months averaging more than $500K per month since
October 2004. The July 2005 revenues are reported at over $1.4 million. The Discovery
team continues to develop new Discovery programs and the Department has committed to
adding three additional resources to support the on-going management of the programs.
Recommendation 2 – The Department should continue to report the following information
about BRITS enforcement revenue in its monthly Project Status Report to GITA:
a. The amounts of discovery revenue, license compliance baseline revenue, and efficiency
revenue separately,
b. Any negative efficiency amounts incurred, and
c. Any cumu