Performance audit, Arizona Department of Transportation, aspects of construction management

              Performance Audit
Arizona Department of
Transportation–
Aspects of Construction
Management
Performance Audit Division
Debra K. Davenport
Auditor General
JULY • 2006
REPORT NO. 06 – 05
A REPORT
TO THE
ARIZONA LEGISLATURE
The is appointed by the Joint Legislative Audit Committee, a bipartisan committee composed of five senators
and five representatives. Her mission is to provide independent and impartial information and specific recommendations to
improve the operations of state and local government entities. To this end, she provides financial audits and accounting services
to the State and political subdivisions, investigates possible misuse of public monies, and conducts performance audits of
school districts, state agencies, and the programs they administer.
The Joint Legislative Audit Committee
Representative Laura Knaperek, Chair Senator Robert Blendu, Vice Chair
Representative Tom Boone Senator Ed Ableser
Representative Ted Downing Senator Carolyn Allen
Representative Pete Rios Senator John Huppenthal
Representative Steve Yarbrough Senator Richard Miranda
Representative Jim Weiers (ex-officio) Senator Ken Bennett (ex-officio)
Audit Staff
Melanie Chesney, Director
Shan Hays, Manager and Contact Person
Brent Nelson, Team Leader
Lori Babbitt
Mark Haldane
Copies of the Auditor General’s reports are free.
You may request them by contacting us at:
Office of the Auditor General
2910 N. 44th Street, Suite 410 • Phoenix, AZ 85018 • (602) 553-0333
Additionally, many of our reports can be found in electronic format at:
www.azauditor.gov
DEBRA K. DAVENPORT, CPA
AUDITOR GENERAL
STATE OF ARIZONA
OFFICE OF THE
AUDITOR GENERAL
WILLIAM THOMSON
DEPUTY AUDITOR GENERAL
2910 NORTH 44th STREET • SUITE 410 • PHOENIX, ARIZONA 85018 • (602) 553-0333 • FAX (602) 553-0051
July 20, 2006
Members of the Arizona Legislature
The Honorable Janet Napolitano, Governor
Mr. Victor Mendez, Director
Arizona Department of Transportation (ADOT)
Transmitted herewith is a report of the Auditor General, A Performance Audit of the Arizona
Department of Transportation, Aspects of Construction Management. This report is in
response to a May 24, 2005, resolution of the Joint Legislative Audit Committee. The
performance audit was conducted as part of the sunset review process prescribed in
Arizona Revised Statutes §41-2951 et seq. I am also transmitting with this report a copy of
the Report Highlights for this audit to provide a quick summary for your convenience.
As outlined in its response, ADOT agrees with all of the findings and plans to implement all
of the recommendations.
My staff and I will be pleased to discuss or clarify items in the report.
This report will be released to the public on July 21, 2006.
Sincerely,
Debbie Davenport
Auditor General
Enclosure
The Office of the Auditor General has conducted a performance audit of the Arizona
Department of Transportation (ADOT) pursuant to a May 24, 2005, resolution of the
Joint Legislative Audit Committee. This is the first in a series of three reports on ADOT
and was conducted as part of the sunset review process prescribed in Arizona
Revised Statutes (A.R.S.) §41-2951 et seq. This audit focuses on the Intermodal
Transportation Division’s (ITD) use of consultants to design and manage construction
projects, the process for inspecting projects under construction, and the audits
conducted by ADOT’s Office of Audit and Analysis on consultant and construction
contracts. The other audit reports will focus on the highway maintenance program
and the 12 statutory sunset factors.
ADOT was established in 1974 to plan, develop, maintain, and operate Arizona’s
highway transportation infrastructure to move people and goods by surface and air
throughout the State. ITD relies on private consultants to help design and contractors
to construct highway projects. In fiscal year 2005, ADOT paid consultants $110
million for transportation corridor-related studies, pre-design, design, and
construction administration. In fiscal year 2005, ITD awarded 126 construction
contracts valued at $510 million. ITD field inspectors and independent quality
assurance (QA) inspectors inspect construction work to verify quality construction
and compliance with specifications. ADOT internal auditors review consultant and
construction contracts to verify that payments are proper.
ADOT should optimize internal resources to reduce
consultant usage (see pages 9 through 17)
The Intermodal Transportation Division (ITD), which plans, designs, constructs,
maintains, and operates the State’s highway transportation infrastructure for the
movement of people and goods throughout Arizona, should evaluate consultant
usage and complete project design, construction management, and other similar
functions in-house when appropriate to control costs and maintain employee core
competency levels. ITD has had to meet an increased workload, including an
accelerated urban freeway construction program that reduced 14 years of
Office of the Auditor General
SUMMARY
page i
construction to 7-½ years, while simultaneously dealing with reduced FTE counts,
vacancies, and an inability to fill many of its engineering and technical positions.
ADOT attributes these vacancies to employees retiring or leaving for higher salaries
offered by private consulting firms and local governments. For example, ITD
surveyed nine consulting firms that provide services to ITD and found that 45 percent
of these consultants’ employees were former ADOT employees. Also, in November
2005, an ADOT salary comparison found that even after a recent 5 percent pay
increase, ADOT engineer salaries were 13 to 26 percent lower than comparable
private and public positions in the Phoenix area. The amount of work planned in
ADOT’s 5-year construction program more than doubled in the past 10 years, but ITD
has been unable to fill nearly one-fourth of its engineering positions. As a result, ITD
has substantially increased its use of private consultants to supplement its staff in
providing project design, construction management, inspection, and other services.
ADOT’s payments to consultants increased 424 percent after adjusting for inflation,
with payments increasing from $17 million in fiscal year 1996 to $110 million in fiscal
year 2005. In June 2005, the Governor’s Efficiency Review Team reported that ITD’s
consultant spending and usage was more than all other state agencies combined.
ITD must use consultants because of its workload and the difficulty in hiring and
retaining experienced staff. However, filling vacant positions and completing more
project design, construction management, and other similar functions in-house can
reduce reliance on consultants and maintain and develop internal core
competencies. One risk of high consultant usage as demonstrated in professional
literature is that consultants can cost more than internal staff for design work. Another
is that excessive consultant usage can reduce internal staff competence. ITD’s
engineering employee experience levels have declined in recent years. ITD has
attempted to stem its turnover and vacancy rates by making counter-offers to some
employees who receive offers of higher-salary jobs and by higher utilization of an
Engineer in Training program to attract new hires. Additional actions needed include
establishing criteria to evaluate whether consultants are necessary and maintaining
adequate management information to evaluate consultant usage and identify where
project design, construction management, and other similar functions could be more
appropriately provided by lower-cost, in-house staff.
ITD should improve implementation and documentation
of inspection process (see pages 19 through 25)
To ensure contractors meet construction standards for highway projects, ITD should
improve construction inspection quality. ADOT employs more than 220 field
inspectors who visit construction sites daily as a means of ensuring construction
quality and compliance with specifications. As an additional quality control measure,
ITD independent QA inspectors inspect the same construction sites at least once
State of Arizona
page ii
using the same inspection standards. However, auditors identified three problems
with the inspection and review processes:
􀁺􇩄 Documentation of inspection results is incomplete—ITD inspectors do not
consistently document the results of their observations. For example, auditors’
review of 9 projects determined that 43 of 47 inspectors’ diaries showed the type
of work observed, but not whether the work met specifications. Lack of
documentation may affect ADOT’s ability to determine the progress and quality
of work and to identify problem areas and determine if sufficient action has been
taken to resolve identified problems. Additionally, 27 of 47 inspectors whose
records were reviewed did not fill out any of the required checklists, which are
based on standard specifications and important to ensure the product meets
quality requirements of workmanship and testing. To comply with ADOT policy
and help ensure the quality of work inspected, ADOT should ensure that field
inspectors complete and submit checklists as part of their daily diaries, and that
diaries document work quality, problems found, and problem resolutions.
􀁺􇩉 Inspection standards are not consistently applied—When ITD’s independent QA
inspectors conduct periodic reviews, they appear to apply a stricter
interpretation of the standards than field inspectors do during their daily visits.
Auditors’ analysis of two construction projects where both field inspectors and
independent QA inspectors conducted inspections during July 1, 2005 through
December 19, 2005, found that field inspectors determined work met
specifications 66 percent of the time, while independent QA inspectors found
work met specifications only 35 percent of the time. ITD should ensure that
checklist results are consistent among field inspectors and independent QA
inspectors by identifying reasons for differences and providing training and/or
developing guidelines to help inspectors interpret the checklist items in a similar
manner.
􀁺􇩆 Followup on important deficiencies is lacking—Although field inspectors’
findings may be resolved at once, making followup unnecessary, ITD has not
developed any follow-up procedures for deficiencies identified by the
independent QA inspectors even when they identify critical or major
noncompliance that in some cases may potentially put human life at risk, or
have a substantial impact on operability, durability, cost, or the environment.
Auditors’ analysis of 1,970 reviews revealed that 80 percent identified one or
more of these types of noncompliance. For example, one independent QA
inspection found noncompliance with rebar spacing and size that, if not
corrected, could result in a shorter lifespan or failure of the roadway structure.
ADOT could better use the results of independent QA inspections by requiring
a followup for critical and major deficiencies.
Office of the Auditor General
page iii
ADOT needs to improve its audits of design and
construction contracts (see pages 27 through 34)
ADOT has not adequately planned and managed the audits of its highway design
and construction contracts. The Office of Audit and Analysis (Office) is required to
conduct audits covering the full range of consulting and construction contracting
practices. The Office conducts several types of audits to ensure that contractors set
rates appropriately, comply with contract requirements, and do not overcharge.
However, office productivity has diminished in recent years. The Office does not
conduct the number of audits required by its own policies and, according to internal
reports, had backlogs of each audit type. Even those contracts with the largest dollar
amounts, as much as $221 million, had insufficient audits.
Several factors have contributed to the low productivity, including ongoing, long-term
vacancies and inadequate workload planning and management. In December 2005,
the Office’s unit responsible for consultant and construction audits had vacancies in
7 of 16 positions, 4 of which had been unfilled for over 3-½ years. In addition, the
Office has not complied with its policy to develop an annual audit plan or select
construction progress audits based on a department-wide audit risk assessment.
These problems continue to exist, but they show signs of being addressed under a
new chief auditor hired in January 2006. The chief auditor has announced plans for
filling vacancies, reinstituting performance measures, revising the audit manual,
prioritizing audit projects using a risk-based approach, and obtaining an automated
audit management system. In addition to these efforts, the Office should measure
the number and types of audit requests it receives and audits it conducts, the
timeliness of its audits, and its audit results.
State of Arizona
page iv
Office of the Auditor General
TABLE OF CONTENTS
continued
1
9
9
11
12
14
17
19
19
20
23
25
27
27
29
31
32
34
Introduction & Background
Finding 1: ADOT should optimize internal resources to
reduce consultant usage
ITD relies on consultants to deliver transportation program
Workload and vacancies increase consultant use
Consultant use may have unintended consequences
ITD should take further actions
Recommendations
Finding 2: ITD should improve implementation and
documentation of inspection process
ITD monitors contractor activities
ITD can improve field inspections
Independent quality assurance inspections lack followup
Recommendations
Finding 3: ADOT needs to improve audits of design
and construction contracts
Audits required during all construction phases
Office fails to complete audits or completes them late
Several factors contribute to low productivity
Department has begun to address problems
Recommendations
page v
State of Arizona
TABLE OF CONTENTS
Endnotes:
Tables:
1 ADOT’s Project Development Process
2 Intermodal Transportation Division
Schedule of Appropriations and Expenditures, in Thousands
Fiscal Years 2004 through 2006
(Unaudited)
3 Average Years’ Experience for Selected ITD Engineer Positions
as of December 31, 2000 through February 6, 2006
4 Status of Required Audits
Ten Largest Completed Projects
Fiscal Year 2005
5 Status of External Audit Unit Positions
As of December 31, 2005
Figures:
1 ITD Engineering Districts
As of April 2006
2 Payments to Consultants
Fiscal Years 1996 through 2005
3 Payments to Construction Contractors
Fiscal Years 1995 through 2005
(Unaudited)
4 Comparison of Responses on Checklists
July 1 through December 19, 2005
a-i
2
5
14
30
31
3
10
11
22
continued
page vi
Office of the Auditor General
TABLE OF CONTENTS
concluded
1
19
24
Photos:
1 View of U.S. 60 from SR 202 overpass under construction
2 Paving the San Tan Freeway
3 Pouring a retaining wall footing
page vii
State of Arizona
page viii
The Office of the Auditor General has conducted a performance audit of the Arizona
Department of Transportation (ADOT) pursuant to a May 24, 2005, resolution of the
Joint Legislative Audit Committee. This is the first in a series of three reports on ADOT
and was conducted as part of the sunset review process prescribed in Arizona
Revised Statutes (A.R.S.) §41-2951 et seq. This audit focuses on the use of
consultants to design and manage construction projects, the process for inspecting
projects under construction, and the audits conducted on consultant and
construction contracts. The other audit reports will focus on the highway
maintenance program and the 12 statutory sunset factors.
ADOT responsible for the State’s transportation
infrastructure—ADOT was established in 1974 and is
statutorily charged under A.R.S Title 28 with planning,
developing, designing, constructing, maintaining, and
operating the State’s highway transportation
infrastructure for the movement of people and goods by
surface and air throughout Arizona. The Intermodal
Transportation Division (ITD) is primarily responsible for
designing, constructing, and maintaining the state
highway system that includes operating interstate
highways and state highway routes. ITD has highway
management activities throughout the highway life cycle,
from conceptual design and scoping to roadway
maintenance.
New highways and existing highway improvement
projects are based upon an annually updated 5-year Transportation Facilities
Construction Program approved by the seven-member State Transportation
Board. As shown in Table 1 (see page 2), highway construction projects go
through a Project Development Process, which includes all the engineering,
construction, and administrative functions required to advance a project from
conception, through design and construction, and into the operation and
maintenance of the highway. The scoping phase includes studies and analysis to
determine where and when highways should be built or improved and what
features should be included. The design phase includes project design,
Office of the Auditor General
INTRODUCTION
& BACKGROUND
page 1
Photo 1: View of U.S. 60 from SR 202
overpass under construction
Photo: Courtesy of the Arizona Department of Transportation.
environmental studies, and right-of-way acquisition. The bidding phase includes
preparing and advertising the project for bidding. After contract award, the
construction phase involves oversight of contractor work, inspecting work
progress and quality, and resolving any construction problems. Finally, when
construction projects are completed, additional work occurs during the operation
and maintenance phase. In addition, ITD district maintenance crews are also
responsible for the day-to-day maintenance of the districts’ highways.
ITD contracts for project planning, design, and
construction
ITD uses the services of private consultants and contractors. ADOT uses
independent contractors to construct roadway projects, while consultants have been
increasingly used to provide services traditionally provided by in-house employees.
State of Arizona
page 2
Table 1: ADOT’s Project Development Process
Phase Description of Activity
Scoping Corridor studies help define individual projects to meet
transportation needs. Additional studies help determine project
alignment, engineering, and environmental issues that must be
addressed; estimated costs for project development; and time
estimates necessary for project completion. Prioritized projects are
included in the 5-year program for approval by the State
Transportation Board.
Design Approved projects in the 5-year program advance to design and
preconstruction activities where a number of design, environmental,
utility, and right-of-way activities take place.
Bidding Once design is completed, the project is prepared for bidding and
awarded to a contractor to complete construction.
Construction After the contract is awarded, the contractor is responsible for
constructing the project in accordance with specifications and
contract provisions. ADOT and its consultants administer
construction contracts and conduct inspections of contractor work.
Design consultant contracts continue during the construction phase
to address any design issues.
Operation and
Maintenance
Once completed, ADOT provides maintenance to new or improved
highway facilities to ensure continued utility and useful life.
Source: Auditor General staff analysis of the ADOT Project Development Process Manual,
February 2004.
Consultants provide services such as project design, construction management, and
various engineering services during the scoping, design, and construction phases of
project development, while contractors perform construction. A national survey
reported in 2003 that virtually all state departments of transportation use contracts
with private consultants and contractors to complete some functions of highway
programs. ADOT’s Office of Audit and Analysis (Office) is responsible for auditing
many of these contracts to ensure appropriate payment, in addition to completing
other internal functions.
In fiscal year 2005, ITD awarded 126 contracts for construction projects at a value of
$510 million to contractors for building roadway projects. According to an ADOT
official, these contracts do not include monies paid to consultants who provide
design, inspection, and construction management services. Many of these
construction projects take several years to construct. For example, a construction
contract for a portion of the Loop 202 freeway in Maricopa County was awarded in
April 2002 and the project was completed in August 2004.
Organization and staffing
Both ITD and the Office in ADOT’s Transportation
Services Group have responsibilities related to
consulting and construction contracts. ITD
administers construction projects through district
offices, as shown in Figure 1. Uniquely, Phoenix,
because of its size, has two districts, one for
construction and one for maintenance, while the
other districts combine both functions into one
physical location for a total of ten districts. Each
district office is headed by a district engineer and
staffed with other employees assigned to
organizational units and sub-units called “orgs.”
As of February 2006, ITD had 2,218 FTE, of which
316 were vacant (14 percent), as follows1:
􀁺􇩉 ITD-State Engineer’s Office (23 FTE, 8
vacant)—The State Engineer’s Office is
responsible for the administration of ITD and
provides overall division support in the areas
of management, budget, personnel, and
training.
􀁺􇨠 ITD-Development (529 FTE, 84 vacant)—ITD-Development
is responsible for project
development and design. For example, once the State Transportation Board
Office of the Auditor General
page 3
Figure 1: ITD Engineering Districts
As of April 2006
Source: ADOT Web site graphic of construction and maintenance districts.
approves a highway project, ITD-Development coordinates pre-construction
engineering functions including roadway and bridge design, compliance with
environmental laws, and the design of traffic control plans. It also acquires right-of-
way needed for highway construction. According to ADOT officials, ITD-Development
designs some projects when sufficient staff is available, but
employs consultants for design work in order to meet increasing workload
demands. After design, ITD-Development is responsible for preparing
construction projects for bidding by construction contractors and overseeing the
bidding process. ITD-Development is organized into seven groups:
Environmental & Enhancement Group, Statewide Project Management Group,
Right-of-Way Group, Engineering Technical Group, Roadway Engineering
Group, Traffic Engineering Group, and the Bridge Group.
􀁺􇩉 ITD-Operations (1,234 FTE, 128 vacant)—ITD-Operations oversees roadway
construction, maintenance, testing and evaluation of materials, pavement
design strategies for new and old pavements, and emergency maintenance
response. It includes the Construction, Maintenance, Transportation Technology
and Materials groups, and eight of the ten state-wide districts. Districts are
responsible for highway operations, construction, and maintenance in their
respective geographical areas. ITD-Operations also include some of the field
inspectors who inspect daily construction work and the Construction Operations
section that conducts independent reviews of construction projects.
􀁺􇩉 ITD-Valley Transportation (421 FTE, 93 vacant)—The Valley Transportation
Program provides and coordinates pre-construction, construction, and
maintenance functions for the urban highway systems in Maricopa County. The
Valley Transportation Program contains the Phoenix Construction District, the
Phoenix Maintenance District, the Regional Freeway System Office, and the
Valley Project Management Group.
􀁺􇩔 Transportation Services Group-Office of Audit and Analysis External Audit Unit
(16 FTE, 7 vacant as of December 31, 2005)—The Office of Audit and Analysis,
an internal auditing office, conducts audits of consultant and construction
contracts to aid in managing and overseeing both design and construction
contracts, and conducts administrative audits of ADOT’s highway construction
organizations. The Office also has other audit duties related to information
technology, revenue, and third-party collections.
Budget
ITD spends most ADOT monies earmarked for its highway program. Table 2 (see
page 5) shows ITD operating budget information for fiscal years 2004 through 2006.
As shown by Table 2, ITD’s expenditures totaled approximately $220.7 million in fiscal
State of Arizona
page 4
year 2005, and estimated expenditures for fiscal year 2006 are $231.3 million. Table
2 contains the operating expenses of ITD and does not include highway construction
costs. About 48 percent of ITD’s annual operating budget is spent on salaries and
benefits, while another 43 percent is spent on other operating costs, including
utilities, landscaping, cable barrier and guardrail repair, and rest area maintenance.
In addition to its operating budget, ITD expends state monies and federal grant funds
for highway construction and improvement projects that are approved by the State
Office of the Auditor General
page 5
Table 2: Intermodal Transportation Division
Schedule of Appropriations and Expenditures, in Thousands1
Fiscal Years 2004 through 2006
(Unaudited)
2004 2005 2006
(Actual) (Actual) (Estimated)
Appropriations:
State Highway Fund 2 $190,269.0 $220,185.4 $230,730.1
Safety Enforcement and Transportation Infrastructure Fund 3 558.7 558.7 558.7
Total revenues 190,827.7 220,744.1 231,288.8
Expenditures:
Personal services and related benefits 103,613.5 106,406.2 113,692.2
Professional and outside services 4 6,824.8 5,919.8 6,766.4
Travel 1,811.7 1,936.6 2,188.0
Other operating 5 70,352.1 94,859.1 100,729.0
Equipment 7,968.8 11,602.6 7,913.2
Total expenditures 190,570.9 220,724.3 231,288.8
Excess of appropriations over expenditures $ 256.8 $ 19.8 $ -0-
1 This table includes only department appropriations and expenditures relating to the Intermodal Transportation Division’s
operating budget and is presented on a budgetary basis, in which expenditures are reported in the budget year incurred.
2 Consists of the Division’s portion of the Department’s appropriation from State Highway Fund monies used to pay for its
operations. The State Highway Fund receives monies from the Highway User Revenue Fund, and fuel and motor carrier taxes.
3 Consists of the Division’s portion of the Department’s appropriation from Safety Enforcement and Transportation Infrastructure
Fund monies used to pay for its operations. This Fund receives monies primarily from motor vehicle licenses and registration
fees.
4 Consists of payments made to external parties for services provided to the Division, such as temporary services; preliminary
engineering costs; and various consultants. For example, the Division contracts for pre-design and design, bridge inspection,
environmental, and asbestos abatement work.
5 Consists of various costs for division operations, including utilities; landscaping; cable barrier and guardrail repair; rest area
maintenance; traffic control; equipment, building, and land rental; general repair and maintenance; and materials. In addition,
the 2005 amount costs increased significantly primarily because the Department began paying the risk management premium
of $16.1 million, effective July 1, 2005, from division monies since the Division actually incurred the associated risk.
Source: Auditor General staff analysis of financial information provided by the Arizona Department of Transportation for fiscal years
2004 through 2006.
Transportation Board. According to ADOT, a total of approximately $834 million was
expended in fiscal year 2005 on highway projects.
Scope and methodology
This audit focused on ITD’s use of consultants, construction inspection practices,
and ADOT’s audits of consultants and construction projects. This audit includes
three findings and associated recommendations as follows:
􀁺􇩉 ITD should optimize its internal resources to reduce consultant usage when
appropriate during completion of project design, construction management,
and other similar functions to control costs and maintain employee core
competency levels. This will require establishing division-wide criteria for
deciding when to use consultants, maintaining better consultant usage
information, and implementing strategies to recruit and retain employees.
􀁺􇩉 ITD should improve the consistency and documentation of daily field
inspections and follow up on deficiencies found by independent quality
assurance (QA) inspections.
􀁺􇩔 The Office of Audit and Analysis should continue to take several steps to
increase productivity, improve audit management, and provide an effective
program for auditing consultant and construction contracts.
Auditors used a variety of methods to review and study the issues addressed in this
audit. Audit methods included interviews with management and staff at ADOT, ITD,
the Audit and Analysis group, the Federal Highway Administration, and private
consulting firms; a review of agency-prepared documents, such as the salary
comparison for engineers; and a review of various policies and procedures regarding
project inspection and ADOT contract-auditing requirements. Auditors analyzed data
provided by ITD to determine the number of filled employee positions and to
compare and evaluate employee vacancies, under-filled positions, and employee
experience levels for the period of December 31, 2000 through February 6, 2006.
Auditors also used the following methods to perform more specific audit steps:
􀁺􇩔 To evaluate ADOT’s use of consultants, auditors summarized annual payments
ADOT made to private consulting firms between fiscal years 1995 and 2005.
This data was obtained from ITD’s Contract Management System (CMS).
Auditors verified the accuracy and completeness of CMS data by comparing it
to 41 payment requests submitted by private consulting firms. To evaluate
trends in ITD workload changes and its impact on staffing issues, auditors
compared 5-year Transportation Facilities Construction Programs from 1987 to
State of Arizona
page 6
2006, and analyzed data provided from ITD on annual payments made to
contractors from fiscal years 1995 to 2005. To determine the impacts of using
large amounts of consultant services instead of conducting work by in-house
staff, auditors reviewed ten state and national reports that addressed core
competency and consultant cost comparisons at several state transportation
agencies. (See Endnotes on pages a-iii through a-v.)
􀁺􇩔 To examine the inspection process, auditors observed the activities of four field
inspectors and two independent QA inspectors. In addition, to determine if field
inspectors and independent QA inspectors were adequately documenting their
quality control activities over construction projects, auditors examined
Certification Acceptance Procedure agreements between ADOT and the
Federal Highway Administration (FHWA); the ITD Construction Manual
containing inspection procedures; FHWA’s Construction Program Management
and Inspection Guide; and ADOT’s Standard Specifications for Road and Bridge
Construction. To examine the documentation of inspections, auditors compared
electronic and paper construction inspection checklists completed by both field
inspectors and QA inspectors. Additionally, auditors assessed the level of
inspection documentation contained in 47 field inspectors daily diaries for 9
projects.
􀁺􇩔 To evaluate consultant and construction auditing activities and determine the
number of audits backlogged, auditors first analyzed records maintained on the
ADOT Help Desk Expert Automation Tool (HEAT) Audit database for audits that
were shown as being open between July 1, 2003 and December 31, 2005. HEAT
is primarily used as the incident and service request tracking tool of ADOT’s
Information Tracking Group (ITG) and was modified to track audits for the Office
of Audit and Analysis (Office). Auditors examined 94 audit files to determine the
database’s reliability and found that the data was not sufficiently reliable to
determine audit backlogs. Therefore, to determine audit backlogs, auditors:
􀁺􇩒 Reviewed pre-award audit data from the Office’s HEAT Audit database,
which was revised and expanded by individual office auditors. The Office
provided three different reports of the pre-award backlog between
February 9, 2006 and May 31, 2006. However, auditors could not validate
the accuracy of the source data, and could only estimate the backlog
based on the Office’s assertions.
􀁺􇩍 Matched a list of pending incurred cost audits provided by the Office,
which was extrapolated from HEAT Audit data, with a list of incurred cost
audits provided by ITD’s Engineering Consultants Section from its
Contract Management System database to determine the minimum
backlog. The Office was unable to produce reliable data that would allow
auditors to determine the actual incurred cost audit backlog.
Office of the Auditor General
page 7
􀁺􇩃 Compared office spreadsheets of construction cost audits completed to
the Office’s audit files for each project and audits scheduled with data
from the Construction Operations database. The data was sufficiently
reliable to determine the number of completed audits, the minimum
number of projects that should have been audited, and the minimum
backlog.
􀁺􇩃 Compared office spreadsheets of administrative compliance audits
completed and in progress to office audit files for each of ADOT’s 26
construction field offices. The data was sufficiently reliable to determine
which audits were completed and to extrapolate the administrative
compliance audit backlog.
Auditors also reviewed the Construction Operations database to identify the ten
largest dollar value construction projects completed in fiscal year 2005 and
reviewed associated audit and contract files to determine whether the Office had
completed audits in accordance with its policies and procedures. In addition,
auditors examined internal reports and associated audit files to determine the
number of construction cost and administrative audits requested but not yet
completed. To determine how the Office of Audit and Analysis should ensure
appropriate audit coverage of design and construction contracts, auditors
reviewed the Office’s Audit and Analysis Audit Manual, a Memorandum of
Understanding between ADOT and FHWA, an article on governmental contract
auditing from the Journal of Government Financial Management,2 and an article
on construction auditing from The Internal Auditor.3
􀁺􇩔 To complete the Introduction and Background section of the report, auditors
compiled unaudited information from ADOT’s Web site and other agency-prepared
documents and interviews with the agency.
The audit was conducted in accordance with government auditing standards.
The Auditor General and staff express appreciation to the director of the Arizona
Department of Transportation, the state engineer, and their staff for their cooperation
and assistance throughout the audit.
State of Arizona
page 8
ADOT should optimize internal resources to
reduce consultant usage
The Intermodal Transportation Division (ITD) should optimize its internal resources to
reduce consultant usage when appropriate during completion of project design,
construction management, and other similar functions to control costs and maintain
employee core competency levels. ITD relies heavily on consultants for project
design and management, largely because consultants provide additional expertise
and enable ITD to complete projects in a timely manner when ITD’s internal
resources are insufficient to complete the construction program. ITD also uses
consultants where staff positions are vacant. However, relying on consultants can be
costly, and too much usage can erode staff competency. ITD has a number of
initiatives under way to address staffing issues, but reports that recruitment and
retention are difficult because current salary levels are below market. ITD should
continue its efforts and pursue additional ways to identify and maintain the proper
level of consultant usage.
ITD relies on consultants to deliver
transportation program
ITD hires private consultants to provide project design,
construction management, and various engineering and other
services. These consultants are separate from the contractors who
complete actual construction of projects. See text box at right for
examples of services consultants provide. As of January 2006, ITD
had 430 contracts with 121 private consulting firms at a total value
of $559 million. Consultant contracts are for multiple years, and
payments are made as work is completed.
The practice of using consultants is common among
transportation agencies in other states, as well. According to a
Office of the Auditor General
page 9
FINDING 1
ADOT hired consultants to
perform the following
functions:
• Design highways;
• Complete right-of-way acquisition;
• Perform environmental reviews;
• Manage construction projects;
• Prepare projects for bidding to
contractors; and
• Perform other services related to
the design and management of
highway construction.
State of Arizona
page 10
2003 national survey report, most state transportation agencies use consultants for
some of the design and management of highway projects.4 The report found
agencies were increasingly using consultants to deliver services driven by growth in
highway programs coupled with the same number of or fewer people in their
workforces. The major factors influencing states’ decisions to contract out were staff
constraints, specialty skills, and equipment. A 2003 review of literature of other state
transportation agencies prepared for the Georgia Department of Transportation
found that of the respondents, 79 percent used consultants for design, 53 percent
for construction management, and 50 percent for right-of-way work.5
ADOT’s use of consultants is by far the largest of any state agency. The Governor’s
Efficiency Review Team (Team) reported in June 2005 that ADOT spent more on
consultant contracts and used more consulting services than all other state agencies
combined.6 The Team attributed this reliance in part to state salaries too low to attract
qualified professionals from the private sector. ADOT’s use of consultants for one
type of work—project design—also appears to be one of the highest among states.
Specifically, during an on-site interview in 2000 with members of the Federal Lands
Highway Bench Marking Study Team of the Federal Highway Administration, ADOT
reported using consultants for 80 to 90 percent of its project design work.7 Of the 11
other states that were sent a questionnaire in preparation for an on-site interview, the
10 that responded to the question reported consultant use for preconstruction
engineering ranging from a low of 3 to
6 percent to a high of 80 percent. In
the same year, the Federal Highway
Administration also conducted an e-mail
survey of 32 states and Puerto
Rico, where the average consultant
use for the design phase was closer
to 60 percent for those responding
with a percentage. The range
extended from a low of 15 percent in
Wyoming to 95 percent in New
Jersey.
In addition to hiring consultants for
specific projects, ITD also hires
“supplemental consultants” to
perform duties of ITD vacant
positions. As of January 2006, ITD
had 72 contracts for supplemental
consultants at a value of $78 million.
The Governor’s
Efficiency Review Team
reported that ADOT
spent more on
consultants than all
other state agencies
combined.
Figure 2: Payments to Consultants
Fiscal Years 1996 through 2005
Source: Auditor General staff analysis of actual payments made to consulting firms
for fiscal years 1996 through 2005 obtained from ITD's Engineering Consultant Section.
$0
$20
$40
$60
$80
$100
$120
$140
1996 1997 1998 1999 2000 2001 2002 2003 2004 2005
Millions
Fiscal Year
Payments to Consultants
Consultant use has increased—ITD’s dependence on consultants has
substantially increased over the past 10 years. Although ITD paid consultants $17
million in fiscal year 1996, in fiscal year 2005 the consultant payments totaled $110
million. This is a 424 percent increase since 1996 after adjusting for inflation using
the Consumer Price Index. As shown by Figure 2, payments to consultants have
remained above $110 million for the past 4 fiscal years and were as high as $119
million for fiscal year 2004. ITD was unable to quantify the percentage of its total
workload completed by consultants. However, ITD reported that it used
consultants for approximately 28 percent of construction administration and 90
percent of design work.
Workload and vacancies increase consultant use
ITD has increased its use of consultants over the past 10 years to keep up with an
expanding workload and high staff vacancies. The State’s demand for highway
construction, including the Regional Freeway System in Maricopa County, has
significantly increased ITD’s workload. At the same time, ITD has experienced
vacancies close to or exceeding one-fourth of its engineer positions.
Workload has increased—
Between fiscal years 1995 and
1998, ITD paid contractors an
average of $331 million annually
for the construction of highway
projects. This amount increased
76 percent to an average of
$581 million between fiscal
years 1999 and 2005. Figure 3
illustrates payments to
contractors for fiscal years 1995
through 2005. In addition, the
amount of work planned in
ADOT’s 5-year construction
programs more than doubled in
the past 10 years, from $1.9
billion to $5.1 billion.8 According
to ADOT, an acceleration of
regional freeway construction in
response to population growth
has contributed to the increased
activity. For example, the 2006
through 2010, 5-year program
allocates $2.8 billion, or 55
percent, of total funding to the
Maricopa Regional Freeway
System. ADOT reported that
since 2000, ITD has been on
track to deliver 14 years of urban freeway construction in 7-½ years.
Office of the Auditor General
page 11
Figure 3: Payments To Construction Contractors
Fical Year 1995 through 2005
(Unaudited)
Source: Auditor General staff analysis of agency documents on construction
contractor payments for fiscal years 1995 through 2005.
$200
$300
$400
$500
$600
$700
$800
1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005
Fiscal Year
Millions
Payments To Construction Contractors
Further, the workload increase has been accompanied by an increase in project
complexity. According to ADOT officials, increased project complexity contributes
to ITD’s use of consultants. They stated that ADOT builds projects differently now
than it did 10 years ago. Specifically, they noted that environmental assessments,
interaction with other state agencies, utility conflicts, plant salvage, noise barriers,
and wildlife fencing requirements all increase the complexity of highway
construction. For example, ADOT reported that environmental impact studies may
take 5 to 7 years, much longer than they took 10 years ago.
Staff vacancies are high—Staff vacancies and reduced FTEs have forced ADOT
to increase its use of consultants. According to ITD records, vacancies among ITD
engineers have not dropped below 23 percent in 5 years. As of February 2006, 79
of 286, or 27.6 percent, of engineering positions were vacant. Vacancies are
particularly acute for resident engineers who provide oversight of the construction
projects. Forty-six percent of resident engineer and senior resident
engineer positions were vacant as of February 6, 2006, and ITD hires
consultants to serve as resident engineers (see text box). In January
2006, ITD’s Operations Division also had 135 supplemental services
consultants in other technical positions, such as field inspectors.
According to ADOT, the supplemental services consultants function as
ITD staff and are physically located in ADOT facilities. In addition to
staff vacancies, the number of ITD-authorized employee positions was
10.9 percent lower in fiscal year 2006 than in fiscal year 1995.
ADOT attributes its high engineer vacancies to employees retiring or
leaving for higher salaries offered by private consulting firms and local
governments. In 2005, ITD surveyed nine consulting firms that provide
services to ITD and found that altogether, 45 percent of the
consultants’ employees were former ADOT employees (see text box).
Based on exit surveys and interviews of employees departing ADOT in
fiscal year 2005, ADOT reported that 18.6 percent cited retirement, and
14.9 percent reported receiving a better job or salary as reason for
leaving. Also, in November 2005, ADOT completed a salary
comparison of ITD’s engineer positions and found that salaries were
13 to 26 percent lower than comparable private and public positions
in the Phoenix area. This comparison was
completed after a 5 percent pay increase had
taken effect.
Consultant use may have
unintended consequences
Although ITD needs to use consultants to
complete its growing workload and to meet a
project schedule, research shows that using
State of Arizona
page 12
Resident Engineer:
Oversees the construction of the project
according to the scope and within the
schedule and budget.
As of February 2006, ITD had the following
resident and senior resident engineer
positions:
28 Filled
24 Vacant
52 Total Positions
15 consultants serve as resident engineers
on projects.
Source: Auditor analysis of unaudited data provided by ITD from
January and February 2006.
ADOT survey of employees at nine consulting firms to
determine employees’ ADOT experience (2005)
Category Number Percent
ADOT retirees 64 26
Other former ADOT employees 46 19
Subtotal (Firm employees with ADOT link) 110 45
Firm employees without ADOT link 137 55
Total firm employees 247 100
As of February 2006,
27.6 percent of ITD’s
engineering positions
were vacant.
Office of the Auditor General
page 13
consultants may be more costly than using highway department staff and can pose
risks to the department’s level of core competency.
Higher cost may be one outcome—Most studies that focus on use of
consultants in state departments of transportation agree that consultants increase
the cost of services, according to a 1999 report.9 The report reviewed 17 studies
since 1977, including reports conducted by state departments of transportation,
other public bodies, universities, and private firms. Thirteen studies found
consultants were generally more expensive, while three found no difference in
cost, and one found that consultants were less expensive. Overall, literature
reviewed for this audit suggests that although consultants give departments more
flexibility to handle their workloads while managing staff size, they do not provide
cost savings and in fact may be more expensive than performing work in-house,
in part because of the added cost of administering consultant contracts. However,
studies differ in their estimates of the cost differential.10 Further, legislative audits in
other states have raised questions about the accuracy of data used to perform
cost comparisons.11
Loss of core competency another potential outcome—Consultant use
may also pose a risk to the core competency needed to manage projects.
According to state and national reports, use of consultants by state transportation
agencies may have other effects besides increasing project costs. Some state
transportation agencies recommend limiting the amount of work outsourced in
order to retain in-house expertise and the ability to review consultants’ work,
according to a national survey published in 2000.12 Another report states that
maintaining technical expertise within the agency may become
more difficult as the percentage of projects contracted out
increases and that it is important to keep interesting and
challenging projects in-house to maintain some level of
expertise.13 A 2003 report for the Georgia Department of
Transportation states that state departments of transportation are
now relying on consultant services for functions that have
traditionally been performed in-house, requiring managers to
learn new sets of skills.14 According to ADOT officials, the Arizona
Chapter of the Associated General Contractors lobbied for a pay
plan to increase ADOT salaries in July 2005 to reduce turnover of ADOT staff. The
contractors were concerned with a lack of experience, competency, and slower
decision-making by consultant resident engineers and inspectors.
In addition to these outcomes, the 2003 report for the Georgia Department of
Transportation stated that the use of consultants may also result in the loss of
accountability, less control of the quality and timing of projects, and less capacity
to serve a traditional role of hiring and training entry-level engineers.15
Although it was impossible to measure staff competency as part of auditors’
review, the length of time employed is generally one indicator of competence. ITD
Core competency:
Specialized technical or scientific
activities that must be conducted by an
organization and its employees in order
to fulfill its mission and execute its
responsibilities.
Consultants are usually
more expensive than
in-house staff.
experience levels are declining. Although ITD’s 207 engineers had an average of
more than 12 years of experience at ADOT as of February 2006, this represents a
decline of 2.07 years since December 2000 (see Table 3). The average years of
experience among Engineer I employees, who represent the bulk of ITD engineer
positions, decreased from more than 11 years to about 8-½ years. According to
an ITD official, an Engineer I employee must be a registered engineer who has an
engineering degree, a passing score on a written engineering registration test, and
4 years of work under a registered engineer. As of February 2006, 50 percent of
the engineer positions have fewer than 10 years’ experience. Auditors could not
verify or measure construction project inspectors’ experience levels because
inspectors share employee classes with other employee groups, and separate
data on inspectors was unavailable.
ITD should take further actions
Although ITD has developed strategies to reduce vacancies and increase retention,
research and additional information shows that ITD should continue to develop
strategies to address staffing issues, develop criteria for deciding when to use a
consultant, and monitor the use of consultants.
ITD working to address vacancies—ITD has taken steps to increase retention
and improve its ability to attract qualified personnel:
􀁺􇩉 In some cases, ITD has made counter-offers when engineers were offered
higher-paid positions in the private sector. For example, in August 2005, an
ITD engineer was offered a private consulting position with a $24,000 annual
salary increase. ITD counter-offered with a salary increase over $7,000 and the
engineer accepted ITD’s offer. In eight of ten cases where ITD made counter-
State of Arizona
page 14
Table 3: Average Years’ Experience for Selected ITD Engineer Positions
As of December 31, 2000 and February 6, 2006
Average years of experience
Position
Number of
filled
positions
December 31,
2000
February 6,
2006
5-yr
change
Engineer I 95 11.23 8.50 (2.73)
Engineer II 67 13.78 13.36 (0.42)
District Engineer 10 25.01 15.25 (9.76)
Average of all 207
engineering positions – 14.31 12.24 (2.07)
Source: Auditor General staff analysis of ADOT’s Human Resource Management System and Human
Resource Information System data for ITD engineer positions as of December 31, 2000 and
February 6, 2006.
ITD experience levels
are declining.
offers, the employees elected to remain employed at ITD. However, according
to ADOT, many employees are not receptive to receiving a counter-offer once
an external offer has been made, or the difference between the two salaries is
too high for ADOT to bridge.
􀁺􇩉 ITD has hired more people into its 18-24-month Engineer in Training (EIT)
program to allow individuals to experience all areas of ITD.
􀁺􇩉 In July 2005, ITD’s engineers and other technical positions received a 5
percent pay increase.
Even with recent pay plan increases, ADOT salary surveys show salaries below
market. The success of ITD’s counter-offers indicates that higher salaries may help
stem turnover rates in engineering positions. ITD should continue to develop
strategies to recruit and retain employees by filling existing vacant employee
positions with competent staff.
Other states’ efforts to recruit and retain—A national report and a report on
the Minnesota Department of Transportation identified that the development of
core competencies was a concern among many state transportation agencies and
offered examples of methods used to increase recruitment and retention of quality
employees.16 These include:
􀁺􇩔 The Wyoming Department of Transportation developed a mentoring program
where a senior administrator mentors two employees. Wyoming also uses a
training program to prepare employees for future positions.
􀁺􇩔 The Minnesota Department of Transportation developed a succession
planning model that identifies the core competencies needed for essential
executive-level positions and reviews potential internal candidates with a 360-
degree assessment to identify the person’s work experience and potential to
fill a position.
􀁺􇩔 The Louisiana Department of Transportation and Development (DOTD) works
with universities in a cooperative program that allows college students to
complete a 30-week rotation through 17 different functions in DOTD. This
program is similar to ADOT’s EIT program.
Although ADOT reports that its EIT program has helped to attract talented
employees, ADOT may also want to consider mentoring, succession, and other
programs to help reduce the impact of employee turnover and retirements.
According to ADOT, it is exploring and piloting a talent management program.
ITD should establish criteria and monitor consultant usage—ITD does
not have formal division-wide criteria for when and how to use the various types of
consultants. According to an ADOT official, each group manager or the district
management team makes the decision to hire consultants based on whether they
Office of the Auditor General
page 15
believe in-house staff has the expertise and time to complete the work.
However, reasons for using consultants, such as lack of staff or time
constraints, are not documented. State and national reports found that using
criteria to decide when work must be done by outside consultants is
important.17 In some cases, such as cyclical workload and projects requiring
one-time or infrequently used expertise or equipment, consultant usage is
usually warranted. However, in other situations, consultants may be used to
provide ongoing, recurrent work that could be provided more cheaply by in-house
staff. Louisiana’s Department of Transportation has developed a
computerized model to systematically evaluate both qualitative and cost
aspects of contracting out.18
ITD should establish division-wide criteria for deciding when to use a
consultant to complete projects and tasks. Possible criteria should include:
needed skill can’t be utilized on a full-time basis; consultants have equipment
or other assets for work that is not cost-effective to purchase; and consultants
can provide needed higher-quality services or services at a lower cost, or
workload is temporary, short-term, or seasonal. Having criteria to determine
when consultants should be used instead of in-house staff will allow
managers to document consultant workload that could be redirected to
internal staff. For example, the Governor’s Efficiency Review Team reviewed
individual consultant contracts, and ADOT officials reported that $80 million in
contracts were for work that regular employees could provide if ITD had
sufficient authorized and filled positions. ITD should identify ongoing,
recurrent work related to project design, construction management, and other
functions currently provided by consultants and perform these services in-house
when appropriate. ADOT currently prepares a weekly manpower roll-up
report to identify projected manpower needs on current projects
throughout the State. Further development of such tools could help ITD to
identify where internal staffing could provide work now done by consultants.
In addition to lacking formal criteria for the appropriate use of consultants, ITD
does not keep adequate data on consultant usage to identify where internal
staff resources can reduce consultant usage. ITD’s administrative office and
most groups within ITD’s development section do not keep records of the
number or type of projects that consultants complete and reasons justifying
consultant contracts. Specifically, bridge construction groups document the
number and percentage of projects completed by consultants in annual
strategic plans, but other groups, including ITD administration, do not keep
similar records. ITD should develop a method of tracking and monitoring
consultant usage, such as compiling data on the dollar amount, quantity, and
type of projects completed by consultants, and also which consultant services
did not meet established criteria, but were obtained because of inadequate
staffing. Such information can be used to quantify and evaluate reasons for
using consultants, and to identify strategies, such as identifying consultant
services that could be more cost-effectively provided by in-house employees.
Such information could also be used to demonstrate how higher salaries
combined with more in-house work and less consultant usage could reduce
total costs while also improving core competency.
State of Arizona
page 16
Recommendations:
1. To better ensure that it develops competency internally and reduces costs, ITD
should aggressively seek to:
a. Fill existing vacant employee positions with competent staff;
b. Identify ongoing, recurrent work related to project design, construction
management, and other similar functions currently provided by consultants
and perform these services in-house when appropriate; and
c. Continue to develop strategies to recruit and retain staff, and consider
mentoring, succession, and other programs to help reduce the impact of
employee turnover and retirements.
2. To better ensure that it identifies and maintains the proper level of consultant
usage, ITD should:
a. Develop division-wide criteria for deciding when to use a consultant to
complete projects or tasks; and
b. Develop methods of tracking and monitoring consultant usage, such as
compiling data on the dollar amount, quantity, and type of projects or
services completed by consultants, and reasons for using consultants or
other relevant information, and evaluate information to identify consultant
services that could be more cost-effectively provided by in-house
employees.
Office of the Auditor General
page 17
State of Arizona
page 18
ITD should improve implementation and
documentation of inspection process
To ensure that contractors meet construction standards, ITD should improve
inspection quality. ITD conducts both daily field inspections and periodic
independent quality assurance (QA) inspections of construction activities. However,
ITD could improve consistency and documentation in daily field inspections. In
addition, ITD should implement follow-up procedures on deficiencies found by
independent QA inspections.
ITD monitors contractor activities
In order to ensure that roads are safe and durable, ADOT
has established Standard Specifications for Road and
Bridge Construction. (See text box at right for an example
of applicable specifications.) Contractors must adhere to
these specifications and each project’s special contract
provisions. ITD monitors contractors’ compliance with all
of these requirements by conducting daily inspections
and periodic independent QA inspections. Inspections
also enable ADOT to make correct payments to
contractors based upon actual material quantities used
and work completed.
ITD employs both field inspectors and independent QA
inspectors to ensure compliance with roadway
construction quality specifications.
􀁺􇩆 Field inspectors monitor on a daily basis—Daily
inspections by more than 220 field inspectors, as of February 6, 2006, monitor
ongoing contractor activities. Under the supervision of resident engineers and
Office of the Auditor General
page 19
FINDING 2
Photo 2: Paving the San Tan Freeway
Photo: Arizona Office of the Auditor General.
Example: Asphalt specifications
ADOT’s Standard Specifications provide standards
for asphalt paving such as the asphalt temperature,
placement of the asphalt in front of the paving
machine, and how to place joints (a narrow space
separating two slabs or sections of pavement) in
the asphalt.
project supervisors, these inspectors observe contractors’ daily work and
compare it to specifications. Field inspectors communicate with contractor staff
on construction requirements and any problems they observe. Field inspectors
must document their observations in daily diaries for the project supervisor’s
review. Further, for most types of work observed, field inspectors must complete
a checklist indicating whether various aspects of the work met specifications.
Field inspectors are authorized to reject work or materials that do not comply
with plans and specifications.
􀁺􇩉 Independent QA inspectors review all projects at least once—Independent QA
inspections by nine inspectors, as of March 8, 2006, in ITD’s construction
operations section provide further assurance that construction work is
completed according to all specifications and requirements. The Federal
Highway Administration (FHWA) conducted these reviews until 1992, when it
transferred this responsibility to ADOT. ITD employees in the construction
operations section conduct the independent QA inspections. According to an
agency official, each construction project receives at least one QA inspection,
and some larger projects receive up to five. Similar to field inspectors, the QA
inspectors observe the work conducted and determine its conformance with
plans and specifications. Independent QA inspectors document their
observations on construction inspection checklists and nonconformities are
compiled in a Certification Acceptance report to project management staff.
ITD can improve field inspections
Although inspections provide important assurance of construction quality, the field
inspection process could be made more effective by consistently documenting
inspection results. Not all inspectors document in their daily diaries whether
construction work met specifications and whether there were problems identified and
resolved as required by ADOT policies. In addition, ADOT’s Construction Manual
requires inspectors to follow checklists to help ensure quality goals are met, but not
all inspectors use the checklists effectively. To ensure its inspection processes are
effective, ITD should ensure that its inspectors follow the requirements.
Inspectors do not consistently record observations in accordance
with policy—Although documentation is critical to ensuring problems are
resolved and payments are accurate, ITD inspectors do not consistently document
the results of their observations. As a result, it is unclear whether the work met
specifications, whether there were problems or if problems were resolved, and
whether payments are accurate. For example, during one inspection an auditor
observed, the inspector determined that approximately 10 feet of concrete pipe
did not meet specifications. The auditor observed the inspector discussing the
problem with the contractor, but the inspector’s diary entry for that day indicated
State of Arizona
page 20
that the pipe met specifications. When asked about this, the inspector explained
that the contractor removed 6 feet of the problematic pipe and the payment was
reduced. However, the inspector did not document the problem in the daily diary
and did not indicate that the 6 feet of pipe was removed later. Because no entry
was made concerning this problem, there was no documentation indicating why
the inspector required only 6 feet be removed instead of the approximately 10 feet
that was found in noncompliance on the day of the auditor observation. Without an
accurate record of the problem and its resolution, ADOT cannot ensure that the
pipe met specifications and that the payment was accurate.
Inspectors vary in the type and level of documentation they maintain. Auditors’
review of nine projects determined that 43 of 47 inspectors’ diaries showed the
type of work observed, but not whether work met specifications. This practice does
not conform to documentation requirements. ADOT’s Standard Specifications for
Road and Bridge Construction states that inspectors’ documentation should
include the level or degree of conformance of the work with plans and
specifications. Further, ADOT’s Construction Manual states that historical
information on how work was constructed is valuable in the future if a project has
to be modified or rebuilt. Finally, the FHWA’s Construction Program Management
and Inspection Guide states that documentation is essential to define the progress
and quality of work, inspectors should identify problem areas and document
resolution of concerns, and that inspectors should follow up on
previous inspection findings and draw conclusions on the
finished product’s acceptability.
Checklists not used consistently—In addition to not
completing diaries, field inspectors do not always use required
checklists. ADOT policy requires field inspectors to complete
checklists applicable to work observed. According to an
independent QA inspector, ITD has developed approximately
80 checklists to cover the variety of construction work
observed by field inspectors and QA inspectors. For example,
separate checklists address traffic control, concrete curing,
grading, and concrete box culverts. The checklists include
“critical” items, some where noncompliance may potentially
put human life at risk, and “major” items, where
noncompliance could cause substantial reduction of highway
operability or durability, increased cost, or major environmental
impact. According to ADOT, it is planning to revise the
definition of “critical” items because some may not put human
life at risk. The checklists, based on standard specifications,
are meant to affirm quality requirements at the beginning of a
project and to ensure the product meets quality goals. In
addition, they may help less-experienced staff feel more confident to approach
contractors with solid evidence of noncompliance.
Office of the Auditor General
page 21
Inspection checklist item
examples:
􀁺􇩔 There are no unprotected edges of 2
inches or greater. (Critical item)
􀁺􇩓 Side slopes conform to current
OSHA requirements. (Critical item)
􀁺􇩃 Curing the concrete has begun
immediately following the required
finishing operations. (Major item)
􀁺􇩓 Sediment control berms are properly
constructed at the specified
locations. (Major item)
Source: ADOT Guidelines for Weighting Attributes on
Quantified Checklists and checklist attributes.
Despite the importance of checklists, in a review of nine projects, auditors
determined that 27 out of 47 inspectors did not fill out either electronic or paper
checklists. When asked why checklists were not used, a supervisor and a resident
engineer said they had difficulty enforcing the requirement. Another supervisor
said checklists could be improved by removing unnecessarily labor-intensive
items. Auditors’ analysis of checklist results from April 5, 2001 through December
19, 2005, showed that of 2,788 checklist items, 276, or approximately 10 percent,
were never used because they were marked not applicable every time the
checklist was used.
Application of inspection standards varies—Field inspectors report fewer
deficiencies than independent QA inspectors when using mandatory inspection
checklists. According to managers who auditors interviewed in both areas,
independent QA inspectors apply a stricter interpretation of the quality standards
than field inspectors. Results of independent QA inspections suggest that when a
strict interpretation of the standards is applied, work does not meet specifications
as often as field inspection results show. Specifically, as shown in Figure 4, field
inspectors determined that work met specifications 66 percent of the time while
independent QA inspectors found work met specifications only 35 percent of the
time, based on an analysis of two projects where both field inspectors and
independent QA inspectors conducted inspections. It should be noted that field
inspectors and independent QA inspectors were not observing the projects at the
same time and the independent QA inspectors had a very high level of “not
applicable” responses for the standards. However, the two groups were using the
same checklists to observe the same type of work
on the same projects, and the “not applicable”
responses have no effect on the checklist items
marked as failed, but may have an impact on the
items that passed. The probability that the two
groups’ conforming results would be so different is
less than 1 in 100, based on statistical analysis.
The result suggests that the two groups may be
applying different standards when determining
what conforms. To ensure that the checklist results
are determined in a consistent manner, ITD should
assess how field inspectors and independent QA
inspectors are interpreting the checklist items.
Further, ITD should provide training and/or
develop guidelines to help field inspectors and QA
inspectors interpret the checklist items in a similar
manner.
Knowledgeable persons have questioned whether
ITD field inspector experience levels are sufficient.
For example, an FHWA official expressed concern
about inspection quality because he had
observed a drain on ADOT inspector expertise
and believed that inspectors were “spread too
thin.” This official stated that he was familiar with
State of Arizona
page 22
0%
10%
20%
30%
40%
50%
60%
70%
Met Specifications Did Not Meet Specifications Marked "Not Applicable"
Inspection Results
Field Inspections Independent QA Inspections
For nine projects, 27 out
of the 47 inspectors did
not fill out any
inspection checklists.
Figure 4: Comparison of Responses on Checklists
July 1 through December 19, 2005
Source: Auditor General staff analysis of checklist inspection records for two projects
reviewed by both field inspectors and independent QA inspectors.
the ADOT situation because although no longer conducting the independent
reviews, FHWA personnel monitor most ADOT highway projects to help ensure
that projects are being built within the design scope proposed when approved for
federal aid. Additionally, an ITD supervisor believed that many contractor firms had
more experienced staff than ADOT, which caused some inspectors to be
intimidated and hesitant to tell contractors about observed problems. ITD records
show that 34 percent of its inspectors had fewer than 5 years’ experience with
ADOT as of February 6, 2006. An ITD official stated that about 5 years’ inspecting
experience was needed to become fully proficient and that inspectors with less
experience were usually assigned to perform less technical inspection work until
they gained adequate experience. Additionally, 29 percent of the inspector
positions were under-filled, and 32 percent of the positions were vacant. An under-filled
position means that ITD could not recruit someone who meets the minimum
experience and qualifications required for the position and instead hired someone
without those qualifications at a lower-level position to perform the job duties. ITD
reported that it has a recruitment strategy to develop inspectors by providing a
technical training and certification program, combined with on-the-job work
experience, to allow people hired at other experience levels to advance to the
desired position level.
ADOT should take steps to increase effectiveness—To better ensure that
inspections serve their intended purposes, ADOT should ensure that inspectors
document pertinent information about their observations, including whether work
conforms to requirements, any problems encountered, and problem resolution. In
addition, to comply with policy and help ensure the quality of work inspected,
ADOT should ensure that field inspectors complete and submit checklists as part
of their daily diaries and document that critical and major checklist items were
addressed.
ADOT has begun taking steps to make it easier for field inspectors to use the
checklists. First, in July 2005, ITD began providing field inspectors access to an
online checklist database previously used only by independent QA inspectors and
began training them on how to use it. Once trained, field inspectors can enter
checklist responses and supporting comments directly onto the checklists in the
database using laptops in the field. Second, according to ADOT, new checklists
are developed and updated with a committee that includes the district’s most
experienced technical and engineering staff. However, an ITD official stated that
field staff has not been consistently involved in the creation and revision of the
checklists. To help ensure the checklists meet the needs of field inspectors and
include appropriate and most applicable content, ADOT should consider a
checklist revision process that includes knowledgeable field inspectors.
Independent quality assurance inspections lack followup
Followup on noted deficiencies would enhance the effectiveness of the independent
QA inspections. Specifically, ADOT could better use the results of independent QA
Office of the Auditor General
page 23
State of Arizona
page 24
inspections by requiring formal responses from the field organizations and followup
for critical and major deficiencies.
ITD does not require followup on important areas of noncompliance identified by
independent QA inspectors. As previously described, independent QA inspectors
from the construction operations section provide periodic independent inspections
of construction work and determine its conformance with plans and specifications by
observing the work and filling out the inspection checklists. According to an agency
official, independent QA inspectors do not have the authority to change work in
progress, but they can recommend to ADOT project personnel that work be rejected
or processes be changed. Any deficiencies they find are
documented in a Certification Acceptance report that is provided
to project management staff.
As discussed previously, ITD classifies inspection checklist items
according to importance, with risks to human life included as
“critical” items and important quality factors considered “major.”
Auditors’ analysis of 1,970 independent QA inspections
determined that in 1,586 cases (80 percent), inspectors identified
one or more critical or major noncompliant item. For example,
during one inspection, an independent QA inspector found two
major items to be noncompliant. One item concerned the
placement of rebar used to reinforce a concrete structure, and the
other item concerned the size of the rebar used to reinforce a
concrete structure. According to ADOT, if a smaller size rebar or
larger rebar spacing occurs, cracks could develop in the concrete
resulting in a shorter lifespan, and if the rebar placement deviates greatly from the
specifications, it may result in a complete failure of the structure. However, ADOT
reported that industry design standards provide safety factors for critical
specifications to ensure that minor variations do not cause failures.
Although critical and major noncompliance items are reported to project managers
through Certification Acceptance reports, ITD has not developed any follow-up
provisions and requirements. According to an agency official, ITD does not require
that resident engineers or project supervisors respond to or specifically address
independent QA inspection results, and while some project staff indicated that they
may respond informally to specific deficiencies, they are not required to do so. As a
result, critical and major items may not receive adequate attention to correct them
and to reduce future occurrences. ADOT could better use the results of independent
QA inspections by requiring followup for critical and major deficiencies. Following up
on QA inspections would supplement existing ADOT quality control practices that
include resident engineers’ consideration of sampling and testing results prior to final
acceptance of a project as well as use of information obtained in daily inspections by
field inspectors.
Photo 3: Pouring a retaining wall footing
Photo: Arizona Office of the Auditor General.
Recommendations:
1. To ensure that the checklist results are determined in a consistent manner, ITD
should assess how field inspectors and independent QA inspectors are
interpreting the checklist items. Further, ITD should provide training and/or
develop guidelines to help field inspectors and independent QA inspectors
interpret the checklist items in a similar manner.
2. To comply with department policy and FHWA inspection practices, ADOT
should ensure that field inspectors document inspection results, including:
􀁺􇨠 Whether work conforms to plans and specifications;
􀁺􇩗 Whether problems occurred; and
􀁺􇨠 Problem resolution.
3. To comply with department policy, ADOT should ensure that field inspectors
complete and submit checklists as part of their daily diaries, and document that
critical and major items were addressed.
4. To help ensure that checklists meet field inspectors’ needs and contain
applicable content, ADOT should consider a checklist revision process that
includes knowledgeable field inspectors.
5. ADOT should require followup on independent QA inspection results that
identify critical or major noncompliance.
Office of the Auditor General
page 25
State of Arizona
page 26
ADOT needs to improve audits of design and
construction contracts
ADOT has not adequately planned and managed the audits of its highway design
and construction contracts. The Office of Audit and Analysis (Office) should audit all
phases of ADOT’s construction projects. However, the Office has not completed the
number of audits required by its own policies, and many completed audits have not
been issued in a timely manner. Several factors have contributed to these problems,
including ongoing, long-term vacancies and inadequate audit management. ADOT
hired a new chief auditor in January 2006 who has begun taking steps to address
problems.
Audits required during all construction phases
The Office is required to conduct audits covering the full range of consulting and
construction contracting practices. Under an intergovernmental agreement with the
Federal Highway Administration, ADOT must provide total operational audit
coverage of both design and construction contracts for all projects with federal
funding. Federal law also requires audits of engineering design contracts to ensure
that they comply with cost principles in Federal Acquisition Regulations. ADOT
adopted a policy to audit all contracts similarly, whether or not federal money is
involved.
The Office conducts four types of audits to comply with its auditing mandate. These
audits are described in the text box (see page 28) and are important to ensure that
ADOT pays the correct amount for construction and consulting work. For example, a
highway design engineering contract with compensation paid on a cost-reimbursement
basis and valued at more than $250,000 would require a pre-award
audit before the contract is awarded to ensure that the contractor has an acceptable
accounting system and to determine an overhead rate that will apply to the contract
Office of the Auditor General
page 27
FINDING 3
State of Arizona
page 28
(see text box, page 29). A subsequent incurred-cost audit is later required after
contract completion.
Audits are important to ensure contract compliance and proper payments.
Ineffectively monitoring consultants’ and contractors’ performance can result in
failing to detect:
􀁺􇩏 Overcharges and underpayments—Failure to perform pre-award audits may
result in improper payments because provisional rates may vary from audited
rates. Additionally, if incurred-cost audits are not conducted or completed in a
timely manner, overpayments owed to ADOT or underpayments owed to the
consultant may be undetected or delayed. Moreover, failure to perform
construction-cost audits increases the risk that incorrect payments will be made
to contractors. For example, one construction progress audit completed in
Construction audit types:
Engineering services (design) contracts:
􀁺􇨠 Consultant Pre-award—Review consultant records to ensure that it has an acceptable
accounting system for indirect and direct costs, and to determine an overhead rate that
will apply to the contract. Required for all contracts over $250,000 and for some smaller
contracts. Should be completed before contract award.
􀁺􇩉 Incurred cost—Review records after contract completion to ensure that all costs,
including overhead, were properly applied. ADOT does not conduct incurred cost audits,
but contracts them to private firms. Required for all cost reimbursement contracts over
$250,000 and optional for smaller contracts. Provided on a sample basis for lump-sum
contracts, but required for all lump-sum contracts over $250,000 that are terminated for
convenience or for default.
Construction contracts:
􀁺􇨠 Construction cost—Review projects that are completed or at least 50 percent complete
to ensure contract compliance with payments, contract documents, change orders, labor
compliance, materials testing, and other items. Required for all projects over $10 million
and judgmentally selected for smaller projects. Required at $20 million intervals on
projects over $20 million.
Construction administration:
􀁺􇩁 Administrative—Review ADOT’s highway construction organizations to determine
compliance with contract management requirements, including purchase orders,
construction schedules, pay and timekeeping, accountable assets, and computer
security. Required once per year for each of 26 construction field offices.
September 2003 questioned $272,500 in payments made
to a contractor, but the report was not issued until October
2005, delaying any required corrective action.
􀁺􇩗 Whether contractors have delivered all services—Not
conducting construction-cost audits can mean paying for
services not delivered. For example, a construction
progress audit completed in July 2004 and issued in
October 2005 found errors in contract quantities for
delivered concrete and questioned over $300,000 in
invoices that ADOT had paid without verifying that the
contractor provided the equipment, labor, and materials.
In addition, according to the state construction engineer, audits
of construction administration are valuable in identifying areas
for improvement that could result in better project management,
decreased cost overruns, and better quality. The state
construction engineer reported that district and resident
engineers and the State Engineer’s Office use audit results to
identify trends in consultant and staff compliance with
requirements.
Office fails to complete audits or completes
them late
Office productivity has diminished in recent years. Specifically, because the Office
has not issued the number of audits required by policy, it has backlogs of each audit
type. Additionally, some of ADOT’s highest-cost projects have not received all the
audits that office policy requires. Finally, completed audits have often not been
issued for at least 6 months after audit work was finished.
Some audits not conducted—The Office does not conduct the
number of audits required by its own policies. The Office had
backlogs of each audit type:
􀁺􇩔 The Office estimates it had approximately 500 pre-award
audits in its backlog as of May 31, 2006.19 The Office could
not provide verifiable data regarding the size of its pre-award
backlog.
Office of the Auditor General
page 29
Overhead rate facts
􀁺􇩏 Overhead consists of allowable
indirect costs, such as rent and
insurance costs. The overhead rate is
calculated by dividing indirect costs
by direct costs, such as labor.
􀁺􇩉 In a cost-reimbursement contract,
ADOT pays the contractor for its direct
costs plus a percentage of the direct
costs to cover the contractor’s
overhead. The percentage that ADOT
pays is determined by the pre-award
audit. If no audit has been performed,
ADOT and the contractor negotiate a
provisional rate.
􀁺􇩏 Overhead rates accepted by audits
were typically between 150 and 160
percent, with a range between 120
and 184 percent.
Estimated Audit Backlogs as of
February 9, 2006:
Pre-award 500 1
Incurred cost 40
Construction cost 47
Administrative 17
1 Office of Audit and Analysis estimate.
Source: Auditor General staff analysis of ADOT data and
Office audit files.
􀁺􇨠 A minimum of 40 incurred cost audits dating back to March 2001 have not
been done as of March 19, 2006. Additionally, the Office did not conduct 17
other audits because they were not begun before a statutory record-keeping
time limit expired or because the company was no longer in business. No
projects started after July 1, 2002, have received the required incurred-cost
audit.
􀁺􇨠 Auditors’ review of office spreadsheets of scheduled and completed
construction cost audits and completed audit files identified that 47 required
construction cost audits dating back to 2003 were backlogged as of
December 31, 2005. Productivity in this area declined from 23 audits
completed in fiscal year 2003 to only 5 completed in fiscal year 2005, but has
increased in fiscal year 2006, with 12 audits completed in the first 6 months.
􀁺􇨠 Auditors’ review of the Office’s audit files found that 17 of 26 ITD construction
organizations have not received an administrative compliance audit in the
2005 calendar year, and 7 have not been audited since July 2003. The
organizations should receive 1 audit per year, and altogether, 41 of 78 such
audits were not conducted in the 3 years ending June 30, 2005.
Projects missing audits include ADOT’s largest
construction projects—The Office has not conducted all
required audits on its largest construction project contracts. To
evaluate the amount of audit coverage that the Office provided,
Auditor General staff selected the ten largest projects by dollar bid
amount that ADOT completed in fiscal year 2005. Those projects
ranged in size from $15.7 million to $221.1 million. As shown by
Table 4, the Office did not conduct the minimum number of audits
required by its policies in any of the three audit categories for these
ten projects. Pre-award audits are required for each of 52
consultants on these ten projects. Multiple audits for each project
are often required because a prime contractor and multiple
subcontractors provide services for each project. An incurred cost
audit was also required for the one project with a cost plus fixed-fee
contract, though not for the nine other projects completed
under lump-sum contracts. Additionally, because office policy
requires construction-cost audits at $20 million intervals, these
projects should have received at least 25 construction-cost audits.
Some audit reports issued late—In addition to not conducting some audits, the
Office has not issued some reports in a timely manner. In the first half of fiscal year
2006, the Office issued 17 construction cost and administrative compliance
reports. At least 14 were issued 6 months or more after the audit exit conference.20
Those 14 reports were issued from 183 to 769 days after the exit conference, with
the average being 371 days. Office policy requires issuing the report within 20 days
after the draft report date. According to office management, the original draft report
is discussed at the exit conference. Thus, it appears these reports were issued
substantially later than the policy requires.
State of Arizona
page 30
Some audits were
issued 6 months to 2
years after the audit exit
conference.
Table 4: Status of Required Audits
Ten Largest Completed Projects
Fiscal Year 2005
Audit Type Required Issued Not done
Pre-award1 52 51 1
Incurred cost 1 0 1
Construction 25 7 18
Total 78 58 20
1 In addition to the required audits shown in this table, contract
files revealed that the Office of Audit and Analysis was not
notified to conduct 17 audits and conducted 2 pre-award audits
on subcontractors that were approved but not used on the
projects.
Source: Auditor General staff analysis of audit reports issued by
Office of Audit and Analysis for projects identified by
analyzing ADOT’s checklist system.
Issuing reports late delays recovery of overpayments. In addition, because
contractors are required to retain records for only 5 years after the final payment,
ADOT may be unable to conduct audits after that time. As a result, long delays
before ADOT conducts audits may prevent ADOT from recovering monies paid in
error. For example, one project was completed in March 2001, but the Office did
not conduct an incurred cost audit within 5 years. Further, the 5-year period will
expire on 21 other projects in calendar year 2006. In addition, long delays hinder
project accounting because ADOT closes completed project accounts and
transfers any excess funds to other accounts, making it more difficult to process
any underpayments identified by untimely audits.
Several factors contribute to low productivity
The Office’s low productivity has resulted from management’s failure or inability to fill
vacant positions, adequately plan and manage its workload, and other factors. The
Office lacked a chief auditor for several months, until January 2006. In addition, an
inadequate database and other problems contribute to the Office’s failure to meet
requirements. Specifically:
􀁺􇨠 Ongoing, long-term vacancies—The Office has not ensured that staffing is
adequate to meet audit requirements. As of December 31, 2005, the Office had
only 9 of 16 positions filled in the unit responsible for consultant and construction
audits, as shown in Table 5. Of the unit’s 7 vacancies, 4 were unfilled for over 3-
½ years, since before July 2002.
The Office has not been able to reduce staff
vacancies. Although the Office has filled recent
departures with new hires, it has only recently
reduced its vacancy level. According to office
managers, it was difficult to find candidates with
the cost accounting experience necessary to
conduct audits in accordance with the Federal
Acquisition Regulations. The Office had not
sought applicants without that experience
because understaffing inhibited its ability to
provide training.
􀁺􇩁 Audit management inadequate—The Office is not
complying with its policy requiring an annual audit
plan that projects the number of audit requests,
considers other audit requirements, and prioritizes audits based on available
resources. For example, the policy requires that audits be selected based on a
department-wide audit risk assessment. This approach to selecting and
Office of the Auditor General
page 31
Four of the Office’s
seven vacancies are
over 3-½ years old.
Table 5: Status of External Audit Unit Positions
As of December 31, 2005
Position
Total
Positions Filled Vacant
Manager 1 1
Audit Supervisor 3 2 1
Construction Auditor 4 1 3
Consultant Auditor 8 5 3
Total 16 9 7
Source: Auditor General staff analysis of ADOT’s Office of Audit
and Analysis Organizational Chart and vacancy data.
conducting audits is intended to ensure that projects with the highest risk factors
are audited. The risk assessment should identify characteristics of contracts and
projects that have a higher probability of errors and may demand more audit
attention. However, the Office does not select these audits based on a risk
assessment approach.
Additionally, the Office lacks information needed to manage its workload and
staff resources. First, it has an inadequate database system for tracking and
monitoring its work. For example, the Office could not provide auditors with
numbers of audits scheduled, in progress, and completed for recent fiscal
years. As a result, auditors could not confirm the reported backlogs. Although
the Office could use other methods to monitor audit functions, such as
spreadsheets and manual calculations, one manager stated that increasing
workloads and staff shortages made these methods impracticable. Second, the
Office stopped using performance measures and producing annual reports and
various activity reports in 1998.
Although ADOT upper management indicated awareness of the Office’s
declining productivity and other problems in December 2004, significant actions
to change Office operations were not taken until 2005. Such actions included
beginning recruitment for a new chief auditor in August 2005 and his
subsequent appointment in January 2006. However, upper management’s
closer, ongoing review of the Office may have resulted in more timely actions to
correct the longstanding deficiencies.
Department has begun to address problems
Although problems identified in this audit remain, ADOT has begun to take steps to
address those issues. ADOT’s new chief auditor indicated that the Office plans to:
􀁺􇨠 Fill staff vacancies—Between January and March 2006, the Office hired three
auditors. Additionally, the Office has established an internship program for
college students who would gain experience and might eventually be employed
with the Office. In April 2006, the Office participated in the State Job Fair with
hopes to fill entry-level auditor and intern positions. The Office hopes to fill all of
its vacancies by July 2006.
􀁺􇩒 Reinstitute annual reporting—The Office plans to issue a Fiscal Year 2006 Annual
Report based on available data and expand that report to incorporate additional
information in fiscal year 2007.
State of Arizona
page 32
The Office could not
provide the number of
audits scheduled or
completed.
The Office hopes to fill
all of its vacancies by
Jluy 2006.
􀁺􇩒 Reinstitute performance measurement practices—By July 1, 2006, the Office
plans to reinstitute performance measures it used in 1998 and implement any
appropriate additional measures.
􀁺􇩒 Revise the Office’s audit manual—By July 31, 2006, the Office plans to revise its
audit manual to reflect current business practices and ensure sufficient audit
coverage. For example, office policy requires construction cost audits in $20
million intervals and does not require audits of all lump-sum contracts. This does
not reflect the monetary growth of projects or changing department contracting
practices. According to ADOT management, very few projects were over $20
million when that interval was established. Now, however, many projects exceed
that amount, and a higher interval may be appropriate. Similarly, ADOT’s
contracting practices have changed. In contrast to prior years when few
contracts were paid on a lump-sum basis, these contracts now account for
more than 60 percent of new design and construction contracts. Risks
associated with lump-sum contracts include contractors using lower-quality
materials or less-skilled employees, or charging unreasonable change order
amounts.
􀁺􇩉 Implement a risk-based approach to audit selection—By July 1, 2006, the Office
plans to apply a risk-based approach to prioritizing audit projects with its existing
resources.
􀁺􇩒 Replace the Office’s database system—The Office plans to obtain and
implement by December 31, 2006, a computer-based audit management
system for audit scheduling, work papers, and reporting.
As the Office proceeds with its efforts to fill vacancies and improve its audit planning,
it should also consider:
􀁺􇩆 Filling positions, if necessary, with auditors who lack cost accounting experience
but meet other requirements, and provide needed job training;
􀁺􇩐 Preparing an annual audit work plan that estimates its workload and prioritizes
its audits based on available resources, and revise it as circumstances change;
􀁺􇨠 Measuring the number and types of audits requested and conducted, and audit
timeliness and results; and
􀁺􇨠 Revising its audit procedures to reflect changes in ADOT’s business practices.
Office of the Auditor General
page 33
Recommendations:
1. The Office should continue its efforts to:
a. Fill vacant positions and, if necessary, develop new recruitment strategies.
If the Office cannot fill positions with experienced auditors, it might consider
hiring auditors without cost accounting experience and providing training or
offering an internship program.
b. Implement performance measurements to monitor its production and work
activities. This should include the number and types of audit requests
received and conducted, the timeliness of completing audits, and audit
results.
c. Ensure that the highest-risk projects are audited by applying a risk-based
approach to selecting and conducting audits that considers items such as
staffing available to complete audits, dollar thresholds at which audits
should be conducted, and office audit requirements for each type of
contract.
d. Replace its database system and obtain a system that can track and
schedule workload and measure production.
e. Annually estimate its workload and prioritize its audits based on available
resources. This plan should be documented in an annual audit work plan
and revised with changing circumstances.
f. Revise its audit manual to reflect changes in business practices and ensure
that it provides adequate audit coverage of department projects.
State of Arizona
page 34
Office of the Auditor General
ENDNOTES
page a-i
State of Arizona
page a-ii
Office of the Auditor General
page a-iii
1. ITD also funds 11 positions (3 vacant) that are not located in an ITD group.
2. Aldhizer III, George R., James D. Cashell. Government Contract Auditing: Best
Practices from New York City’s Metropolitan Transportation Authority. The
Journal of Government Financial Management. 2003. 44-49.
3. Aldhizer III, George R., James D. Cashell, and Rick Eichmann. Construction
Auditing. Internal Auditor, Feb. 1999.
4. Warne, Thomas R., State DOT Outsourcing and Private Sector Utilization: A
Synthesis of Highway Practice. Washington D.C.: National Cooperative Highway
Research Program for the Transportation Research Board of the National
Academies, 2003.
5. Kingsley, Gordon, Sheldon Gen, Sue Gosnell, Cliff Lipscomb, Dara O’Neil, and
Patrick Wolfe. Strategies to Strengthen Consultant Management in the Georgia
Department of Transportation, Task Report 1: A Literature Review of Consultant
Management. Atlanta: Georgia Institute of Technology, Oct. 2003.
6. In January 2003, Governor Janet Napolitano established the Efficiency Review
Initiative to improve the state government’s performance and efficiency. The
Governor’s Efficiency Review team is charged with finding practical and sensible
ways for state agencies to (1) reduce costs, (2) cut bureaucracy, (3) eliminate
duplication, and (4) improve customer service.
7. Calderon, Eduardo, Rick West, Terri Jurkofsky, Howe Crockett, and Daniel
Alexander II. Contracting Out: Bench Marking Study. Washington D.C.: Federal
Lands Highway, Executive Quality Council of the Federal Highway
Administration, Department of Transportation, Sept. 2000.
8. ADOT 5-year Transportation Facilities Construction Programs reports for fiscal
years 1995 through 1999, and for fiscal years 2006 through 2010.
9. Wilmot, Chester G., Donald R. Deis, Helmut Schneider, and Charles H. Coates,
Jr. In-House Versus Consultant Design Costs in State Departments of
Transportation. Paper No. 99-1403. Baton Rouge, LA: Transportation Research
Record, 1999.
Endnotes:
continued
State of Arizona
page a-iv
10. Wilmot. Chester G., Investigation into the Cost-Effectiveness of Using
Consultants Versus In-house Staff in Providing Professional Engineering Services
for Louisiana’s Department of Transportation and Development. Baton Rouge,
LA: Louisiana Transportation Research Center, June 1995.
11. Warne, Thomas R., State DOT Outsourcing and Private Sector Utilization: A
Synthesis of Highway Practice. Washington D.C.: National Cooperative Highway
Research Program for the Transportation Research Board of the National
Academies, 2003.
12. Smith, Larry, Tony Welch, Rick West, Patrick Wlashchin, Eduardo Calderon, Scott
Rustay, Bill Pollock, and Ron Recker. Phase II: Benchmarking Study.
Washington, D.C.: Federal Lands Highway, Executive Quality Council of the
Federal Highway Administration, Department of Transportation, Nov. 2000.
13. Rogge, David F., Tomas Carbonell, and Randy Hinrichsen. Evaluation of Oregon
Department of Transportation Project Delivery: Outsourcing Project Delivery in
State Departments of Transportation. Corvallis, OR: Department of Civil,
Construction and Environmental Engineering at Oregon State University, Dec.
2003.
14. Kingsley, Gordon, Sheldon Gen, Sue Gosnell, Cliff Lipscomb, Dara O’Neil, and
Patrick Wolfe. Strategies to Strengthen Consultant Management in the Georgia
Department of Transportation, Task Report 1: A Literature Review of Consultant
Management. Atlanta: Georgia Institute of Technology, Oct. 2003.
15. Kingsley, Gordon, Sheldon Gen, Sue Gosnell, Cliff Lipscomb, Dara O’Neil, and
Patrick Wolfe. Strategies to Strengthen Consultant Management in the Georgia
Department of Transportation, Task Report 1: A Literature Review of Consultant
Management. Atlanta: Georgia Institute of Technology, Oct. 2003.
16. Transportation Research Board. The State of Transportation: Findings from the
Transportation Research Board’s 2004 Field Visit Program. TR News, Jan.-Feb.
2005. Federal Highway Administration Office of Professional Development.
Innovative Practices in State DOT Workforce Management: Minnesota
Succession Planning.
17. Deis, Donald R., Edward Watson, and Chester G. Wilmot. Designing a
Comprehensive Model to Evaluate Outsourcing of Louisiana DOTD Functions
and Activities. Baton Rouge, LA: Louisiana Transportation Research Center,
June 2002. Warne, Thomas R., State DOT Outsourcing and Private Sector
Utilization: A Synthesis of Highway Practice. Washington D.C.: National
Endnotes (continued):
continued
Office of the Auditor General
page a-v
Cooperative Highway Research Program for the Transportation Research Board
of the National Academies, 2003. Joint Legislative Audit and Review
Commission of the Virginia General Assembly. Review of the Use of Consultants
by the Virginia Department of Transportation. Richmond, VA: State of Virginia,
Nov. 1998. Wilmot. Chester G., Investigation into the Cost-Effectiveness of Using
Consultants Versus In-house Staff in Providing Professional Engineering Services
for Louisiana’s Department of Transportation and Development. Baton Rouge,
LA: Louisiana Transportation Research Center, June 1995. Hancher, Donn E.,
April Brenneman, Robin Meagher, and Paul Goodrum. Outsourcing of KyTC
Project Delivery Functions. Lexington, KY: Kentucky Transportation Center at the
University of Kentucky’s College of Engineering, June 2005.
18. Deis, Donald R., Edward Watson, and Chester G. Wilmot. Designing a
Comprehensive Model to Evaluate Outsourcing of Louisiana DOTD Functions
and Activities. Baton Rouge, LA: Louisiana Transportation Research Center,
June 2002.
19. The Office estimated this number by preparing a list of open pre-award audit
requests from its HEAT Audit system and distributed the list among office staff
to eliminate duplicate entries and audits previously issued. Office of the Auditor
General staff analysis had determined that data in the Office’s HEAT Audit
system was incomplete and unreliable, and auditors could not verify the Office’s
estimate of backlogged pre-award audits.
20. The Office could not locate the files for the remaining three reports, and therefore
their timeliness cannot be determined.
Endnotes (concluded):
concluded
State of Arizona
page a-vi
Office of the Auditor General
AGENCY RESPONSE
State of Arizona
04-05 Department of Environmental
Quality—Water Quality Division
04-06 Department of Environmental
Quality—Waste Programs
Division
04-07 Department of Environmental
Quality—Air Quality Division
04-08 Department of Environmental
Quality—Sunset Factors
04-09 Arizona Department of
Transportation, Motor Vehicle
Division— State Revenue
Collection Functions
04-10 Arizona Department of
Transportation, Motor Vehicle
Division—Information Security
and E-government Services
04-11 Arizona Department of
Transportation, Motor Vehicle
Division—Sunset Factors
04-12 Board of Examiners of Nursing
Care Institution Administrators
and Assisted Living Facility
Managers
05-L1 Letter Report—Department
of Health Services—
Ultrasound Reviews
05-01 Department of Economic
Security—Division of
Employment and
Rehabilitation Services—
Unemployment Insurance
Program
05-02 Department of Administration—
Financial Services Division
05-03 Government Information
Technology Agency (GITA) &
Information Technology
Authorization Committee (ITAC)
05-04 Department of Economic
Security—Information Security
05-05 Department of Economic
Security—Service Integration
Initiative
05-06 Department of Revenue—Audit
Division
05-07 Department of Economic
Security—Division of
Developmental Disabilities
05-08 Department of Economic
Security—Sunset Factors
05-09 Arizona State Retirement
System
05-10 Foster Care Review Board
05-11 Department of Administration—
Information Services Division
and Telecommunications
Program Office
05-12 Department of Administration—
Human Resources Division
05-13 Department of Administration—
Sunset Factors
05-14 Department of Revenue—
Collections Division
05-15 Department of Revenue—
Business Reengineering/
Integrated Tax System
05-16 Department of Revenue
Sunset Factors
06-01 Governor’s Regulatory Review
Council
06-02 Arizona Health Care Cost
Containment System—
Healthcare Group Program
06-03 Pinal County Transportation
Excise Tax
06-04 Arizona Department of
Education—Accountability
Programs
Performance Audit Division reports issued within the last 24 months
Future Performance Audit Division reports
Arizona Department of Education—Administration and Allocation of Funds
Arizona Department of Education—Information Management Function
Arizona Supreme Court—Administrative Office of the Courts—Information Technology and
FARE Program