Performance audit, Department of Environmental Quality, Vehicle Emissions Inspection Program

              Debra K. Davenport
Auditor General
Performance Audit
Department of
Environmental Quality—
Vehicle Emissions Inspection Program
Performance Audit Division
December • 2007
REPORT NO. 07-12
A REPORT
TO THE
ARIZONA LEGISLATURE
The is appointed by the Joint Legislative Audit Committee, a bipartisan committee composed of five senators
and five representatives. Her mission is to provide independent and impartial information and specific recommendations to
improve the operations of state and local government entities. To this end, she provides financial audits and accounting services
to the State and political subdivisions, investigates possible misuse of public monies, and conducts performance audits of
school districts, state agencies, and the programs they administer.
The Joint Legislative Audit Committee
Audit Staff
Copies of the Auditor General’s reports are free.
You may request them by contacting us at:
Office of the Auditor General
2910 N. 44th Street, Suite 410 • Phoenix, AZ 85018 • (602) 553-0333
Additionally, many of our reports can be found in electronic format at:
www.azauditor.gov
Senator Robert Blendu, Chair Representative John Nelson, Vice-Chair
Senator Carolyn Allen Representative Tom Boone
Senator Pamela Gorman Representative Jack Brown
Senator Richard Miranda Representative Peter Rios
Senator Rebecca Rios Representative Steve Yarbrough
Senator Tim Bee (ex-officio) Representative Jim Weiers (ex-officio)
Melanie M. Chesney, Director and Contact Person
Brent Nelson, Team Leader
Karl Kulick
Cheya Wilson
Shaun Yunt
DEBRA K. DAVENPORT, CPA
AUDITOR GENERAL
STATE OF ARIZONA
OFFICE OF THE
AUDITOR GENERAL
WILLIAM THOMSON
DEPUTY AUDITOR GENERAL
2910 NORTH 44th STREET • SUITE 410 • PHOENIX, ARIZONA 85018 • (602) 553-0333 • FAX (602) 553-0051
December 3, 2007
Members of the Arizona Legislature
The Honorable Janet Napolitano, Governor
Mr. Stephen A. Owens, Director
Arizona Department of Environmental Quality
Transmitted herewith is a report of the Auditor General, A Performance Audit of the Arizona
Department of Environmental Quality (Department)—Vehicle Emissions Inspection
Program (Program). This report is in response to an October 5, 2006, resolution of the
Joint Legislative Audit Committee and was conducted as part of the sunset review process
prescribed in Arizona Revised Statutes §41-2951 et seq. but does not include an evaluation of
the 12 sunset factors because Laws 2007, Ch. 171, §2 extended the Program’s sunset date until
January 1, 2017.
As outlined in its response, the Department agrees with the finding and plans to
implement all of the recommendations.
My staff and I will be pleased to discuss or clarify items in the report.
This report will be released to the public on December 4, 2007.
Sincerely,
Debbie Davenport
Auditor General
Enclosure
The Office of the Auditor General has conducted a performance audit of the
Department of Environmental Quality—Vehicle Emissions Inspection Program
(Program) pursuant to an October 5, 2006, resolution of the Joint Legislative Audit
Committee. This audit was conducted as part of the sunset review process
prescribed in Arizona Revised Statutes (A.R.S.) §41-2951 et seq.1 This audit
evaluated how well the Department of Environmental Quality (Department) monitors
the contractor that conducts the emissions tests.
The Program, begun in 1976, is limited to the Phoenix and Tucson metropolitan
areas. An independent consultant reported in 2005 that the Phoenix Program
effectively reduced emissions by requiring vehicles that fail tests to be repaired. In
fiscal year 2007, more than 1.4 million vehicles were tested, and of these, nearly
200,000 failed the test on the first try. About 150,000 received repairs and passed a
subsequent test. The current contractor, who administers 13 testing stations in
Phoenix and 3 in Tucson, has held the contract since 1991. In July 2007, the
Department again awarded the contract to this contractor for a 7-year period
beginning in January 2009. The new contract includes provisions for lower vehicle
testing fees, three new testing stations in the Phoenix area, and shorter customer wait
times.
Department should adjust its monitoring activities to
improve effectiveness, efficiency, and coverage (see
pages 13 through 21)
The contractor and the Department have developed a good framework for ensuring
that vehicle emissions testing is properly conducted, with the contractor performing
most of the quality control activities. Both parties periodically audit vehicle testing
equipment to ensure accurate measurements. The contractor also has a program to
ensure the equipment is properly calibrated, and the Department receives reports
that can allow it to see if any testing lanes deviate significantly from the average in
their pass/fail rates. The contractor and the Department also review the performance
of the inspectors who conduct the tests.
Office of the Auditor General
SUMMARY
page i
1 A sunset audit normally includes an evaluation of the agency using 12 factors set forth in A.R.S. §41-2954. This report
does not include an evaluation using the 12 factors because Laws 2007, Ch. 171, §2 extended the Program's sunset
date until January 1, 2017.
Although the contractor and the Department have developed a good framework, the
Department may be able to reduce the number of its audits. The Department's rules
require it to perform more equipment audits than federally required and the
contractor conducts many equipment audits. In addition, the contractor's program
for ensuring that the equipment is properly calibrated will not allow employees to use
the equipment if the checks are not performed or if the equipment is found out of
calibration. Similarly, both the Department and the contractor are reviewing
inspectors' performance. Contract monitoring best practices indicate that monitoring
frequency and extent can be reduced when results demonstrate consistent,
satisfactory performance. The Department should evaluate whether it can reduce its
number of audits and still meet monitoring objectives.
Reducing the number and frequency of equipment and inspector audits would allow
the Department to use the freed-up resources to address other aspects of the
monitoring process. For example, although audits are frequently conducted, there is
a lack of followup to ensure that equipment failing audits is repaired and that the
contractor takes corrective actions when inspectors fail audits. In realigning its efforts,
the Department would benefit from developing a comprehensive contract monitoring
plan, including assessing whether employees need additional training to perform the
new monitoring practices.
State of Arizona
page ii
Office of the Auditor General
TABLE OF CONTENTS
continued
page iii
Introduction & Background 1
Finding 1: Department should adjust its monitoring activities
to improve effectiveness, efficiency, and coverage 13
Good overall quality control and monitoring framework in place 13
Department may be able to redirect its monitoring practices 15
Recommendations 20
Appendix a-i
Agency Response
TABLE OF CONTENTS
concluded
page iv
State of Arizona
Tables:
1 State-Wide Emissions Test Results for Contractor Initial Tests
Fiscal Year 2007 4
2 Maximum Repair Cost Limits to Qualify for Waivers
As of July 5, 2007 5
3 Emissions Testing Fees as of May 14, 2007 and
Under the New Contract Effective January 2, 2009 9
4 Schedule of Revenues, Expenditures, and Changes in Fund Balance
Fiscal Years 2005 through 2007
(Unaudited) 10
Figures:
1 Areas Requiring Emissions Testing in Arizona
As of July 30, 2007 2
2 Emissions Test Failure Rate by Vehicle Model Year
Calendar Year 2006 3
3 Comparison of Contractor Initial Tests and
Fleet Self-Tests
Fiscal Years 2003 through 2007 5
The Office of the Auditor General has conducted a performance audit of the
Department of Environmental Quality—Vehicle Emissions Inspection Program
(Program) pursuant to an October 5, 2006, resolution of the Joint Legislative Audit
Committee. This audit was conducted as part of the sunset review process
prescribed in Arizona Revised Statutes (A.R.S.) §41-2951 et seq.1 This audit
evaluated how well the Department of Environmental Quality (Department) monitors
the contractor that conducts the emissions tests.
Program history
The State of Arizona established the Vehicle Emissions
Inspection Program in 1976 to reduce vehicle emissions
and improve air quality.2 The Legislature limited the Program
to the Phoenix and Tucson metropolitan areas, where higher
vehicle populations reduced air quality. Because these
areas did not meet federal air quality standards by 1982, the
State had to include the vehicle emissions inspection
program in its State Implementation Plan (see textbox). The
Program, which originally fell under the jurisdiction of the
Department of Health Services, has functioned under the
Arizona Department of Environmental Quality since the
Department's creation in 1987. Since its inception, the
Program has consisted of a centralized vehicle emissions testing program with most
vehicle emissions testing services provided by an independent contractor. Motorists
can choose to have the Department test their vehicles if the vehicles have twice failed
emissions tests performed by the contractor.
1 A sunset audit normally includes an evaluation of the agency using 12 factors set forth in A.R.S. §41-2954. However, this
report does not include an evaluation using the 12 factors, because Laws 2007, Ch. 171, §2 extended the Program's
sunset date until January 1, 2017.
2 Although at inception the Program covered only Maricopa and Pima Counties, the boundaries have changed over time.
As of 2007, A.R.S. §49-542 requires emissions testing in Area A, which includes parts of Maricopa, Pinal, and Yavapai
Counties, and in Area B, which includes part of Pima County. This report will refer to Area A as the Phoenix area and Area
B as the Tucson area. Other areas in the State are not required to perform testing and, according to the Department, local
governments do not elect to require emissions testing.
Office of the Auditor General
INTRODUCTION
& BACKGROUND
page 1
State Implementation Plan (SIP)
The federal Clean Air Act amendments of 1970
required the Environmental Protection Agency
(EPA) to establish standards to protect public
health from air pollution. Amendments passed in
1977 called on states to attain national air quality
standards for carbon monoxide and ozone—two
pollutants caused primarily by vehicle
emissions—no later than December 31, 1982.
States were required to develop and implement
State Implementation Plans, which detailed how
the national standards would be attained.
Some vehicles are
exempted from
emissions testing
requirements.
Program regulations have changed over time. At
inception, the Legislature established both the Phoenix
and Tucson areas as "basic" inspection and
maintenance (I/M) programs that tested vehicles for
carbon monoxide (CO) and hydrocarbon (HC)
emissions. Vehicles not meeting EPA emissions
standards for these gases could not be registered until
they were repaired and met the standards. In 1995,
because Phoenix did not meet EPA air quality
standards, the Legislature changed the Phoenix area to
an "enhanced" I/M program, which has more stringent
emissions and testing requirements than the basic I/M
program. For example, vehicles must also meet oxides
of nitrogen (NOx) emissions standards and 1981
through 1995 vehicles must take a transient load test
(see Appendix, page a-i) which is not required of
vehicles in the Tucson area. Because of better air quality
than Phoenix, the Tucson area has continued as a basic
I/M program. As population growth has increased over
the years, the Phoenix testing area has expanded.
Figure 1 shows the Phoenix and Tucson testing areas'
boundaries as of July 30, 2007. Additionally, in 2007 the
Legislature passed air quality legislation to help reduce
ozone levels in the Phoenix area by requiring a test to identify and repair liquid fuel
leaks in vehicles.1
Program provisions
The Program operates to identify vehicles with emissions that exceed EPA standards
and to require repair so that vehicles will meet the standards. Most vehicles located
in the Phoenix and Tucson areas must successfully pass an emissions test before
registering with Arizona's Motor Vehicle Division, but some vehicles are exempt from
testing. Although the majority pass the initial test, almost 14 percent fail. Most of
those vehicles later pass after some repairs; some of those that fail may qualify for a
one-time waiver and become registered.
Vehicles requiring testing—Vehicles located in the Phoenix and Tucson testing
areas generally must pass emissions testing annually or biennially before being
registered. In addition, most vehicles located outside the Phoenix and Tucson
testing areas that regularly commute into these areas must also be tested. The
required emissions tests and frequencies differ by test area, vehicle type, and
vehicle age (see Appendix, page a-i). Arizona statutes exempt some categories of
vehicles from emissions testing, including:
1 Although effective on September 19, 2007, according to a department official, the legislation cannot be implemented until
at least mid-2008 because of required processing steps, such as department development of new testing procedures,
rule-making, changes in the testing contract, contractor creation of new testing procedures, and contractor employee
training.
State of Arizona
page 2
Figure 1: Areas Requiring Emissions Testing in Arizona
As of July 30, 2007
Department’s Web site as of July 30
Source: Auditor General staff analysis of information contained on the
, 2007
Source: Auditor General staff analysis of information contained on the
Department’s Web site as of July 30, 2007.
􀁺􇩖 Vehicles manufactured before 1967 and most vehicles of the current model
year and four prior model years;
􀁺􇩇 Golf carts, rare vehicles, and historic collectible vehicles; and
􀁺􇩍 Motorcycles in the Tucson area (in the Phoenix area, motorcycle testing and
repairing provides significant emissions reductions, according to a 2004
department evaluation).
Vehicle age is a significant factor in failure rates, as shown in Figure 2. For example,
for vehicles tested in calendar year 2006, only 5.5 percent of model year 2002
vehicles failed their initial emissions tests as compared to 44.1 percent of model year
1979 vehicles.
Testing results—The Program has identified a significant number of over-polluting
vehicles, most of which were repaired and became emissions-compliant. As
illustrated in Table 1 (see page 4), in fiscal year 2007, almost 14 percent of the
more than 1.4 million contractor-tested vehicles state-wide failed their initial
emissions tests, but most were repaired and passed a retest. Specifically, in fiscal
year 2007 the contractor tested 998,488 vehicles in the Phoenix area and 417,530
vehicles in the Tucson area. Phoenix-area vehicle failure rates were at 15.2 percent,
which was higher than the 10.6 percent Tucson-area failure rate, probably because
of Phoenix's more stringent emission standards. According to the contractor's
Office of the Auditor General
page 3
0
10
20
30
40
50
1967-69
1970-72
1973-75
1976-78
1979-81
1982-84
1985-87
1988-90
1991-93
1994-96
1997-99
2000-02
Failure Rate (Percent)
Model Years
Figure 2: Emissions Test Failure Rate by Vehicle Model Year
Calendar Year 20061
1 Model years 2003 and later were exempt from vehicle emissions testing in calendar year 2006.
Source: Auditor General staff analysis of the Department’s inspection statistics by model year for calendar year 2006.
Although 14 percent of
vehicles failed initial
emissions testing, most
later passed after being
repaired.
records, 46,190, or 3.3 percent, of contractor-tested vehicles did not become
emissions-compliant and were not registered in the Phoenix or Tucson areas.
Vehicles that do not pass emissions testing or receive a test waiver cannot be
registered and operated within the test area.
Failing vehicles—In accordance with statute, the owner of a vehicle failing the
initial emissions test has several options. First, the owner can have the vehicle
repaired and receive one free retest within 60 days of the initial test. A vehicle failing
the second time may receive further repairs and be tested again, but the owner
may have to pay another testing fee. Additionally, the owner may choose to
remove the failed vehicle from operation. Finally, if certain requirements are met,
the owner can go to one of the two department-operated waiver stations and
request a once-in-a-vehicle-lifetime certificate of waiver. This waiver allows the
vehicle to be registered for one registration cycle without passing emissions
testing. As shown by Table 1, less than 0.1 percent of all tested vehicles received
waivers in fiscal year 2007.
The Department may grant a waiver when a vehicle meets several requirements:
􀁺􇩴 the vehicle's emissions malfunction was diagnosed and the vehicle received
a low emissions tune-up following the initial failure;
􀁺􇩩 it failed the contractor's emissions test at least two times;
􀁺􇩩 it is not emitting more than two times the applicable emission standard;
􀁺􇩩 it does not have a faulty catalytic converter; and
􀁺􇩡 additional repair costs to reduce emissions levels would exceed the maximum
repair cost limits established in department rules (see Table 2, page 5).
State of Arizona
page 4
Table 1: State-wide Emissions Test Results for Contractor Initial Tests
Fiscal Year 2007
Result Number of Vehicles
Percentage of
Initial Tests
Initial test results:
Vehicles that initially passed 1,220,290 86.18%
Vehicles that initially failed 195,728 13.82
Total vehicles initially tested 1,416,018 100.00%
Outcomes for the 195,728 failed vehicles:
Passed later after repairs 148,207 10.47%
Granted a waiver 1,331 0.09
No record of passing or registering 46,190 3.26
Total vehicles that initially failed 195,728 13.82%
Source: Auditor General staff analysis of the contractor’s data on initial free retest, and paid retest reports for fiscal year 2007.
Table 1: State-wide Emissions Test Results for Contractor Initial Tests
Fiscal Year 2007
Source: Auditor General staff analysis of the contractor’s data on initial test results, waivers,
free
retest, and paid retest reports for fiscal year 2007.
Fleet vehicles may self-test—Although the contractor provides most testing,
almost 5 percent of emissions testing is done by fleet self-test facilities. Arizona law
allows fleet owners who lease or own 25 or more vehicles to self-test their own
vehicles for emissions compliance. If electing this option, the fleet owner applies
for a permit from the Department and is subject to the same vehicle emissions
testing requirements as the contractor, with the exception that car dealers with fleet
permits only perform low-idle and high-idle (2,500 rpm) tests. Figure 3 shows the
number of contractor and fleet initial vehicle emissions tests for fiscal years 2003
through 2007.
Almost 5 percent of
vehicles that require
emissions testing are
self-tested by fleet
owners.
Office of the Auditor General
page 5
Table 2: Maximum Repair Cost Limits to Qualify for Waivers
As of July 5, 2007
Vehicle Description Phoenix Area Tucson Area
1980 and newer $450 $300
1975—1979 300 200
1974 and older 200 50
Heavy-duty diesel 500 300
Source: Arizona Administrative Code R18-2-1010 as of July 5, 2007.
Number of Vehicles
0
200
400
600
800
1,000
1,200
1,400
1,600
2003 2004 2005 2006 2007
Source: Auditor General staff analysis of emissions testing statistics and monthly sales from fleet
self-test certificates of inspection provided by the Department for fiscal years 2003 through 2007.
1,206K
67K
1,416K
73K
1,395K
67K
1,322K
59K
1,174K
68K
(In Thousands)
Contractor Initial Tests Fleet Self -Tests
Figure 3: Comparison of Contractor Initial Tests and
Fleet Self-Tests
Fiscal Years 2003 through 2007
Source: Auditor General staff analysis of emissions testing statistics and monthly sales from fleet self-test and
certificates of inspections provided by the Department for fiscal years 2003 through 2007.
The new testing contract
provides lower test fees
and higher performance
standards.
Testing performed at contractor facilities—With the exceptions of fleet self-testing
and testing performed at the department-run waiver stations, all testing is
performed by the contractor using testing facilities and equipment it owns. The
current contractor, Gordon-Darby Inc., has provided vehicle emissions testing in
Arizona since 1991 and as of July 2007 operated 13 testing stations with 56 lanes
in Phoenix and 3 testing stations with 13 lanes in Tucson. Effective July 25, 2007,
the Department awarded a new 7-year contract to Gordon-Darby Arizona Testing
with new features effective January 2, 2009.
As stipulated in the current contract, the contractor's operation of its testing
facilities must meet several contract-required performance standards or the
contractor must pay a fine called "liquidated damages" for loss of productivity. As
of July 2007, the Department reported that the contractor had always met these
standards and therefore never had to pay any liquidated damages. The
contractor's failure to meet ongoing performance standards results in liquidated
damages when:
􀁺􇩍 More than 40 percent of customers wait longer than 15 minutes, or more than
20 percent of customers wait longer than 30 minutes;
􀁺􇩔 Testing lane downtime exceeds 0.5 percent of available lane hours;
􀁺􇩎 None of a station's lanes are able to provide testing services for more than a
total of 4 hours per day; and
􀁺􇩄 Daily, weekly, monthly, quarterly, semi-annual, and annual reports are
submitted late to the Department.
The new contract starting on January 2, 2009, will provide several customer service
improvements, including lower vehicle testing fees (see Table 3, page 9), higher
performance standards regarding customer wait times, additional testing
equipment to reduce customer wait times, three new testing stations in the Phoenix
area, and acceptance of credit card payments. Additionally, the contractor plans
to implement several automated system improvements, including improved
security cameras for identifying license plate numbers of vehicles being tested and
computer system changes to identify vehicles that do not require testing or have
already successfully passed a test within the past 90 days. According to the
Department, this last improvement is in response to previous customer
complaints.
State of Arizona
page 6
Program effectiveness
Independent consultants have reported on the Program's effectiveness. The Eastern
Research Group’s (ERG) June 30, 2005, report evaluated the Phoenix-area
emissions testing program and found that the Program effectively reduced emissions
by requiring vehicles that fail tests to be repaired. ERG found that the Program
reduced emissions for dangerous gases such as hydrocarbons, carbon monoxide,
and nitrogen oxide. However, based upon information in an ERG 2006 report,
emissions could be further reduced by ensuring that vehicles registered outside the
Phoenix and Tucson areas are not commuting into these areas without receiving
emissions testing. Also, the 2006 report stated that data from vehicles tested by both
tailpipe measurements and the relatively new "on-board diagnostic" (OBDII) test
indicate that the OBDII may not be identifying some vehicles designated as high-emitting
by tailpipe tests (see OBDII in Appendix, page a-i), but additional testing and
analysis are needed to verify OBDII performance. In 2007, the Department hired an
individual who served as a consultant for ERG on the previous studies to conduct
another effectiveness study of the Phoenix area. According to a department official,
the report will be available sometime after January 1, 2008.
Department's organization for emissions testing
The Department's Vehicle Emissions Inspection Section (Section) inspects the
contractor and the fleet self-inspection test sites to verify compliance with testing
requirements. The Section also conducts waiver testing and carries out
administrative functions. As of August 2007, the Department was divided into five
units with a total of 33 full-time equivalent (FTE) employees, of which 3 positions were
vacant. According to department management, these are:
􀁺􇩐 Phoenix Inspections and Compliance Unit (9 FTE, 1 vacant)—This unit monitors
Phoenix-area emissions testing activities and licenses and trains inspectors, and
researches new technology in both the Phoenix and Tucson areas. Three
positions provide supervisory and clerical support. Four positions inspect
contractor stations and fleet self-test facilities by conducting equipment and
inspector audits. According to the unit manager, the unit spends about 60
percent of its time monitoring contractor stations and 40 percent of its time
monitoring fleet stations. Two positions train, test, and license the fleet self-test
inspectors in both the Phoenix and Tucson areas. Although the contractor trains
and tests its own employees, the unit licenses the contractor inspectors in both
the Phoenix and Tucson areas who receive successful examination scores as
provided by the contractor. The unit also monitors contractor training classes.
The remaining two employee positions provide supervisory and clerical support.
Consultant studies
concluded that the
testing program has
effectively reduced
vehicle emissions.
Office of the Auditor General
page 7
The vehicle emissions
testing program is self-funded
by testing fees.
􀁺􇩐 Phoenix Operations Unit (9 FTE, 1 vacant)—This unit operates the emissions
waiver station in Phoenix, tests vehicles for emissions compliance, inspects and
evaluates repairs performed on vehicles failing emissions testing, makes
recommendations to vehicle owners regarding future potential repairs, and
determines if the vehicle meets all waiver requirements. This unit also tracks
waiver-lane statistics and calibrates waiver-lane equipment.
􀁺􇩔 Tucson Operations Unit (7 FTE, 0 vacant)—This unit performs for the Tucson
area most of the duties that are performed in the Phoenix area by both the
Phoenix Operations Unit and the Phoenix Inspections and Compliance Unit.
These duties include operating the Tucson waiver station and performing
equipment and inspector audits at fleet stations and the three Tucson contractor
stations.
􀁺􇩐 Program Support and Evaluation Unit (3 FTE, 1 vacant)—This unit receives and
reviews all required daily, weekly, monthly, quarterly, semiannual, and annual
reports from the contractor. It monitors financial and emissions testing activity
reports and reconciles information among the various contractor reports. This
unit provides required reporting to the EPA based upon contractor-provided
data. The unit also performs routine checks of financial reports to ensure
contractor billings are correct, analyzes data, and provides reports to
government, public, or private institutions.
􀁺􇩇 General Administrative Unit (5 FTE, 0 vacant)—This unit performs general
administrative functions for the entire emissions testing program. It also provides
office support for the Phoenix waiver station by staffing the front desk and the
call center, and assisting customers by processing payments for out-of-state/
out-of-area exemptions, self-test fleet inspection certificates, waiver and
inspection certificates, and out-of-state exemptions. Further, this unit partners
with the contractor to provide public information through a vehicle emissions
testing Web site that includes digitally captured screens of vehicle wait lines at
test sites, current wait time estimates, fee and waiver information, exemption
qualifications, and various other public information regarding the Program.
Budget
The Program is self-funded through test fees charged to motorists at the time of
inspection. The contractor collects the total test fee and deposits the monies directly
into a department bank account established exclusively to receive contractor testing
fees. The State Treasurer then transfers the money into the Emissions Inspection
Fund (Fund) based on a deposit form completed by the Department. The
Department then pays the contractor its fee amount from the Fund. The Department
determines test fees based on the types of tests administered, the vehicle model
State of Arizona
page 8
Office of the Auditor General
page 9
As of May 14, 2007 January 2, 2009
Contractor
Fee
Administrative
Fee
Total
Fee
Contractor
Fee
Total
Fee
Phoenix Area
1981 and newer cars and most light-duty trucks
(under 8,501 lbs.) that receive Transient Load
or On-Board Diagnostic II tests $25.94 $1.81 $27.75 $13.50 TBD1
Most other vehicles (except heavy-duty diesels) 17.05 1.95 19.00 13.50 TBD1
Heavy-duty diesel vehicles (greater than 8,500
lbs. GVW) 25.94 2.06 28.00 23.50 TBD1
Tucson Area2
All vehicles 11.39 0.86 12.25 NA NA
Biennial On-Board Diagnostic II test for 1996 and
newer light-duty vehicles NA NA NA 13.50 TBD1
All annual tests for 1967 to 1995 vehicles
including light-duty diesels NA NA NA 10.50 TBD1
Heavy-duty diesel vehicles (greater than 8,500
lbs. GVW) NA NA NA 23.50 TBD1
Table 3: Emissions Testing Fees as of May 14, 2007 and
Under the New Contract Effective January 2, 2009
1 As of August 24, 2007, the Department had not determined the administrative fee amount it will charge beginning January 2009.
TBD signifies that the amount is yet to be determined.
2 Prior to calendar year 2009, the Tucson testing fee is the same for all vehicles. Beginning in calendar year 2009, the testing fee will
differ by test type.
Source: Auditor General staff analysis of fee data provided by the Department and the contract pricing schedule included in the
awarded contract beginning calendar year 2009.
year, and the test area. In addition, the Department charges a $15 fee for all
waiver applicants, but does not charge the fee if a wavier is denied. The total test
fee paid by motorists consists of both a contractor testing fee component and
an administrative fee component. The administrative component is used to fund
the Department's administration and program oversight activities. Current and
future testing fees are shown in Table 3. Because of more stringent testing
procedures in Phoenix, testing fees and categories differ between the Phoenix
and Tucson areas.
State of Arizona
page 10
Table 4 below shows the Fund's actual revenues and expenditures for fiscal years
2005 through 2007. As shown by the table, most expenditures are for payments to
the contractor for emissions testing and other contract services.
Table 4: Schedule of Revenues, Expenditures, and Changes in Fund Balance
Fiscal Years 2005 through 2007
(Unaudited)
2005 2006 2007
Revenues:
Emission inspection program fees1 $31,485,003 $32,948,951 $33,667,495
Investment earnings 1,951 30,512 51,310
Total revenues 31,486,954 32,979,463 33,718,805
Expenditures:2
Personal services and employee-related 1,385,904 1,451,496 1,516,647
Professional and outside services3 29,745,716 29,615,792 30,284,928
Travel 46,888 46,473 39,554
Other operating 191,724 172,840 140,242
Equipment 13,794 6,910 21,832
Allocated costs4 515,834 540,034 555,527
Total expenditures 31,899,860 31,833,545 32,558,730
Excess (deficiency) of revenues over expenditures (412,906) 1,145,918 1,160,075
Fund balance, beginning of year 741,406 328,500 1,474,418
Fund balance, end of year5 $ 328,500 $ 1,474,418 $ 2,634,493
1 Consists primarily of testing fees collected by the Department’s vehicle emissions program vendor. The vendor
collected approximately $30.6, $32, and $32.8 million in 2005, 2006, and 2007, respectively.
2 Administrative adjustments are included in the fiscal year paid.
3 Consists primarily of contract payments made to the Vehicle Emissions Inspection Program vendor for operating the
Program in Maricopa and Pima Counties.
4 Amount is the portion of department-wide overhead expenditures allocated to the Vehicle Emissions Inspection
Program, such as administrative personnel, rent, general accounting, telecommunications system, and risk
management costs.
5 As required by A.R.S. §49-544, VEIP monies can be used only for program enforcement activities, vendor contractual
payments, program administrative costs, and the State’s portion of the catalytic converter program.
Source: Auditor General staff analysis of the Department’s Fund Balancing Worksheet for fiscal years 2005 through 2007.
The majority of program
spending comprises
payments to the
contractor.
Office of the Auditor General
page 11
Scope and methodology
This audit focused on the Department's monitoring of the vehicle emissions testing
program contractor. The report contains one finding and associated
recommendations that the Department should adjust its monitoring of the vehicle
emissions testing contractor to improve effectiveness, efficiency, and coverage.
Auditors used a number of methods to obtain information about the issues
addressed in this report. General methods included reviewing federal code, Arizona
statutes and administrative rules, the State Implementation Plan, the Vehicle
Emissions Front Office Policy and Procedure Manual, and the Inspection and
Compliance Unit Manual; and interviews with department management, staff, and
stakeholders, including EPA representatives. In addition, the following specific
methods were used:
􀁺􇩔 To evaluate the Department's contract monitoring activities, auditors observed
the inspections process alongside a department compliance officer while the
officer inspected a contractor test site; reviewed the 1994 and 2001 SIP
Revisions, the Inspection and Compliance Unit audit schedule, and audit
procedures contained in the Vehicle Emissions Section's Inspection and
Compliance Unit Manual. Further, auditors reviewed HC gas analyzer audit
reports for the contractor and the Department for 1 year to compare failure rates;
performed a file review of department audits of the contractor including all
inspector and equipment audits recorded from 2002 through 2007; and
reviewed department equipment failure notifications for March through June
2007. To assess department compliance with audit schedules and EPA
requirements, auditors reviewed department files for failed equipment
calibrations of gas analyzers for all four gases (HC, CO, carbon dioxide, and
NOx) from May 2006 through April 2007. To obtain information about customer
satisfaction, auditors reviewed the Marketing Intelligence survey reports for 2005
and 2006. Auditors reviewed authoritative literature regarding contract
management and contractor documents, such as the Surveillance and Audit
Plan, Preventative Maintenance Plan, Training Plan, and Standard Operating
Procedures Manual; and reviewed contractor reports, including the
Performance Information Report, Surveillance and Audit Report, Covert Audit
Report, and Audited Financial Statements.1 Finally, to observe department and
contractor relations, auditors attended one biweekly operations meeting
between the Department and the contractor where day-to-day station
operations issues such as Web site errors and test station traffic were
discussed, and attended one biweekly program manager meeting where
department and contractor staff discussed program issues such as testing
procedures and audits.
1 Literature reviewed included Texas Building and Procurement Commission. Contract Management Guide. Mar 1, 2007.
Texas Building and Procurement Commission. Aug. 21, 2007; National State Auditors Association. Contracting for
Services. A National State Auditors Association Best Practice Document. Lexington, KY: National State Auditors
Association, 2003; and the Washington State Office of Financial Management contract monitoring guidance found at
http://www.ofm.wa.gov/contracts/resources/managing_monitoring.pdf.
State of Arizona
page 12
􀁺􇩔 To prepare the Introduction and Background section, auditors reviewed
department-prepared information including an organization chart, program
history document, a comparison of emissions program characteristics among
32 states and the District of Columbia, and its Web site; documents from other
agencies, such as the 2005 EPA Inspector General Report, the Governor's Office
of Strategic Planning and Budgeting (OSPB) Master List of State Government
Programs 2005-2007, a July 2007 press release on emissions legislation, the
1988 Auditor General performance audit report on the Vehicle Emissions
Inspection Program (see Report No. 88-11), and the Joint Legislative Budget
Committee (JLBC) appropriations report for fiscal year 2008; and third-party
documents such as the July 2007 Gordon-Darby Best and Final Offer and the
2005 Eastern Research Group emissions program effectiveness study. Auditors
also reviewed the Department's fiscal years 2005, 2006, and 2007 Fund
Balancing Worksheet reports for the Vehicle Emissions Inspection Fund; Bank
of America bank statements; reports from the Department's system based on
information input from the bank statement and deposit slips; the Monthly
Deposit Reconciliation Report; the deposit slips for two deposits; a Gordon-
Darby Daily Recap Sheet and a weekly and monthly contractor billing to the
Department for emission services; a state treasurer deposit form; and a
department accounts payable spreadsheet.
This audit was conducted in accordance with government auditing standards.
The Auditor General and staff express appreciation to the Director and staff of the
Department of Environmental Quality and its Air Quality Division and Vehicle
Emissions Inspection Section for their cooperation and assistance throughout the
audit.
Department should adjust its monitoring activities
to improve effectiveness, efficiency, and coverage
A good overall vehicle emissions program quality control and monitoring framework
is in place, but the Department should adjust its monitoring to improve effectiveness,
efficiency, and coverage. Under the current framework, the contractor and the
Department both perform quality control and monitoring activities focused on
equipment testing, inspector performance, and customer service. The Department
may be able to reduce some monitoring activities and should redirect its resources
to follow up on identified problems and conduct other important monitoring activities.
Further, the Department should develop and implement a contract monitoring plan to
help ensure more efficient, effective, and comprehensive monitoring coverage.
Good overall quality control and monitoring framework in
place
The contractor and the Department have developed a good framework for ensuring
that vehicle emissions testing is properly conducted, with the contractor performing
most of the quality control activities. The quality control and monitoring activities
focus on the following three main areas:
􀁺􇩖 Vehicle Emissions Testing Equipment—The contractor and the Department are
required by the emissions testing program contract and federal regulations to
audit various emissions testing equipment to ensure accurate vehicle emissions
measurements. For example, they are required to test and audit the gas
analyzers that are used to measure tailpipe emissions to ensure that the
analyzers are properly calibrated and measure gas levels, such as carbon
dioxide and monoxide, within acceptable tolerance. It is vital that the gas
analyzers accurately measure harmful gas volumes to properly identify vehicles
exceeding allowable emissions levels.
Office of the Auditor General
page 13
The Department and the
contractor verify that
testing equipment works
properly.
FINDING 1
State of Arizona
page 14
The contractor's and the Department's equipment quality control and monitoring
includes several other activities. For example, the emissions testing contract
requires the contractor to implement a preventive maintenance and quality
control program and to properly calibrate and maintain testing equipment. To
meet these requirements, the contractor has established a series of periodic
automatic equipment calibration checks to ensure emissions testing equipment
will accurately measure vehicle emissions. Additionally, the Department receives
contractually required monthly reports from the contractor, such as vehicle
pass/fail rates that it can review to compare failing rates by lane to identify lanes
that significantly deviate from the average. Such a deviation may indicate that
equipment is not measuring correctly or inspectors are not testing appropriately.
􀁺􇩌 Lane Inspector Performance—The contractor and the Department also focus on
evaluating the performance of the lane inspectors through overt and covert
inspector audits (see textbox). Lane inspectors are contractor employees who
perform the vehicle emissions tests in the testing lanes. These audits, required
by the emissions testing program contract and federal regulations, are designed
to ensure that the lane inspectors perform the emissions tests according to
proper procedures, which are designed to ensure valid testing results.
Additionally, the contractor performs covert observations where it observes
inspectors through video camera surveillance.
Inspector performance audits are based on contractual and regulatory
requirements. For example, the emissions testing program contract requires the
contractor to submit a surveillance and audit plan. The contractor's plan
provides for quarterly overt inspector audits and at least one covert audit per
inspector position per year. Additionally, the Department's administrative rules
require the Department to perform a minimum of two overt lane inspector audits
for each inspection lane on a yearly basis. In addition, the Department receives
and reviews monthly contractor overt and covert inspector audit reports.
􀁺􇩃 Customer Service—The contractor and the Department also focus on providing
a high level of customer service. Customer service practices are based on
contractual and federal regulatory requirements. For example, federal
regulations require that the emissions test system be designed to provide
convenient service to motorists who are required to have their vehicles tested.
To ensure prompt and timely customer service, the contract includes
performance standards on customer wait times at testing stations and testing-lane
downtimes. The Department receives contractor reports that allow it to
verify that performance standards are met. The contract also requires the
contractor to provide a Web site with station wait times; the contractor went even
further by providing maps and station queue cameras that show actual waiting
lines. Additionally, the Department receives contractually required monthly
vehicle damage complaint and complaint resolution reports from the contractor
that can be used to monitor inspection station performance and contractor
customer relations interactions.
Overt Inspector Audits:
The inspectors are aware
that contractor
management or the
Department is auditing
their performance by
observing them performing
the emissions tests.
Covert Inspector Audits:
Persons unknown to
inspectors present vehicles
for emissions testing that
have been pre-set to fail
the inspections and record
whether the inspectors
conducted the tests
according to procedures.
Office of the Auditor General
page 15
Further, processes are in place for customers to comment on the quality of
services provided. Customers provide the feedback directly to the contractor
through customer comment forms, which are offered to the customer as an
option to address their comments or concerns regarding inspection station
personnel. The contractor then reports the feedback to the Department in a
monthly customer comment report. Additionally, contractor employees are
required to consult with owners of failed vehicles after the emissions tests to help
them understand why their vehicles failed. Finally, the public demonstrated its
satisfaction with the contractor's customer service efforts in independently
administered surveys in which 72.5 percent of respondents in 2005 and 76
percent in 2006 rated their satisfaction level at 4 or 5 on a 5-point scale with 5
being the highest level of satisfaction.1
In addition to these three areas, the Department oversees the emissions program's
financial controls by reconciling emissions test activity and contractor billing reports.
The Department reconciles contractor billing reports to emissions testing data on a
weekly and monthly basis as an internal accuracy check to verify that the contractor
correctly calculated billing amounts.2 Then monthly, the Department reconciles
contractor revenue deposits, based on deposit slips and bank statements, to
expected revenues, based on emissions test counts and billing reports.3
Department may be able to redirect its monitoring
practices
The Department should evaluate whether it can reduce some of its audits and shift
those resources to fulfilling federal requirements and verifying contractor compliance
with contract provisions that the Department has not monitored. In conjunction with
considering these changes, the Department should develop an annual plan for
monitoring the contractor's performance.
Department and contractor conduct many equipment and inspector
audits—The Department's rules require it to perform more equipment audits
than federally required, and the contractor conducts a large number of equipment
and inspector performance audits. In addition, the Department's and the
contractor's equipment audits find similar failure rates, which provides assurance
that the Department may be able to reduce its audit numbers and also monitor
contractor audit results to ensure satisfactory contractor performance. The
contract monitoring best practices indicate that when monitoring results
1 Marketing Intelligence. Phoenix/Tucson Emissions Testing Survey Results. Tucson, Arizona., Aug 18, 2005. Marketing
Intelligence. Emission Testing Knowledge and Campaign Efficacy. Tucson, Arizona. Sep. 7, 2006.
2 The Department reported that in the last 2 years the reconciliations rarely found discrepancies, but that it had frequently
found errors when the reconciliations were first implemented.
3 Minor discrepancies, less than .07 of 1 percent, have been found because of differences in test date and deposit date
for the collected emissions test fee amounts.
The Department and
contractor conduct
many audits.
Surveys in 2005 and
2006 have found a high
level of customer
satisfaction.
demonstrate consistent, satisfactory performance, monitoring frequency and
extent may be adjusted accordingly.1 Specifically:
􀁺􇩅 Equipment Audits—The Department's rules provide for more equipment
audits than federal regulations require, and the contractor's audit plan
provides even more audits. For example, although federal regulations require
that gas analyzer audits be conducted only semiannually, the Department's
administrative rules require it to conduct these audits every other month in the
Phoenix area and bimonthly in the Tucson area. In addition, the contractor
generally audits the analyzers at least monthly. Auditors reviewed contractor
automated reports for contractor and department gas analyzer audits
conducted in fiscal year 2007 and found that the Department conducted 242
audits and found a 2.07 percent failure rate, and the contractor conducted 760
audits and found a 1.32 percent failure rate. The difference between the two
failure rates is not statistically significant.
The contractor has additional system and internal controls, which reduce the
need for the Department to conduct equipment audits. In order to allow
emissions tests to be performed, the contractor's emissions testing system
requires periodic equipment calibration checks that verify equipment is
properly calibrated within tolerances. These automated checks must be done
every morning before lane opening, every 4 hours thereafter in Phoenix or
every 5 hours thereafter in Tucson, and immediately preceding each vehicle
emissions inspection. Any equipment failing a periodic check is
automatically locked out of operation and cannot be used for vehicle
emissions testing until properly calibrated or repaired. As an additional
check, the contractor reported that it reviews calibration reports daily to
identify marginal equipment (see textbox) and performs preventative
maintenance before that equipment exceeds allowable tolerances.
These checks help provide reasonable assurance that some failing
equipment will be identified in a timely manner. However, audits are still
needed because the system calibration checks mostly verify the zero
and the highest value that the equipment can read, while audits verify
accurate readings between those two points and more thoroughly
examine the equipment. In addition, according to a department official,
equipment operators can adjust equipment settings after failing an automatic
calibration and barely pass within tolerances, whereas the equipment may be
out of tolerance for other measurements and later exceed tolerance again for
the zero and high values. Further, the automatic calibrations check
measurements directly at the equipment, while audits simulate measurements
through the entire emissions testing system.
􀁺􇩉 Inspector Audits—Both the Department and the contractor conduct
inspector performance audits. The Department's rules require the Department
to conduct semi-annual inspector performance audits as required by federal
State of Arizona
page 16
Marginal Equipment
EPA regulations allow testing equipment
tolerances or accuracy to be plus or minus
4 percent of the highest value the
equipment can measure. The contractor
considers equipment to be marginal when
its reading tolerances reach plus or minus 3
percent.
1 Washington State Office of Financial Management contract monitoring guidance found at
http://www.ofm.wa.gov/contracts/resources/managing_monitoring.pdf.
regulation. In addition, the contractor, through its contract-required
surveillance and audit plan, has obligated itself to conduct these audits
quarterly for each of its inspectors. Auditors' review of department audit
reports for 2005 and 2006 found that the Department had conducted 135
inspection lane audits in the Phoenix area and found a 2 percent failure rate.
Auditors' review of contractor audit summary sheets for the same period found
that the contractor had conducted 2,735 audits of inspectors and found a 10
percent failure rate. The Department audits inspectors using a 19-item
checklist, but the contractor uses a 44-item checklist that includes reviewing
many specific inspector actions that are less significant to the emissions
testing outcome. For example, one contractor inspection criterion not included
on the department checklist is assessing whether the inspector gave the
customer the option to sit in the vehicle passenger seat during vehicle testing.
In line with contractor monitoring best practices, the Department should evaluate
whether it can conduct a smaller random sample of equipment and inspector
audits and compare the results with the contractor's audit failure rates to verify that
contractor performance is satisfactory. The Department should also consider
planning equipment and performance audits on a risk basis using the results of
the contractor's audits to determine risk areas. As appropriate, the Department
should revise its administrative rules to allow these changes.
Followup on adverse audit results is limited—By reducing the number and
frequency of equipment and inspector audits, the Department should use its
resources to address areas of weakness in its monitoring processes. Specifically,
the Department does not provide followup on equipment and inspector audits. The
Department does not track the status of equipment failing its audits, nor does it
adequately ensure that the contractor takes corrective actions when inspectors fail
audits. Followup on equipment audit failures is critical because, unlike the
contractor's periodic system checks, both the Department's and the contractor's
audit failures will not trigger a system lockout to prevent equipment from being
used until it is properly calibrated. The need for equipment followup is
compounded because the contractor does not consistently provide the
Department with timely evidence that it has repaired equipment. The Department
reported that it is developing a formal follow-up system for failed equipment.
Followup on inspectors who fail audits is also critical because the Department
relies on the contractor to suspend and retrain, or terminate, noncompliant
inspectors, instead of the Department suspending or revoking the inspector's
license. According to department officials, the Department relies on the contractor
to discipline the emissions inspectors who fail audits because the contractor can
take immediate action, whereas the Department's formal license discipline
process would take too long and would allow deficient inspectors to continue
inspecting vehicles. However, the Department does not have a formal follow-up
process to ensure that the contractor takes appropriate and timely corrective
Office of the Auditor General
page 17
The Department has not
established and
conducted adequate
audit followup.
action when inspectors fail audits. The Department's lack of followup could allow
an inspector who failed an audit (for example, by passing a vehicle that should
have failed an inspection) to continue to conduct emissions tests.
According to the Department, its administrative rules are ambiguous regarding
suspending and revoking an emissions inspector's license when he/she fails an
audit. One rule appears to require mandatory license suspension, but another rule
appears to give the Department some discretion. The rules do not define what an
inspector must successfully do to demonstrate that he/she has the ability to
conduct a proper emissions inspection. Further, the Department does not have
any written criteria regarding which department or contractor audit checklist items
or combination of items, if failed, results in an inspector failing to demonstrate the
ability to conduct a proper emissions inspection. The Department should confer
with its Assistant Attorney General regarding the need to modify its administrative
rules and/or its and the contractor's audit checklists to establish criteria for
determining when an inspector fails to demonstrate the ability to properly conduct
an emissions inspection, and should have his/her license suspended or revoked.
Department should conduct required federal monitoring and verify
contractor compliance with contract provisions that Department
has not monitored—In addition to improving audit follow-up, the Department
should adjust its contract monitoring activities to include contract provisions and
federal requirements that it has not previously monitored. For example, the
Department should perform activities such as:
􀁺􇩐 Periodically evaluating contractor quality assurance and quality control
procedures—The Department has not conducted these EPA-required
reviews, which are intended to evaluate whether the contractor's procedures
would prevent, discover, and correct fraud, waste, and abuse. Such an
evaluation would determine if the contractor is following procedures and that
they are adequate. Further, it would allow the Department to identify any
problems that may be impeding program performance. The Department
could either conduct specific or comprehensive quality control reviews. An
example of a specific review would be to evaluate contractor controls over
confirming the identity of vehicles being emissions tested, whereas a
comprehensive review would evaluate the contractor's quality control system
as a whole to identify vulnerabilities and then testing controls to ensure that the
contractor uses them and they are working as intended.
􀁺􇩖 Verifying the contractor's compliance with its surveillance schedule and audit
plan—Contract provisions require the contractor to submit a detailed
surveillance and audit plan and schedule, but the Department has not verified
that the contractor has adhered to its plan and schedule. Verification would
provide the Department with continued assurance that the contractor is
maintaining its levels of equipment and inspector performance audits.
State of Arizona
page 18
The Department lacks
written criteria to assess
when an inspector has
failed to demonstrate
the ability to conduct a
proper emissions
inspection.
Office of the Auditor General
page 19
􀁺􇩒 Reviewing and ensuring that the contractor conducts other internal audits
required by the contract—The Department has not verified that the contractor
has conducted various internal audits. The contract requires the contractor to
conduct audits of management controls; program performance; contract
conformance auditing; effectiveness of equipment maintenance and quality
control activities; employee training, safety and security measures; employee
integrity; and contractor adherence to laws, rules, policies and procedures,
and applicable guidance. By reviewing such internal audits, the Department
can identify risk areas in the contractor's operation and redirect its monitoring
activities to address those areas.
Contract monitoring plan needed—As part of adjusting its monitoring
activities and focusing on important monitoring elements, the Department should
develop and implement an annual contract monitoring plan to help ensure more
effective, efficient, and comprehensive coverage. Monitoring plans are important
to identify what needs to be done and the resources to do it, but as of August 2007,
the Department had no written monitoring plan. The Department should use a
contract monitoring plan to identify those activities needed to adequately monitor
the contractor's compliance with the many contractual and federal requirements,
document how its monitoring activities will be carried out, and identify the required
resources. A plan would also be helpful in ensuring that the level of monitoring is
commensurate with the importance of the activity. Further, the Department should
review the contract monitoring plan annually and make adjustments based on the
contractor's success in meeting contract requirements and on the Department's
risk assessments of monitored areas. To assist the Department in developing a
plan, the Department may want to review the Texas and Washington contract
monitoring guides.1 These guides provide details that should enable the
Department to develop a comprehensive monitoring plan and identify specific
monitoring practices, such as corrective action follow-up procedures and sample
contract monitoring documentation, to effectively track monitoring activities.
Finally, in developing the contract monitoring plan, the Department should assess
whether its employees need additional training in specific contract monitoring
activities and provide such training. By decreasing its focus on equipment and
inspector performance audits, and increasing its focus on risk-based auditing and
other monitoring activities, department employees may need training in new or
different skills.
A contract monitoring
plan is needed to
prioritize and focus the
Department's monitoring
activities.
1 Texas Building and Procurement Commission. Contract Management Guide. Mar. 1, 2007. Texas Building and
Procurement Commission. Aug. 21, 2007. Washington State Office of Financial Management contract monitoring
guidance found at http://www.ofm.wa.gov/contracts/resources/managing_monitoring.pdf.
Recommendations:
1. In line with contractor monitoring best practices, the Department should
evaluate whether it can conduct a smaller random sample of equipment and
inspector audits and compare the results with the contractor's audit failure rates
to verify that the contractor’s performance is satisfactory. The Department
should also consider planning equipment and performance audits on a risk
basis using the results of the contractor's audits to determine risk areas.
2. As appropriate, the Department should amend its administrative rules to reduce
the number of equipment and inspector audits it is required to conduct.
3. The Department should develop and implement a follow-up process for
equipment and inspector audits that ensures that:
a. The contractor appropriately repairs equipment failing audits before
returning it to service, and
b. Inspectors who fail department or contractor audits receive appropriate and
timely corrective action, which may include the contractor retraining,
suspending, or terminating noncompliant inspectors and/or the
Department suspending or revoking licenses.
4. The Department should confer with its Assistant Attorney General regarding the
need to modify its administrative rules and/or its and the contractor's audit
checklists and make appropriate changes in rules and/or checklists to establish
criteria for determining when an inspector fails to demonstrate the ability to
properly conduct an emissions inspection, and should have his/her license
suspended or revoked.
5. The Department should expand its contractor monitoring activities to provide
more comprehensive coverage and include important monitoring activities not
previously provided, such as:
a. Periodically evaluating contractor quality assurance and quality control
procedures;
b. Verifying the contractor's compliance with its surveillance schedule and
audit plan; and
c. Reviewing and ensuring that the contractor conducts other internal audits
required by the contract.
State of Arizona
page 20
6. The Department should develop and implement an annual contract monitoring
plan to help ensure more effective, efficient, and comprehensive coverage of its
monitoring activities.
7. In developing the contract monitoring plan, the Department should assess
whether its employees need additional training in specific contract monitoring
activities and provide any needed training.
Office of the Auditor General
page 21
State of Arizona
page 22
Office of the Auditor General
APPENDIX A
page a-i
Test Type Description
Idle inspection* Used annually for 1967 to 1980 light-duty gas engines and 1967 to current heavy-duty gas
engines in Area A and 1967 and newer light-duty and heavy-duty gas vehicles in Area B. The
vehicle is tested at idle. This inspection measures emission while a vehicle is idled within
100 rpm (revolutions per minute) of the manufacturer's specified idle setting once readings
have stabilized, or at the end of 90 seconds.
Loaded cruise* Used annually for 1967 to 1980 light-duty gas engines and 1967 and newer heavy-duty gas
engines in Area A and 1981 to 1995 light-duty gas and 1981 and newer heavy-duty gas
vehicles in Area B. The vehicle is operated on a dynamometer, an apparatus for measuring
mechanical power, at various loads and speeds of approximately 22 to 40 miles per hour
(depending on vehicle class) while the tailpipe emissions are measured until the readings for
hydrocarbons (HC) and carbon monoxide (CO) have stabilized, or at the end of 90 seconds.
On-Board Diagnostic
(OBD) II*
Used biennially in Area A and annually in Area B on 1996 and newer light-duty vehicles only,
engine operating data is accessed by connecting directly to a computer in the vehicle that
continuously monitors engine emission control systems operation. The on-board diagnostic
test can identify problems before they lead to engine damage and emissions system failure.
Transient load
(IM147)*
An enhanced test used biennially for most 1981 through 1995 gasoline-powered vehicles in
Area A. In this test, the vehicle is driven on a dynamometer at varying speeds to simulate
urban driving. The exhaust is continuously measured for 147 seconds and hydrocarbons
(HC), carbon monoxide (CO), carbon dioxide (CO2), and nitrous oxides (NOx) emissions are
shown in grams per mile.
Diesel* Diesel vehicles are tested annually in Area A and Area B for opacity (smoke density). The test
uses an opacity meter, which is an instrument that measures the percentage of opacity of the
exhaust. Light-duty diesel vehicles in Area A and all Area B diesel vehicles are tested under
load on a dynamometer. In Area A, heavy-duty diesel vehicles are tested using a procedure
called "snap acceleration" or "snap idle," which consists of measuring the opacity of three,
wide-open throttle, no-load accelerations from idle to maximum governed engine speed with
the transmissions in neutral. The average opacity for the three accelerations cannot exceed
the standard for the engine model year.
Functional Gas Cap* Visual and functional inspection conducted for vehicles both in Area A and Area B in
conjunction with other tests and ensures that gas fumes will not escape from the gas tank.
*Vehicles may be required to take more than one type of test.
Source: Auditor General staff analysis of Arizona Administrative Code, Arizona Revised Statutes, the department Web site, and interviews with department management.
State of Arizona
page a-ii
Office of the Auditor General
AGENCY RESPONSE
State of Arizona
page a-iv
06-01 Governor’s Regulatory Review
Council
06-02 Arizona Health Care Cost
Containment System—
Healthcare Group Program
06-03 Pinal County Transportation
Excise Tax
06-04 Arizona Department of
Education—Accountability
Programs
06-05 Arizona Department of
Transportation—Aspects of
Construction Management
06-06 Arizona Department of
Education—Administration and
Allocation of Funds
06-07 Arizona Department of
Education—Information
Management
06-08 Arizona Supreme Court,
Administrative Office of the
Courts—Information
Technology and FARE Program
06-09 Department of Health
Services—Behavioral Health
Services for Adults with Serious
Mental Illness in Maricopa
County
07-01 Arizona Board of Fingerprinting
07-02 Arizona Department of Racing
and Arizona Racing
Commission
07-03 Arizona Department of
Transportation—Highway
Maintenance
07-04 Arizona Department of
Transportation—Sunset Factors
07-05 Arizona Structural Pest Control
Commission
07-06 Arizona School Facilities Board
07-07 Board of Homeopathic Medical
Examiners
07-08 Arizona State Land Department
07-09 Commission for Postsecondary
Education
07-10 Department of Economic
Security—Division of Child
Support Enforcement
07-11 Arizona Supreme Court,
Administrative Office of the
Courts—Juvenile Detention
Centers
Performance Audit Division reports issued within the last 24 months
Future Performance Audit Division reports
Arizona Supreme Court, Administrative Office of the Courts—Juvenile Treatment Programs