A performance audit of the Department of Health Services vehicle emissions inspection program

              STATE OF ARIZONA

OFFICE OF THE

AUDITOR GENERAL

A PERFORMANCE AUDIT

OF THE

DEPARTMENT OF HEALTH SERVICES

VEHICLE EMISSIONS INSPECTION PROGRAM

FEBRUARY 1983

A REPORT TO THE

ARIZONA STATE LEGISLATURE

REPORT 83- 1

DOUGLAS R. NORTON. CPA

AUDITOR GENERAL

STATE OF ARIZONA

OFFICE OF THE

AUDITOR GENERAL

February 4 , 1983

Members of the Arizona L e g i s l a t u r e

The Honorable Bruce Babbitt, Governor

James Sarn, M. D., M. P. H., Director

Department of Health Services

Transmitted herewith is a report of the Auditor General, A Performance

Audit of the Department of Health Services, Vehicle Emissions Inspection

Program. This report is i n response t o Senate B i l l 1220 enacted by the

Thirty- fourth L e g i s l a t u r e i n 1980.

The blue pages present a summary of the r e p o r t ; responses from the

Department of Health Services and o t h e r a f f e c t e d p a r t i e s a r e found on the

yellow pages preceding the appendices.

My s t a f f and I w i l l be pleased t o discuss or c l a r i f y i t e m s i n the report.

Respectfully submitted,

9+ a-

Doug a s R. Norton

Auditor General

Staff : William Thomson

Peter Francis

Mark Fleming

Sylvia Forte

William Wright

Richard Booth

Enclosure

1 1 1 WEST MONROE @ SUITE 600 PHOENIX. ARIZONA 85003 ( 602) 255- 4385

OFFICE OF THE AUDITOR GENEKAL

A PERFORMANCE AUDIT OF THE

DEPARTPENT OF HEALTH SERVICES

VEHICLE EMISSIONS INSPECTION PROGRmi

A KEPORT TO THE

ARIZONA STATE LEGISLATURE

REPORT s3- 1

TABLE OF CONTENTS

Page

INTKODUCTION AND BACKGROUND 1

FINDINGS

FINDING I 7

The Vehicle Emissions Inspection Program has not reduced

carbon monoxide l e v e l s i n Phoenix and Tucson.

CONCLUSION 15

KECOFPE NDATIONS 15

FINDING I1 17

The Vehicle Emissions Inspection Program boundaries

do not e f f e c t i v e l y address automotive a i r p o l l u t i o n

problems i n Pima County.

CONCLUSION

FINDING I11

The Bureau of Vehicular Emissions Inspection can save

$ 121,700 i n S t a t e and Federal funds annually by eliminating

a i r l f u e l waivers.

CONCLUSION

RECOMPENDAT IONS

Page

FINDING I V 3 1

Although the Bureau of Vehicular Emissions Inspection and

the c o n t r a c t o r have increased the frequency of equipment

checks, a d d i t i o n a l c o n t r o l s a r e needed t o a s s u r e a c c u r a t e

and r e l i a b l e emissions t e s t i n g .

CONCLUSION

RECOMMENDATIONS

OTHER PERTINENT INFORMATION

GLOSSARY

WRITTEN RESPONSES TO THE AUDITOR GENERAL'S REPOKT

APPENDIX I

Time S e r i e s Analysis of the Impact of the Vehicle Emissions

Inspection Program on Ambient A i r Quality i n Phoenix and

Tucson

APPENDIX I1

Metropolitan Arizona Vehicle Emissions Inspection Survey

LIST OF TABLES

Page

TABLE 1 - Vehicle Emissions I n s p e c t i o n Program Budget f o r

F i s c a l Year 1982- 83

TABLE 2 - CarbonNonoxideEmissionsTest S e r i e s R e s u l t s

TABLE 3 - AirIFuel Waivers P r o c e s s i n g C o s t , F i s c a l Year

1982- 83

TABLE 4 - P r o j e c t e d Annual Savings by E l i m i n a t i n g Air/ Fuel

Waivers

TABLE 5 - BVEI and HTS S t a t e S t a t i o n Audits, F i s c a l Year

1981- 82

TABLE 6 - Timeliness of Q u a l i t y Assurance I n s p e c t i o n s f o r

Sample of F l e e t and R e g i s t e r e d Analyzers, F i s c a l

Year 1981- 82

TABLE 7 - B i l l i n g s and Payments f o r Emissions I n s p e c t i o n s ,

January 1981 through September 1982

LIST OF FIGURES

F i g u r e 1 - Commuting P a t t e r n s i n Arizona Carbon Monoxide

Nonattainment Areas

SUMMARY

The Office of the Auditor General has completed a performance audit of the

Department of Health Services, Vehicle Emissions Inspection Program. This

audit was conducted in response to Senate B i l l 1220 enacted by the

Thirty- fourth Legislature in 1980.

The Arizona Vehicle Emissions Inspection ( VEI) Program requires that

c e r t a i n motor vehicles pass an annual inspection to ensure that t h e i r

exhaust emissions meet standards established by the Department of Health

Services ( DHS). The program goal i s to protect public health from the

e f f e c t s of motor vehicle pollution. Although the Arizona Legislature

o r i g i n a l l y i n i t i a t e d the VEI Program in 1974 i n response to a i r quality

problems i n the Phoenix and Tucson areas, emissions inspection i s now a

central component of the S t a t e ' s plan for meeting requirements of the

Federal Clean A i r Act.

We found no evidence that the VEI Program has been effective. Results of

a time s e r i e s analysis covering the eight- year period 1974 through 1981

indicated that the emissions inspect ion program has not reduced carbon

monoxide levels in Phoenix and Tucson. Neither the implementation of the

mandatory program in 1977 nor dramatic increases i n t e s t f a i l u r e s in 1979

had an impact on carbon monoxide concentrations i n e i t h e r c i t y ( see page

Several problems and factors may, e i t h e r individually or together, explain

why the VEI Program has not been effective. The Program may not have

s i g n i f i c a n t l y changed vehicle maintenance behavior. Further, a

s u b s t a n t i a l proportion of motorists admit to circumventing program

requirements by readjusting t h e i r engines a f t e r the emissions t e s t s .

Exclusion of older and out- of- state vehicles from t e s t requirements and

problems resulting from the v a r i a b i l i t y of automobile emissions may also

contribute to the Program1 s ineffectiveness. However, we do not know for

c e r t a i n why it i s ineffective, and thus the e f f e c t of policy or program

changes addressing any or a l l of these factors i s also unknown ( see page

11).

Federal law requires ~ r i z o n at o operate an emissions i n s p e c t i o n program.

Unless a d d i t i o n a l research i d e n t i f i e s f e a s i b l e a l t e r n a t i v e s , i f any, f o r

an e f f e c t i v e program, we recommend t h a t DHS operate t h e VEI Program only

as required t o avoid impositions of Federal sanctions. Any program

changes should be rigorously and independently evaluated before being

implemented. We f u r t h e r recommend t h a t DHS i d e n t i f y a l t e r n a t e s t r a t e g i e s

f o r reducing automotive a i r p o l l u t i o n i n the absence of an e f f e c t i v e

emissions inspection program. In a d d i t i o n , the ~ e g i s l a t u r e should

consider p e t i t i o n i n g the U. S. Congress t o review t h e appropriateness of

the emissions t e s t i n g requirement i n the Clean Air Act.

The VEI Program boundaries i n Pima County a r e n o t a p p r o p r i a t e . Although

the boundaries a r e l a r g e r than necessary t o address carbon monoxide

problems i n Tucson, the Program boundaries exclude areas which may be

required f o r e f f e c t i v e ozone c o n t r o l . We a l s o found, however, e f f e c t i v e

c o n t r o l of ozone through the VEI Program may not be possible because

1) the Program has not been e f f e c t i v e i n reducing carbon monoxide l e v e l s ;

2) the Program does not address n i t r o u s oxides, one of the two precursors

of ozone; and 3) few vehicles f a i l emissions t e s t s f o r hydrocarbons, the

other component of ozone.

The Pima Association of Governments ( PAG) and t h e Department of Health

Services should analyze the nature and sources of automotive p o l l u t i o n in

the Tucson area. PAG should a d j u s t the program boundaries i n Pima County

t o include a l l areas which c o n t r i b u t e t o Tucson's current and p o t e n t i a l

automotive a i r p o l l u t i o n and exclude those areas which do not ( s e e page

17).

The a i r / f u e l waiver provision of Arizona's Vehicle missions Inspection

Program is not c o s t e f f e c t i v e . Although s t a t u t e s r e q u i r e the Department

of Health Services, Bureau of Vehicular Emissions Inspection ( BVEI) t o

i s s u e a i r / f u e l waivers, l e s s than 2 percent of t h e motorists use t h i s

service. As a r e s u l t , t h e c o s t per waiver is approximately $ 61. The

Bureau could save an estimated $ 121,700 i n S t a t e and Federal funds

annually by eliminating the a i r / f u e l waiver program. We recommend t h a t

the L e g i s l a t u r e consider amending A. R. S. $ 36- 1772. E. t o allow such an

a c t i o n ( s e e page 23).

Neither the Bureau nor Hamilton Test Systems ( HTS) conducted the

appropriate number of inspection s t a t i o n f i e l d a u d i t s i n f i s c a l year

1981- 82. BVEI conducted only 56 percent of t h e a u d i t s required by i t s

r e g u l a t i o n s during the year. HTS conducted only 53 percent of t h e a u d i t s

required by company policy. Although BVEI and HTS have increased the

frequency of inspection s t a t i o n a u d i t s , t h e Bureau needs t o strengthen

c o n t r o l by 1) i n s t i t u t i n g a formal management reporting system t o inform

the Bureau chief t h a t necessary a u d i t s a r e conducted and 2) requiring HTS

by c o n t r a c t t o conduct two a u d i t s per month on each inspection s t a t i o n

lane ( s e e page 3 2 ) .

I n a d d i t i o n , t h e Bureau needs t o ensure t h a t i t s i n s p e c t o r s conduct timely

q u a l i t y assurance i n s p e c t i o n s of analyzers used by f l e e t inspectors and

p r i v a t e garages a s required by Bureau r u l e s . R e s u l t s of these inspections

should be reported t o the Bureau chief a t l e a s t monthly ( s e e page 3 4 ) .

iii

INTRODUCTION AND BACKGROUND

The Office of the Auditor General has completed a performance a u d i t of the

Department of Health Services, Vehicle Emissions Inspection Program. This

audit was conducted i n response t o Senate B i l l 1220 enacted by the

Thirty- fourth L e g i s l a t u r e i n 1980.

Arizona Revised S t a t u t e s $ 36- 1772, which e s t a b l i s h e s the Vehicle Emissions

Inspection ( VEI) Program, requires t h a t c e r t a i n motor vehicles pass an

annual inspection t o ensure t h a t t h e i r exhaust emissions meet standards

e s t a b l i s h e d by the Department of Health Services ( DHS). The program goal

i s t o p r o t e c t public h e a l t h from the e f f e c t s of motor v e h i c l e p o l l u t i o n .

Although the Arizona L e g i s l a t u r e o r i g i n a l l y enacted the program i n 1974 i n

response t o a i r q u a l i t y problems i n the Phoenix and Tucson areas,

emissions inspection is now a c e n t r a l component of t h e S t a t e ' s plan f o r

meeting requirements of the Federal Clean Air Act.

Program Operations

The Bureau of Vehicular Emissions Inspection ( BVEI) within DHS administers

the inspection program. To achieve the program goal of p r o t e c t i n g p u b l i c

h e a l t h , the Bureau has e s t a b l i s h e d t h r e e o b j e c t i v e s . These are:

- To inspect a l l vehicles i n the nonattainment areas of Maricopa

and Pima Counties as required by law,

- To adopt emissions standards which w i l l i d e n t i f y vehicles which

a r e gross and high p o l l u t e r s , and

- To require r e p a i r s t o vehicles which a r e gross and high p o l l u t e r s

which w i l l reduce carbon monoxide and hydrocarbon emissions t o

t h e l e v e l of vehicles i n i t i a l l y passing the inspection.

The VEI Program t e s t s vehicles t o ensure t h a t carbon monoxide and

hydrocarbons i n t h e i r exhaust emissions meet standards e s t a b l i s h e d by

DHS. The inspect ion requirement a p p l i e s generally t o gasoline- powered

vehicles which a r e l e s s than 14 y e a r s o l d and located within designated

portions of Pima and Maricopa Counties. The Program includes those

p a r t s of the two counties which do not meet the carbon monoxide standards

of the Federal Clean A i r Act. Plotorists whose vehicles f a i l t o meet these

standards must r e p a i r t h e i r vehicles and submit t o a r e t e s t . Motorists

must e i t h e r pass the emissions t e s t o r seek one of the s e v e r a l waivers

provided under the law or BVEI regulations.

The Bureau does not conduct the a c t u a l inspections. A p r i v a t e company,

Hamilton Test Systems ( HTS), under contract with DHS o p e r a t e s n i n e

permanent f a c i l i t i e s and one mobile s t a t i o n where its employees conduct

emissions t e s t s f o r the S t a t e . Hamilton conducts about 1.4 m i l l i o n t e s t s

per year. Payments t o the c o n t r a c t o r during 1981 were approximately $ 5.8

million.

Importance of the Vehicle

Emissions Inspection Program

The Vehicle Emissions Inspection Program i s the p r i n c i p a l s t r a t e g y f o r

reducing carbon monoxide p o l l u t i o n i n Phoenix and Tucson. The Federal

Clean A i r Act requires Arizona t o implement an emissions inspection

program i n these two c i t i e s because n e i t h e r met the Federal standards f o r

carbon monoxide by December 1982. F a i l u r e t o implement such a program and

develop a plan f o r meeting the standards by December 31, 1987, could cause

the two c i t i e s t o lose more than $ 100 m i l l i o n i n Federal a i d . However,

Arizona has an inspection program i n e f f e c t and Phoenix and Tucson a r e

making a c c e p t a b l e p r o g r e s s i n developing the required plans. Thus,

n e i t h e r c i t y appears t o be i n danger of losing Federal funds i n the

f o r s e e a b l e f u t u r e .

Current plans f o r meeting the 1987 carbon monoxide standards r e l y on the

VEI Program a s the primary strategy.* Although other s t r a t e g i e s w i l l a l s o

be implemented, n e i t h e r t h e Environmental Protection Agency ( EPA) nor

Arizona's regional planning agencies consider these s t r a t e g i e s s u f f i c i e n t

* Although carbon monoxide l e v e l s i n Pima County a r e s u b s t a n t i a l l y l e s s

than i n Maricopa County and the F e d e r a l s t a n d a r d s may be e a s i e r to

a t t a i n , t h e primary s t r a t e g y w i l l most l i k e l y be the VEI Program.

Pima County has not yet completed work on i t s d r a f t plan.

t o produce the carbon monoxide reductions needed t o meet the Federal

standards. For example, the d r a f t plan f o r Maricopa County p r o j e c t s

needed carbon monoxide reductions of 449 tons per day t o a t t a i n the 1987

goal. The p l a n e s t i m a t e s t h a t t h e e x i s t i n g VEI Program combined with

Federal emission requirements f o r new motor vehicles w i l l reduce carbon

monoxide by 349 tons per day ( 78 percent). The Maricopa plan p r o j e c t s

a d d i t i o n a l reductions of 71 tons per day ( 16 percent) i f the VEI Program

a l s o includes a l l vehicles produced since 1969.

In addition t o the VEI Program, the Maricopa plan f o r a t t a i n i n g Federal

standards a l s o includes s t r a t e g i e s t o reduce v e h i c l e use and t o improve

t r a f f i c flow. The major e f f o r t s i n t h i s d i r e c t i o n a r e t o maintain t r a n s i t

r i d e r s h i p a t current: l e v e l s and implement s p e c i f i c t r a n s p o r t a t i o n c o n t r o l

measures ( r e v e r s i b l e lanes, f o r example). These e f f o r t s , however, account

f o r reductions of only 27 tons per day, 6 percent of the needed

reduction. Other s t r a t e g i e s , such as increasing t r a n s i t r i d e r s h i p and

v e h i c l e occupancy would provide 38 tons per day of the needed reduction.

Organization and Budget

BVEI is one of four bureaus i n the DHS Environmental Health Services

Division. The Bureau has 23 FTE p o s i t i o n s f o r the current f i s c a l year and

employed 22 persons i n Phoenix and Tucson a s of August 1982. Seventeen

employees work i n four s e c t i o n s located i n Phoenix. These sections a r e :

- Administration-- processes requests f o r exemptions, waivers and

renewals of mechanic c e r t i f i c a t e s and provides research and

c l e r i c a l support;

- Licensing and C e r t i f i c a t i o n - - t r a i n s and c e r t i f i e s mechanics t o

perform procedures required f o r a i r l f u e l waivers and f l e e t

inspect ions.

- Waiver and Testing-- operates the BVEI waiver and r e t e s t f a c i l i t y

and i n s p e c t s v e h i c l e s owned by small governments;

- Quality Assurance-- inspects S t a t e s t a t i o n s operated by HTS t o

ensure compliance with procedures, r e l i a b i l i t y of equipment and

inspects and c e r t i f i e s analyzers used by f l e e t inspectors and

p r i v a t e r e p a i r f a c i l i t i e s .

Five persons work i n the Bureau's Tucson o f f i c e and perform e s s e n t i a l l y

a l l of the functions of the Phoenix o f f i c e with the exception of

inspecting S t a t e s t a t i o n s . Personnel from the Phoenix o f f i c e inspect

S t a t e s t a t i o n s i n both c i t i e s .

As shown i n Table 1, the t o t a l budget f o r the Vehicle Emissions Inspection

Program i n f i s c a l year 1982- 83 i s approximately 66.7 m i l l i o n . The VEI

Program has s e v e r a l sources of funding. General Fund appropriations

supplemented by Federal g r a n t s support most BVEI personnel and

a c t i v i t i e s . I n a d d i t i o n , f e e s c o l l e c t e d f o r emissions t e s t s flow i n t o a

revolving fund. BVEI uses these funds t o pay Hamilton Test Systems f o r

the emissions t e s t s it conducts. Payments t o the c o n t r a c t o r account f o r

90 percent of the VEI Program budget. The revolving fund a l s o includes

fees charged f o r waivers and exemptions and supports t h e c o s t of personnel

who provide these services.

TABLE 1

VEHICLE EMISSIONS INSPECTION PROGRAM BUDGET

FOR FISCAL YEAR 1982- 83

A1 l General Revolving

Sources Fund Fund Federal

FTE s 2 3 16 5

Personal s e r v i c e s

Employee- related

expenditures

P r o f e s s i o n a l and

o u t s i d e s e r v i c e s

Travel:

In- State

Out- of- State

Other operating

expenses

Other payments

Totals

* Estimated payment t o Hamilton Test Systems for emission t e s t s .

Audit Scope

The scope of our a u d i t focused primarily on the need f o r t h e emissions

i n s p e c t i o n program, i t s impact and the p o l i c i e s and procedures which

govern its operation. The a u d i t addressed the following s p e c i f i c i s s u e s :

- The e f f e c t of the Vehicle Emissions Inspection Program on the

carbon monoxide l e v e l s i n Phoenix and Tucson;

- The appropriateness of VEI Program boundaries i n Pima County;

- The cost e f f e c t i v e n e s s of the a i r / f u e l waiver program; and

- The adequacy of q u a l i t y assurance f o r S t a t e emissions t e s t i n g

s t a t i o n s and analyzers used by p r i v a t e and f l e e t f a c i l i t i e s .

The report s e c t i o n , Other P e r t i n e n t Information, presents d e t a i l e d

i n f o r n ~ a t i o n on v e h i c l e inspect ion f e e payments and c o l l e c t ions and the

i n s p e c t i o n equipment used i n the Program and HTS t e s t and b i l l i n g

procedures.

Collection and a n a l y s i s of d a t a i n two areas r e q u i r e d u s e of o u t s i d e

p r o f e s s i o n a l a s s i s t a n c e . The Auditor General c o n t r a c t e d w i t h the Center

f o r Informative Evaluation t o conduct a time s e r i e s a n a l y s i s t o a s s e s s the

impact of t h e VEI Program on carbon monoxide l e v e l s i n Phoenix and

Tucson. The r e s u l t s of t h i s a n a l y s i s form t h e b a s i s f o r Finding I. A

second c o n t r a c t o r , t h e Behavior Research Center, surveyed 800 persons i n

Phoenix and Tucson who had experiences with the VEI Program. The survey

provided in£ ormation o n p u b l i c p e r c e p t i o n of a i r q u a l i t y , a t t i t u d e s toward

the VEI Program, and t h e i r experiences i n complying with i t s

requirements. Survey r e s u l t s supplement Auditor General s t a f f research on

program e f f e c t i v e n e s s .

The Auditor General and s t a f f express a p p r e c i a t i o n t o the Department of

Health Services and Hamilton Test Systems f o r t h e i r cooperation and

a s s i s t a n c e during the course of the a u d i t .

FINDING I

THE VEHICLE EMISSIONS INSPECTION PROGRAM HAS NOT REDUCED CARBON MONOXIDE

LEVELS I N PHOENIX AND TUCSON.

The Vehicle Emissions Inspection ( VEI) Program has not reduced carbon

monoxide ( CO) l e v e l s i n Phoenix and Tucson. Neither the implementation of

the mandatory program i n 1977 nor dramatic increases i n t e s t f a i l u r e s i n

1979 had an impact on carbon monoxide concentrations i n e i t h e r c i t y .

Thus, the VEI Program does not appear t o b e a n e f f e c t i v e s t r a t e g y f o r

meeting t h e Federal Air Quality Standards i n 1987. Although a number of

f a c t o r s may impair the Program's e f f e c t i v e n e s s , we do not know f o r c e r t a i n

why it is i n e f f e c t i v e . Thus, the e f f e c t of policy o r program changes

addressing any or a l l of these f a c t o r s is unknown.

Program Was ~ s t a b l i s h e d

t o Improve A i r q u a l i t y

The goal of the Vehicle Emissions Inspection Program is t o p r o t e c t public

h e a l t h by reducing p o t e n t i a l l y harmful automotive emissions. These

emissions include carbon monoxide, a h e a l t h hazard i n i t s e l f , and

hydrocarbons, which a r e a p r e c u r s o r o f the p o l l u t a n t ozone.* Unlike

ozone, carbon monoxide is generated almost e n t i r e l y from automotive

emissions. The Arizona L e g i s l a t u r e enacted t h e program i n 1974 t o address

a i r q u a l i t y problems i n Phoenix and Tucson. In 1977 the U. S. Congress

amended the Clean Air Act t o require emissions inspection i n areas not

meeting Federal carbon monoxide and ozone standards by December 1982. The

a c t provides f o r withholding Federal aid f o r highway c o n s t r u c t i o n and

sewage treatment p l a n t c o n s t r u c t i o n i n s t a t e s not complying with t h i s

requirement. I f Arizona d i d n o t have the VEI Program i n e f f e c t , the S t a t e

could lose up t o $ 116 m i l l i o n p e r y e a r i n Federal aid.

* Ozone is formed by sunlight a c t i n g on hydrocarbons and n i t r o u s oxides.

Previous Emissions I n s p e c t i o n S t u d i e s

Because carbon monoxide is generated l a r g e l y by motor v e h i c l e s , previous

evaluations of emissions inspection programs have focused on how the

program has a f f e c t e d carbon monoxide l e v e l s i n the atmosphere. Some

previous s t u d i e s suggest t h a t carbon monoxide l e v e l s have been reduced by

inspection programs. Department of Health Services ( DHS) assessments i n

the e a r l y years of the Program show improvement i n carbon monoxide

emissions. Federal Environmental Protection Agency ( EPA) s t u d i e s a l s o

indicated t h a t emissions t e s t i n g programs reduced carbon monoxide l e v e l s

i n Phoenix as well a s Portland, Oregon.

However, the Oregon study found a reduction i n carbon monoxide

a t t r i b u t a b l e t o emissions t e s t i n g a t only one of four monitoring s t a t i o n s

i n Portland. Furthermore, a t the s i t e where t h i s e f f e c t was found,

r e s u l t s were confounded by movements of t h e monitoring probe and by major

t r a f f i c d i s r u p t i o n s i n the area. A New Jersey study using a s i m i l a r time

s e r i e s methodology a l s o found reductions i n carbon monoxide l e v e l s . In

t h i s study, however, t h e e f f e c t s a t t r i b u t a b l e t o emissions t e s t i n g could

not be separated from the e f f e c t s of new car emission standards.

UHS evaluations of the VEI Program have reported t h a t carbon monoxide

l e v e l s were decreasing. However, DHS a n a l y s t s concluded i n two recent

s t u d i e s t h a t the VEI Program, even when combined with other carbon

monoxide c o n t r o l s t r a t e g i e s , w i l l not enable Maricopa County t o achieve

F e d e r a l s t a n d a r d s f o r carbon monoxide by 1987.

To evaluate the e f f e c t i v e n e s s of t h e Arizona Vehicle Emissions Inspection

Program i n Phoenix and Tucson, we contracted f o r an impact assessment of

the Program using time s e r i e s a n a l y s i s covering the eight- year period 1974

through 1981. k Time s e r i e s a n a l y s i s has been used t o evaluate a i r q u a l i t y

data i n other s t a t e s ( Oregon and New ~ e r s e ~ bu) t h a s n o t previously been

employed i n Arizona. In our opinion, t h i s methodology r e p r e s e n t s the most

t e c h n i c a l l y s o p h i s t i c a t e d and appropriate approach f o r assessing the

impact of the s t a t e ' s Vehicle Emissions Inspection Program.

The VEI Program Has Not Reduced Carbon

Monoxide Levels i n Phoenix and Tucson

Although the VEI Program is intended t o p r o t e c t public h e a l t h by reducing

carbon monoxide l e v e l s , r e s u l t s of the time s e r i e s a n a l y s i s i n d i c a t e t h a t

the Program h a s n o t achieved its desired e f f e c t . As a r e s u l t , Maricopa

and Pima Counties cannot r e l y on the emissions inspection program t o meet

the 1987 Federal A i r Quality Standards.

Arizona implemented the mandatory VEI Program i n January 1977. For the

f i r s t time, motorists whose vehicles f a i l e d t h e emissions t e s t were

required t o r e p a i r t h e v e h i c l e and r e t u r n f o r a r e t e s t . Beginning i n t h a t

year, the program required proof of compliance f o r v e h i c l e r e g i s t r a t i o n .

By the end of 1977, a l l light- duty vehicles had been inspected and i f

necessary, maintained t o standards. Moreover, i n January 1979, Program

standards were r a i s e d and s i g n i f i c a n t l y more c a r s f a i l e d t h e emissions

t e s t . The average number of v e h i c l e s f a i l i n g the inspection each month

increased by almost 5,900 i n Phoenix and more than 2,000 i n Tucson.

* The time s e r i e s methodology employed i n t h i s study is the

Autoregressive, I n t e g r a t e d , Moving Average ( AKIMA) models and methods

developed by Box and Jenkins. Carbon monoxide d a t a analyzed i n t h i s

study included four d i f f e r e n t measures of carbon monoxide ( 8- hour

monthly high, mean highest 8- hour average, monthly average and mean

5 p. m. readings) a t two monitoring s t a t i o n s i n Phoenix ( Central

Phoenix and North phoenix) and one measure of carbon monoxide ( mean

highest 8- hour average) a t the downtown monitoring s t a t i o n i n Tucson.

Highest 8- hour average concentrations of carbon monoxide a r e c o l l e c t e d

and reported t o the Federal Environmental Protection Agency as the

b a s i s f o r determining compliance w i t h F e d e r a l A i r Q u a l i t y Standards.

For a d e t a i l e d , t e c h n i c a l d i s c u s s i o n of study r e s u l t s , see Appendix I.

Results of the time s e r i e s a n a l y s i s revealed t h a t carbon monoxide l e v e l s

i n Phoenix and Tucson were not reduced by the implementation of the

mandatory emissions inspection program i n 1977 or by more s t r i n g e n t

standards which took e f f e c t i n January 1979. After conducting a

s u b s t a n t i a l number of analyses, the consultants concluded t h a t the Program

has not been e f f e c t i v e :

" Our findings a r e t h a t n e i t h e r the 1977 i n t e r v e n t i o n

( mandatory I/ M) nor the 1979 i n t e r v e n t i o n ( higher I / M

standards) had any s t a t i s t i c a l l y s i g n i f i c a n t impact on

CO l e v e l s . Our analyses gave the I / M program every

possible " benefit of the doubt." Yet we found not one

i o t a of evidence t o support the hypothesis t h a t the

Arizona I / M program had an impact on ambient a i r

q u a l i t y . Given the remarkable degree of

consistency-- our findings were c o n s i s t e n t across

i n t e r v e n t i o n s , s i t e s , CO i n d i c a t o r s , and models-- we have

t h e g r e a t e s t possible confidence i n our major

c ~ n c l u s i o n . ' ~

Results of the time s e r i e s study strongly suggest t h a t Maricopa and Pima

Counties cannot rely on the Vehicle Emissions Inspection Program t o meet

Federal Air Quality Standards by the end of 1987. To meet the deadline,

Maricopa Association of Governments has prepared a nonattainment area plan

which i d e n t i f i e s s t r a t e g i e s t o reduce a i r p o l l u t i o n l e v e l s i n Maricopa

County. The Vehicle Emissions Inspection Program, although not t h e s o l e

c o n t r o l s t r a t e g y , is the major element i d e n t i f i e d i n the plan.* Because

the time s e r i e s analysis shows t h a t the Vehicle Emissions Inspection

Program has not reduced carbon monoxide l e v e l s i n Phoenix and Tucson, the

Program cannot be r e l i e d on t o meet the 1987 deadline f o r a t t a i n i n g

Federal Air Quality Standards.

* Pima County has not yet d r a f t e d a nonattainment area plan f o r 1987.

The plan now under development w i l l i d e n t i f y the VEI Program as a major

s t r a t e g y f o r meeting Federal standards i n 1987.

Several Factors May Account f o r

the Program's I n e f f e c t i v e n e s s

Several problems and f a c t o r s may e i t h e r i n d i v i d u a l l y o r t o g e t h e r e x p l a i n

why the Vehicle Emissions Inspection Program has not been e f f e c t i v e . These

f a c t o r s include unchanged behavior, circumvention, automotive engine

v a r i a b i l i t y and vehicle exclusions. However, t h e e f f e c t of policy or

program changes addressing any o r a l l of these f a c t o r s i s uncertain.

Unchanged Behavior - As noted by our c o n s u l t a n t s ( s e e Appendix I ) , p a r t of

the theory behind the VEI Program is t h a t motorists cannot or w i l l not

v o l u n t a r i l y inspect and maintain t h e i r v e h i c l e s . However, most motorists

tune t h e i r vehicles and would continue t o do so i n the absence of the

Program.* Thus, the VEI Program may not a f f e c t v e h i c l e maintenance

behavior t o a s l a r g e a n extent a s was o r i g i n a l l y thought. This, i n t u r n ,

reduces the impact of the Program.

Circumvention - Post- test engine adjustments may c o n t r i b u t e t o the reduced

e f f e c t i v e n e s s of the VEI Program. Approximately one i n n i n e m o t o r i s t s ( 11

percent) circumvented VEI Program requirements during the year. In

a d d i t i o n , 20 percent of the survey respondents admitted t o circumventing

the requirements a t some time i n the p a s t . Because many persons may be

unwilling t o admit t o such behavior, the a c t u a l proportion of motorists

circumventing the VEI Program may be higher. The r e s u l t is t h a t many

vehicles which pass the emissions t e s t subsequently operate a t a l e v e l

which does not comply with program requirements.

Motorists circumvent the program by adjusting t h e i r engines t o pass the

emissions test: and r e a d j u s t i n g t h e i r vehicles a f t e r completing the t e s t .

These i n d i v i d u a l s t a k e such a c t i o n because they f e e l t h a t t h e i r vehicles do

not operate properly when tuned f o r low emissions a s i l l u s t r a t e d by t h e

following examples:

* 93 percent of survey respondents s t a t e d t h a t i f an emissions program

were not required, they would tune or a d j u s t t h e i r c a r s about a s o f t e n

a s they do now. For a f u l l report on t h e public survey r e s u l t s , see

Appendix 11.

- One survey respondent had a mechanic a d j u s t the carburetor so t h a t

the vehicle would pass the emissions t e s t . The same mechanic l a t e r

readjusted the carburetor so t h e t h e c a r would " run right." The

motorist noted t h a t " A l o t of places a r e t e l l i n g people up f r o n t t h a t

t h e y ' l l do t h a t now."

- A respondent noted t h a t a mechanic had t o r e a d j u s t the carburetor

a f t e r it had passed the emissions t e s t because the vehicle kept

s t a l l i n g .

- Another person reported r e s e t t i n g the i d l e a f t e r the emissions t e s t .

P r i o r t o the t e s t , a mechanic had tuned the vehicle which reduced the

f u e l economy from 31 mpg t o 23 mpg. The i n d i v i d u a l ' s adjustments

increased mileage t o 27 mpg.

Since a t l e a s t 11 percent of v e h i c l e o p e r a t o r s circumvent the program by

making pos t - t e s t adjustments, t h e i r impact on program e f f e c t i v e n e s s is

l i k e l y t o be s i g n i f i c a n t .

Engine V a r i a b i l i t y - The v a r i a b i l i t y of automotive engines may a l s o l i m i t

program e f f e c t i v e n e s s . missions from an automobile can vary during the 12

months between inspect ions. As a r e s u l t , the Vehicle Emissions Inspection

Program may not produce the a n t i c i p a t e d reduction i n carbon monoxide

emissions from a given v e h i c l e .

The Motor Vehicle Manufacturers Association ( MvMA) c i t e s t h r e e major

problems associated with automobiles which a f f e c t t h e r e l i a b i l i t y of

emissions t e s t i n g :

1. Vehicle emissions w i l l vary over a wide range of weather conditions.

Temperature, humidity and barometric p r e s s u r e c a n cause emissions t o

vary s u b s t a n t i a l l y . For example, a 10 degree temperature change caused

carbon monoxide emissions t o increase by 50 percent i n one t e s t .

2. Vehicle usage immediately p r i o r t o the emissions t e s t can a f f e c t t e s t

r e s u l t s . Measurements taken a f t e r a long drive a r e q u i t e d i f f e r e n t from

those taken a f t e r short d r i v e s as might occur during in- town

use. I d l i n g while waiting i n l i n e f o r a t e s t can a l s o be a

f a c t o r .

3. The v a r i e t y of f u e l s used i n d i f f e r e n t vehicles have d i f f e r e n t

c h a r a c t e r s t i c s t h a t a f f e c t measured emissions l e v e l s .

In a d d i t ion, the v a r i a t i o n r e s u l t i n g from t h e d i f f e r e n t v a r i a b l e s is

n o t c o n s i s t e n t from one model t o another o r from t e s t t o t e s t . AS a

r e s u l t , MVPlA suggests t h a t emissions t e s t i n g focus on detecting only

" gross" p o l l u t e r s .

The problem with the v a r i a b i l i t y of v e h i c l e emissions was

demonstrated by automobiles submitted f o r emissions t e s t i n g by

Auditor General s t a f f . Two vehicles provided and tuned by Pioneer

Ford were submitted f o r f i v e s e r i e s of t e s t s . Each s e r i e s consisted

of emissions adjustments and a p r e t e s t reading by an experienced

mechanic on a State- inspected analyzer, two o r t h r e e emissions t e s t s

a t S t a t e s t a t i o n s i n Maricopa County and a p o s t - t e s t reading a t

Pioneer Ford. In t h r e e of t h e s e r i e s , t h e v e h i c l e e i t h e r passed or

f a i l e d a l l t e s t s a s expected. In two of the s e r i e s , one of the t h r e e

S t a t e s t a t i o n t e s t s showed unexpected r e s u l t s . A vehicle s e t t o f a i l

passed one of t h r e e inspections and a vehicle s e t t o pass f a i l e d one

of t h r e e inspections. Table 2 presents data f o r the two s e r i e s which

produced unexpected r e s u l t s .

TABLE 2

CARBON PIONOXIDE EMISSIONS TEST SERIES RESULTS

1980 Ford Fairmont 1981 Ford Fairmont

Carbon Monoxide Standard 2.20% o r Less 1.50% or Less

I) - Time Reading Result - Time Reading Result

P r e t e s t 5: 00 pm* 6.80% F a i l 9: 05 am 0.01% Pass

S t a t i o n t e s t s 9: 20am 8.41 F a i l 10: 05 am 0.02 Pass

l0: 07 am 6.34 F a i l 10: 39 am 0.02 Pass

10: 41 am 1.54 Pass 10: 58 am 3.03 F a i l

Post- test 12: 45 pm 6.20 F a i l 11: 40 am 0.01 Pass

* Tested on previous evening

Vehicle Exclusions - The e f f e c t of excluding v e h i c l e s from t h e emissions

inspection program over t h e p a s t f i v e years is uncertain. A i r q u a l i t y

planners p r o j e c t t h a t excluding older vehicles w i l l increase i n

s i g n i f i c a n c e i n f u t u r e years. In a d d i t i o n , out- of- state v e h i c l e s d r i v e n

by winter r e s i d e n t s may c o n t r i b u t e t o the lack of program e f f e c t i v e n e s s .

Although o l d e r vehicles have not been the source of a s i g n i f i c a n t

proportion of carbon monoxide emissions i n r e c e n t y e a r s , planners expect

t h e i r s i g n i f i c a n c e t o increase. The VEI Program excludes vehicles more

than 13 years old. Because emissions from older vehicles a r e l i k e l y t o be

more s i g n i f i c a n t i n f u t u r e years, t h e Maricopa Association of Governments

i s recommending t h a t t h e VEI Program include a l l v e h i c l e s manufactured

a f t e r 1969.

Out- of- state v e h i c l e s driven by v i s i t o r s f o r extended periods during the

winter months may a l s o c o n t r i b u t e t o carbon monoxide problems i n Phoenix

and Tucson. Their impact is unknown and no information on t h e i r numbers

or emissions i s a v a i l a b l e . However, both t r a f f i c volume and the number of

carbon monoxide v i o l a t i o n s increase i n Maricopa County during the winter

months when these v i s i t o r s a r e i n residence.

Improving VEI Program

Effectiveness

The p o t e n t i a l f o r improving VEI Program e f f e c t i v e n e s s by addressing the

above problems i s not c e r t a i n . Bringing o l d e r vehicles i n t o the program

can be r e a d i l y achieved although t h e impact is unclear. Other problems may

be impossible t o address e f f e c t i v e l y .

MAG and DHS have taken a c t i o n t o include older v e h i c l e s . Expanding the

program t o include o l d e r vehicles would r e s u l t i n a projected drop i n

emissions. However, the r e l a t i v e impact of t h i s a c t i o n depends on

continuation of Federal new c a r emissions standards which a r e under review

by the U. S. Congress. No data i n d i c a t e the extent t o which these changes

w i l l a f f e c t carbon monoxide l e v e l s .

Problems such a s the v a r i a b i l i t y of automotive emissions, circumvention

and the impact of winter v i s i t o r s w i l l be more d i f f i c u l t , i f not

impossible, t o address. The v a r i a b i l i t y of v e h i c l e emissions means t h a t

even vehicles passing the inspection may continue t o c o n t r i b u t e t o a i r

p o l l u t i o n . E f f e c t i v e l y reducing circumvention would require s t r i c t e r

c o n t r o l of motorist and mechanic behavior. Violations may be v i r t u a l l y

impossible t o d e t e c t without use of unacceptable, i n t r u s i v e methods.

F i n a l l y , c o n t r o l l i n g emissions from out- of- state vehicles would be

impractical.

CONCLUSION

The Arizona Vehicle Emissions Inspection Program h a s n o t been e f f e c t i v e i n

reducing carbon monoxide l e v e l s i n Phoenix and Tucson and may not be a

r e l i a b l e s t r a t e g y f o r meeting the 1987 Federal a i r q u a l i t y standards.

Although several problems and f a c t o r s associated with the Program may

account f o r i t s i n e f f e c t i v e n e s s , we do not know f o r c e r t a i n why it is

i n e f f e c t i v e . Thus, the e f f e c t of any policy o r program changes i n these

areas is a l s o unknown. As a r e s u l t , Arizona needs t o develop a l t e r n a t i v e s

t o the VEI Program f o r reducing automotive p o l l u t a n t s .

1. Unless a d d i t i o n a l research i d e n t i f i e s f e a s i b l e a l t e r n a t i v e s , i f any,

f o r an e f f e c t i v e VEI Program, DHS should operate the Program only as

necessary t o avoid imposition o f F e d e r a l s a n c t i o n s . Any program

changes should be rigorously and independently evaluated before being

implemented.

2. In the absence of an e f f e c t i v e VEI Program, DHS and Arizona's regional

a i r q u a l i t y planning agencies should develop and evaluate a l t e r n a t e

s t r a t e g i e s f o r reducing motor v e h i c l e p o l l u t i o n .

3. The L e g i s l a t u r e should consider p e t i t i o n i n g t h e U. S. Congress t o

review the appropriateness of the Federal requirements f o r emissions

t e s t i n g i n l i g h t of the findings i n t h i s report and the a l t e r n a t e

s t r a t e g i e s f o r reducing a i r p o l l u t i o n developed by DHS.

FINDING I1

THE VEHICLE EMISSIONS INSPECTION PROGRAlV BOUNDARIES DO NOT EFFECTIVELY

ADDRESS AUTOMOTIVE AIR POLLUTION PROBLEMS I N PIYA COUNTY.

The Vehicle Emissions Inspect ion ( VEI) Program boundaries a r e not

appropriate i n the Tucson area. The current boundaries a r e l a r g e r than

necessary t o c o n t r o l t h e e x i s t i n g carbon monoxide problem. However, the

program boundaries do n o t i n c l u d e a l l areas which may be required t o

control f u t u r e ozone p o l l u t i o n .

Establishment of Nonattainment Area Boundaries

The VEI Program covers the Pima County nonattainment area ( NU) f o r carbon

monoxide ( ~ 0 ) . The Clean A i r Act r e q u i r e s t h a t each s t a t e designate such

areas wherever a i r q u a l i t y does not meet Federal standards. NAAs must

encompass the areas where v i o l a t i o n s occur and be s u f f i c i e n t l y l a r g e t o

adequately address the source of the problem. Tucson exceeds a i r q u a l i t y

standards f o r carbon monoxide and, as a r e s u l t , comes under these

requirements.

The Pima Association of Governments ( PAG), working i n conjunction w i t h t h e

Department of Health Services ( DHS), designated the NAA i n 1978 based on

the concept of an a i r shed. The high mountain ridges surrounding Tucson

d e f i n e a n a i r shed which is t h e b a s i s f o r the Pima County NAA. An a i r

shed is an area i n which a i r p o l l u t a n t s mix but cannot escape i n the

absence of a major weather f r o n t . Thus, the a i r shed is intended t o

provide the b a s i s f o r area- wide a i r q u a l i t y management.

The one exception t o t h e a i r shed approach is t h e Town of Marana which

requested t o be excluded from the NAA on the grounds t h a t t h e town had no

a i r q u a l i t y v i o l a t i o n s . Citing r e p o r t s t h a t Marana showed no ozone

v i o l a t i o n s , PAG approved t h i s exclusion.* DHS concurred and the

Environmental Protection Agency ( EPA) approved the nonattainment area

boundaries i n 1979. EPA based i t s approval on the f a c t t h a t 1) the

carbon monoxide NAA encompassed t h e a r e a s where carbon monoxide v i o l a t i o n s

occurred and 2) no data supported including Marana i n the ozone area.

Pima NAA Boundaries Are Larger Than

Needed t o Control Carbon Monoxide i n Tucson

The Pima NAA is l a r g e r than necessary t o address carbon monoxide problems

i n Tucson. Carbon monoxide p o l l u t i o n is r e l a t i v e l y i s o l a t e d i n Tucson,

caused l a r g e l y by commuter t r a f f i c . However, the NAA encompasses an

extensive a r e a , almost half of which does n o t c o n t r i b u t e t o ~ u c s o n ' s

commuter t r a f f i c .

Carbon monoxide p o l l u t i o n i s limited t o a small area of Tucson. Violation

of Federal carbon monoxide standards occurred a t only one of the t h r e e

monitors i n the c i t y during 1980. Two monitors recorded v i o l a t i o n s i n

1981. These monitors a r e located a t i n t e r s e c t i o n s which experience the

heaviest t r a f f i c flow. Thus, the carbon monoxide problem in Tucson

appears t o b e l a r g e l y a matter of emissions from commuter t r a f f i c i n

s p e c i f i c places r a t h e r than a general problem throughout the a i r shed.

Commuter t r a f f i c t r a v e l s i n t o Tucson from only about half of the Pima

County NAA-- those areas within approximately 10 t o 20 miles of the c e n t r a l

c i t y ( Figure 1). In c o n t r a s t , the Maricopa NAA c o n s i s t s almost e n t i r e l y

of areas which c o n t r i b u t e commuting t r a f f i c t o Phoenix. Between 72 and 83

percent of the v e h i c l e miles traveled i n Pima County occurs w i t h i n t h e

urbanized portion of t h e county. Thus, t o t h e extent t h a t any program is

e f f e c t i v e , c o n t r o l of carbon monoxide p o l l u t i o n i n Tucson probably can be

achieved with smaller VEI Program boundaries than a r e presently i n e f f e c t .

* A t the time the nonattainment area was defined, Tucson exceeded

standards f o r both ozone and carbon monoxide.

FIGURE 1

COMMUTING PATTERNS IN ARIZONA CARBON MONOXIDE

NONATTAINMENT AREAS

Pirna C o u n t y

N o n a t t a i n m e n t A r e a

M a r i c o p a C o u n t y

N o n a t t a i n r n e n t A r e a

A r e a s generating m a j o r c o m m u t e r t r a f f i c

19

Our a n a l y s i s i n d i c a t e s t h a t o u t l y i n g a r e a s can be excluded from t h e NAA

without a f f e c t i n g carbon monoxide p o l l u t i o n i n Tucson. Thus, excluding

the Town of Marana is appropriate. To be c o n s i s t e n t , however, other

outlying areas should be removed from the carbon monoxide nonattainment

area i f they do n o t g e n e r a t e commuter t r a f f i c . Two areas, Green Valley

and Oro Valley, requested exclusion i n 1979. Although PAG d i r e c t e d its

s t a f f t o i n v e s t i g a t e such a c t i o n , t h e d a t a required by EPA were not

a v a i l a b l e t o support exclusion. However, EPA now only requires t h a t an

area demonstrate by any means t h a t no problem e x i s t s . Our a n a l y s i s

suggests t h a t Green Valley c o n t r i b u t e s l i t t l e t o the carbon monoxide

problem i n Tucson. Consequently, excluding Green Valley would probably

n o t a d v e r s e l y a f f e c t the problem. Oro Valley, on t h e o t h e r hand, l i e s

within the commuting radius and, thus, c o n t r i b u t e s t o Tucson's carbon

monoxide problem.

Larger Boundaries May Be Required

t o E f f e c t i v e l y Control Ozone

If e f f e c t i v e c o n t r o l of ozone can be accomplished through t h e VEI Program,

it may require an area- wide VEI Program. Because ozone occurs and spreads

throughout the a i r basin, the more circumscribed area which would

e f f e c t i v e l y c o n t r o l carbon monoxide may not a l l e v i a t e ozone p o l l u t i o n . I f

ozone c o n t r o l requires a l a r g e r area, then no outlying areas should be

excluded from the emissions inspection program. However, e f f e c t i v e

c o n t r o l of ozone through the e x i s t i n g VEI Program may not be f e a s i b l e .

Although Pima County i s n o t c u r r e n t l y a nonattainment area f o r ozone,

h e a l t h o f f i c i a l s i n d i c a t e t h a t t h e problem i s growing and may r e q u i r e

a t t e n t i o n i n coming years. Unlike carbon monoxide, ozone is an area- wide

problem because it forms i n the upper atmosphere and spreads with the

p r e v a i l i n g winds. Since the Pima County NAA is a well- defined a i r shed,

t h i s mass of a i r may s t a y i n the area f o r several days u n t i l a major

weather f r o n t pushes it out. To the extent t h a t any emissions inspection

program is e f f e c t i v e , c o n t r o l of ozone may require including a l l vehicles*

i n the a i r shed i n the VEI Program.

A l l vehicles means t h o s e r e q u i r e d by law t o undergo emissions t e s t s .

Current law requires gasoline- powered vehicles l e s s than 14 years old

t o be t e s t e d .

The extent t o which e f f e c t i v e c o n t r o l of ozone r e q u i r e s an a r e a l a r g e r

than needed f o r carbon monoxide p o l l u t i o n c o n t r o l depends on the extent t o

which nonurban a r e a s c o n t r i b u t e the precursors of ozone-- hydrocarbons ( HC)

and n i t r o u s oxides ( NOx)-- to the atmosphere. These outlying areas

account f o r between approximately 17 and 28 percent of v e h i c l e m i l e s

traveled i n the NAA. PAG and DHS need t o analyze c a r e f u l l y t h e nature of

ozone p o l l u t i o n i n t h e a r e a t o determine i f nonurban v e h i c l e t r a v e l

c o n t r i b u t e s s i g n i f i c a n t l y t o the ozone problem i n the area. I f nonurban

vehicles a r e s i g n i f i c a n t c o n t r i b u t o r s , then the NAA boundary should be

changed t o include Marana. Although no one area is l i k e l y t o show

s i g n i f i c a n t impact on a i r q u a l i t y , a finding t h a t the o v e r a l l nonurban

c o n t r i b u t i o n is s i g n i f i c a n t would provide no b a s i s f o r excluding any

o u t l y i n g a r e a . I f the nonurban c o n t r i b u t i o n is not s i g n i f i c a n t , then PAG

should consider reducing the s i z e of the NAA t o exclude a l l outlying areas.

In a l l decisions on the NAA boundaries, however, DHS and PAG should

consider t h a t e f f e c t i v e c o n t r o l of ozone may not be p o s s i b l e through the

e x i s t i n g VEI Program, r e g a r d l e s s of i t s boundaries, because of the

following reasons:

1. Available data i n d i c a t e the VEI Program h a s n o t had an impact on

carbon monoxide l e v e l s ( see Finding I ) . Thus, f a c t o r s which

l i m i t the Program's e f f e c t on carbon monoxide may a l s o a f f e c t i t s

impact on ozone.

2. Some h e a l t h o f f i c i a l s i n d i c a t e t h a t n i t r o u s oxides, which a r e not

c o n t r o l l e d by the VEI Program, may have more impact on ozone than

hydrocarbons. Since the VEI Program has been i n e f f e c t

hydrocarbon l e v e l s have decreased, but n i t r o u s oxides and ozone

l e v e l s have r i s e n a t the r a t e of 4 t o 5 percent annually i n Pima

County.

3. Few vehicles f a i l emissions t e s t s f o r hydrocarbon l e v e l s . Only

20 percent of 200,000 t e s t f a i l u r e s involve hydrocarbons. The

remaining f a i l u r e s involve carbon monoxide only.

CONCLUSION

The Pima County NAA boundaries do not provide an appropriate b a s i s f o r

addressing current and p o t e n t i a l a i r q u a l i t y problems caused by motor

vehicle emissions i n the Tucson area. Depending on the nature of the

p o l l u t i o n problem, the boundaries a r e e i t h e r too l a r g e o r too small f o r an

e f f e c t i v e VEI Program. However, other f a c t o r s suggest t h a t the e x i s t i n g

VEI Program may not e f f e c t i v e l y reduce ozone, r e g a r d l e s s of i t s boundaries.

RECOMMENDATIONS

1. The Pima Association of Governments and Department of Health Services

should analyze the nature and sources of automotive p o l l u t i o n i n the

Tucson a i r shed t o determine the most e f f e c t i v e area f o r c o n t r o l l i n g

c u r r e n t and a n t i c i p a t e d a i r q u a l i t y problems. In evaluating the

optimum s i z e f o r the nonattainment area, PAG and DHS should a l s o

consider the a b i l i t y of the e x i s t i n g VEI Program t o e f f e c t i v e l y

c o n t r o l ozone.

2. Based on the r e s u l t s of the analysis recommended above, PAG and DHS

should a d j u s t the nonattainment area boundaries t o e i t h e r a) include

a l l areas which c o n t r i b u t e t o Tucson's current and p o t e n t i a l

automotive a i r p o l l u t i o n o r b) exclude those areas which do not.

FINDING I11

THE BUREAU OF VEHICULAR EMISSIONS INSPECTION CAN SAVE $ 121,700 I N STATE

AND FEDERAL FUNDS ANNUALLY BY ELIMINATING AIR/ FUEL WAIVERS.

The a i r / f u e l waiver provision of Arizona's Vehicle Emissions Inspection

Program is not c o s t e f f e c t i v e . Although s t a t u t e s require the Department

of Health Services ( DHS), Bureau of Vehicular Emissions Inspec tion ( BVEI

or Bureau) t o issue a i r / f u e l waivers, l e s s than 2 percent of the motorists

use t h i s service. The Bureau could save an estimated $ 89,500 i n State

funds and $ 32,300 i n Federal funds annually by eliminating the a i r / f u e l

waiver program. While the Bureau maintains t h a t such a change would l i m i t

its authority t o regulate the repair industry, there is no indication t h a t

current s t a t u t o r y provisions intended to g i v e t h e Bureau such broad

regulatory powers.

Statutory Provisions

f o r Air/ Fuel Waivers

As a service t o motorists, S t a t e law requires the Bureau t o issue a i r / f u e l

waivers f o r vehicles f a i l i n g the emissions inspection t e s t for carbon

monoxide ( GO) only. Arizona Revised Statutes ( A. R. S.) $ 36- 1772. E. s t a t e s :

" Vehicles which f a i l the curb i d l e t e s t solely because

of a i r gas mixture a r e e n t i t l e d t o a c e r t i f i c a t e of

waiver upon correction of the problem, a f t e r having

furnished s a t i s f a c t o r y evidence of correction to the

d i r e c t o r on a form t o be prescribed by the director.''

To comply with A. R. S. $ 36- 1772. E., the Bureau requires a i r / f u e l waiver

applicants t o repair t h e v e h i c l e s a t State- approved f a c i l i t i e s and t o have

the waiver form signed by a S t a t e - c e r t i f i e d mechanic. To p a r t i c i p a t e i n

the program, repair f a c i l i t i e s must r e g i s t e r t h e i r analyzers with the

Bureau and employ a t l e a s t one automotive technician c e r t i f i e d by the

Bureau. The Bureau i n s p e c t s r e g i s t e r e d analyzers p e r i o d i c a l l y and t r a i n s

mechanics prior to issuing c e r t i f i c a t i o n .

Carbon monoxide emissions or f a u l t y a i r / £ uel mixture is the most common

cause f o r f a i l i n g emissions inspection. The Bureau inspects over 1.2

m i l l i o n vehicles annually of which approximately 160,000 f a i l the t e s t

s o l e l y due t o carbon monoxide problems.* These v e h i c l e owners a r e

e l i g i b l e t o seek an a i r / f u e l waiver.

Most E l i g i b l e Motorists Do Not

Use the Air/ Fuel Waiver Program

Although most motorists whose vehicles f a i l t h e emissions t e s t s a r e

e l i g i b l e t o apply f o r an a i r / f u e l waiver, the overwhelming majority simply

r e t u r n t h e i r vehicles t o a S t a t e s t a t i o n f o r a f r e e retest.** Compared t o

the r e t e s t , o b t a i n i n g a n a i r l f u e l waiver i s a complicated, lengthy

procedure.

Only 2,400 ( o r 1.5 p e r c e n t ) o f the 160,000 m o t o r i s t s e l i g i b l e t o apply f o r

a i r / f u e l waivers u t i l i z e d the waiver procedure during f i s c a l year

1981- 82. Instead the vast majority ( up t o 98 percent) of e l i g i b l e

motorists returned t o the inspection s t a t i o n f o r a f r e e r e t e s t . According

t o BVEI, the low u t i l i z a t i o n i n f i s c a l year 1981- 82 i s t y p i c a l of most

years.

The a i r l f u e l waiver procedure is more complicated and c o s t l y than the f r e e

r e t e s t . The f r e e r e t e s t is simple and convenient. Motorists have r e p a i r s

made a t f a c i l i t i e s of t h e i r own choice and t h e n t a k e the f r e e r e t e s t a t

the inspection s t a t i o n s . To obtain a waiver, however, the Bureau requires

a p p l i c a n t s t o 1) r e p a i r t h e vehicles a t State- approved f a c i l i t i e s , 2 ) have

completed forms signed by a S t a t e - c e r t i f i e d mechanic, 3) pay b1 f o r the

c e r t i f i c a t e i n a d d i t i o n t o the $ 5.44 paid f o r the i n i t i a l t e s t , and 4)

mail t h e forms t o the Bureau's c e n t r a l o f f i c e . Motorists may not have

time t o mail the forms requesting the waiver t o the Bureau e s p e c i a l l y i f

it i s c l o s e t o t h e v e h i c l e ' s r e - r e g i s t r a t i o n deadline. Thus, providing

a i r / f u e l waivers a s an a l t e r n a t i v e t o the f r e e r e t e s t a t the inspection

s t a t i o n s has only marginal b e n e f i t s f o r motorists.

* An a d d i t i o n a l 40,000 vehicles f a i l both the CO and hydrocarbon ( HC)

t e s t s .

- 9- ,. ,. Under the terms of the c u r r e n t c o n t r a c t with Hamilton Test Systems, a

motorist whose vehicle f a i l e d the f i r s t t e s t can have a r e t e s t a t no

a d d i t i o n a l charge.

Air/ Fuel Waiver Program I s Costly

The cost of administering the a i r l f u e l waiver program is s u b s t a n t i a l and

not supported by f e e s charged. The estimated annual cost f o r processing

2,400 a i r l f u e l waivers i s $ 147,000, or 19 percent of the Bureau's budget,

while the income from the $ 1 f e e charged f o r each waiver is about $ 2,400-

The waivers do not generate s u f f i c i e n t income t o reasonably cover the

a d m i n i s t r a t i v e c o s t s . Thus, $ 144,600 of the waiver s e r v i c e ' s

a d m i n i s t r a t i v e expenses a r e subsidized by other funding sources.*

Increasing the f e e charged f o r the waiver, however, is not a f e a s i b l e

s o l u t i o n s i n c e t h e current cost is $ 61 per waiver.

The a i r l f u e l waiver s e r v i c e ' s a d m i n i s t r a t i v e c o s t s i n c l u d e i n s p e c t i o n o f

r e g i s t e r e d analyzers i n r e p a i r f a c i l i t i e s , yearly c e r t i f i c a t i o n of

mechanics, t r a i n i n g , c l e r i c a l , t r a v e l and equipment expenses. Table 3

gives the cost breakdown f o r work a c t i v i t i e s r e l a t e d t o processing 2,400

a i r l f u e l waivers.

TABLE 3

AIRIFUEL WAIVERS PROCESSING COST,

FISCAL YEAR 1982- 83

ITEM

Personne 1 :

Inspectors

Trainers

Administrative

( Asst. Bureau c h i e f )

C l e r i c a l

Employee r e l a t e d

Operating overhead

Vehicles ( t r a v e l )

C a l i b r a t i n g g a s

Total

ANNUAL COST

* Of the $ 144,600, $ 112,400 a r e General Fund appropriations and $ 32,200

a r e Federal funds.

Eliminating t h e Air/ Fuel Waiver

Program Would Save $ 121,700

BVEI could save approximately $ 89,500 i n S t a t e funds and $ 32,200 i n

Federal funds i f S t a t e law did not require a i r l f u e l waivers. Without t h i s

requirement the Bureau would not need t o maintain s u r v e i l l a n c e over

p r i v a t e r e p a i r f a c i l i t i e s .

Since the waiver program is of marginal b e n e f i t t o m o t o r i s t s , the Bureau

could save approximately $ 121,700 i n S t a t e and Federal funds annually by

eliminating the program. Currently, the Bureau has 700 r e g i s t e r e d

analyzers i n r e p a i r f a c i l i t i e s and 800 c e r t i f i e d mechanics employed by

these f a c i l i t i e s . * The Bureau inspects analyzers and t r a i n s mechanics f o r

c e r t i f i c a t i o n . The Bureau can eliminate the requirements f o r r e g i s t e r e d

analyzers and c e r t i f i e d mechanics i n r e p a i r f a c i l i t i e s i f the s t a t u t e s did

n o t r e q u i r e a i r / f u e l waivers. As Table 4 i n d i c a t e s , t h i s would save

$ 121,700 i n S t a t e and Federal funds o r 16 percent of the Bureau's f i s c a l

year 1982- 83 budget.

* The Bureau a l s o has approximately 400 r e g i s t e r e d analyzers f o r f l e e t

operations and c e r t i f i e s 765 mechanics who inspect f l e e t vehicles. The

cost savings p r o j e c t i o n is f o r a i r / f u e l waivers procedures only and does

( I

not reduce inspection and t r a i n i n g f o r f l e e t operations and f o r the

Hamilton inspection s t a t i o n s .

TABLE 4

ITEM

PKOJECTED ANNUAL SAVINGS

BY ELIMINATING AIRIFUEL WAIVERS*

- S t a t e - funds:

Personal Services -

Inspec t o r s

C l e r i c a l

Employee- related

expenditures

Operating overhead

expenses

Vehicles ( t r a v e l )

C a l i b r a t i n g g a s

Subtotal S t a t e

Less current income

Net S t a t e savings

FTEs

Federal Funds:

Personal Services -

Trainer 1.0

Employee- related

Operating overhead -

Subtotal Federal - 1.0

Total: All Funds 4-. 0

AMOUNT OF SAVINGS

* Based on the 1982- 83 budget

By eliminating the a i r / f u e l waiver program t h e Bureau could reduce i t s

s t a f f by four p o s i t i o n s : 2.5 i n s p e c t o r s , one t r a i n e r and .5 c l e r i c a l

p o s i t i o n s . The four FTEs' s a l a r i e s and r e l a t e d employee and operating

c o s t s amounts t o a $ 112,800 annual reduction. In a d d i t i o n , t h e Bureau

would save $ 11,300 i n t r a v e l and c a l i b r a t i n g gas expenses. The t o t a l

reduction would be $ 124,100. Since the Bureau w i l l n o t r e c e i v e the $ 2,400

annual income from the waivers f e e s , t h e net savings a r e $ 121,700 i n S t a t e

and Federal funds annually o r 1 6 p e r c e n t of t h e Bureau's 1982- 83 budget.

These savings do not include the c o s t of s t a f f members who devote only a

small p o r t i o n of t h e i r time t o the a i r l f u e l waiver function while the

$ 147,000 i s based on t o t a l processing cost. Thus, the Bureau can

implement the savings without jeopardizing other functions. Moreover, the

S t a t e could save a t l e a s t $ 89,500 of General Fund monies even i f the

f e d e r a l l y sponsored t r a i n i n g program is not reduced i n scope.

Bureau May Be Exceeding

L e g i s l a t i v e I n t e n t

Although a i r l f u e l waivers a r e not c o s t e f f e c t i v e , t h e Bureau has not

requested the L e g i s l a t u r e t o a b o l i s h t h e s t a t u t o r y requirement. Rather,

the Bureau u t i l i z e s the requirement a s a t o o l t o r e g u l a t e t h e automobile

r e p a i r industry i n general. The Bureau chief s t a t e d t h a t t h e purpose of

c e r t i f i c a t i o n is t o ensure t h a t mechanics a r e t r a i n e d t o properly r e p a i r

a l l vehicles r a t h e r than t o s i g n o f f on the few a i r / f u e l waivers.

However, there is no i n d i c a t i o n t h a t t h e provisions of A. R. S. 536- 1772. E.

were meant t o give the Bureau such broad regulatory power over the repair

industry.

CONCLUSION

The Bureau could save $ 89,500 i n S t a t e funds and $ 32,200 i n Federal funds

annually i f S t a t e law d i d n o t r e q u i r e a i r l f u e l waivers. Few motorists use

the waiver process, which is c o s t l y . Eliminating the a i r l f u e l waivers

w i l l not inconvenience the ~ u b l i c . Currently over 98 percent of those

e l i g i b l e f o r a i r l f u e l waivers simply r e t u r n t o the inspection s t a t i o n f o r

a f r e e r e t e s t .

RECOPDIE NDATIONS

1. The L e g i s l a t u r e should consider amending A. R. S. 536- 1772. E. t o d e l e t e

t h e requirement f o r an a i r / f u e l waiver program.

I f the Legislature d e l e t e s the a i r l f u e l waiver requirement, BVEI

should :

a. Delete i t s administrative rules and regulations r e l a t i n g t o

a i r / f u e l waivers and

b. Review i t s p o l i c i e s , procedures and work a c t i v i t i e s and

eliminate those functions related t o a i r l f u e l waivers. The

Bureau should revise i t s budget and reduce personnel t o r e f l e c t

t h i s program change.

FINDING I V

ALTHOUGH THE BUREAU OF VEHICULAR EMISSIONS INSPECTION AND THE CONTRACTOR

HAVE INCREASED THE FREQUENCY OF EQUIPMENT CHECKS, ADDITIONAL CONTROLS ARE

NEEDED TO ASSURE ACCURATE AND RELIABLE EMISSIONS TESTING.

Both the Bureau of Vehicular Emissions Inspection ( BvEI) and Hamilton Test

Systems ( FITS) have increased the frequency of equipment f i e l d a u d i t s a t

inspection s t a t i o n s ; however, a d d i t i o n a l c o n t r o l s a r e needed t o ensure

accurate and r e l i a b l e emissions t e s t i n g . S p e c i f i c a l l y , BVEI needs t o

1) improve management c o n t r o l over i t s f i e l d a u d i t program and 2) require

HTS by c o n t r a c t t o conduct inspection s t a t i o n f i e l d a u d i t s . In a d d i t i o n ,

BVEI needs t o conduct more timely a u d i t s of f l e e t and r e g i s t e r e d analyzers

i n p r i v a t e f a c i l i t i e s .

Need f o r Frequent

Emissions Eauioment Checks

BVEI and Hamilton Test Systems assure the accuracy and r e l i a b i l i t y of

emissions t e s t i n g equipment by conducting p e r i o d i c f i e l d a u d i t s a t the

vehicle emissions inspection s t a t i o n s . A f i e l d a u d i t is an inspection

which uses a blend of gases of known proportions t o t e s t the a b i l i t y of

equipment t o sample and analyze emissions accurately. In e f f e c t , a f i e l d

a u d i t approximates a c t u a l emissions t e s t conditions.

HTS equipment is among the b e s t a v a i l a b l e f o r emissions t e s t i n g , but even

the most r e l i a b l e equipment needs t o be checked frequently.

Administrative Rule R- 9- 3- 1025. A. r e q u i r e s BVEI t o conduct equipment

checks twice each month a t each of the 30 permanent t e s t lanes i n Phoenix

and Tucson.*

* The mobile inspection s t a t i o n i n Green Valley i s checked once each

month.

In a d d i t i o n , HTS has e s t a b l i s h e d i t s own p o l i c i e s c a l l i n g f o r two

a d d i t i o n a l BVEI- type f i e l d a u d i t s p e r month. Thus, a t o t a l of four f i e l d

a u d i t s should be conducted monthly a t each t e s t lane. This frequency of

a u d i t i n g is comparable t o the minimum requirements i n o t h e r s t a t e s with

c e n t r a l i z e d programs. For example, C a l i f o r n i a , which operates a

c e n t r a l i z e d emissions i n s p e c t i o n program using Hamilton equipment,

reported t h a t f i e l d i n s p e c t o r s check s t a t i o n s t h r e e t o four times

monthly.

Field auditing is not t h e s o l e q u a l i t y assurance procedure. The DHS- HTS

c o n t r a c t r e q u i r e s HTS t o c a l i b r a t e analyzers p e r i o d i c a l l y . C a l i b r a t i o n

d i f f e r s from f i e l d auditing i n t h a t separate t e s t gases a r e used f o r each

component of the system ( e. g., carbon monoxide, hydrocarbons). Although

t h i s procedure does not t e s t the e n t i r e system's a b i l i t y t o sample and

analyze t h e combination of gases i n vehicle exhausts, c a l i b r a t i o n does

ensure t h a t each component a c c u r a t e l y records gas concentrations. Auditor

General review o f a sample of HTS records f o r f i s c a l year 1981- 82 shows

the c o n t r a c t o r t o be i n compliance with c a l i b r a t i o n requirements of the

c o n t r a c t .

BVEI and HTS Did Not Conduct

Appropriate Number of Audits

BVEI did not conduct t h e number of i n s p e c t i o n s t a t i o n f i e l d a u d i t s

required by r e g u l a t i o n during f i s c a l year 1981- 82. Further, HTS did not

conduct a l l f i e l d a u d i t s required by i n t e r n a l company policy.

BVEI conducted only 56 percent of required f i e l d a u d i t s i n f i s c a l year

1981- 82. The Bureau did not conduct required f i e l d a u d i t s from July 1981

through March 1982. No f i e l d a u d i t s were conducted a t a l l during

February. These f i e l d a u d i t s were not conducted because t h e employee

r e s p o n s i b l e f o r t h i s function f a i l e d t o c a r r y out h i s duties.* BVEI

management was unaware of t h i s deficiency u n t i l a q u a l i t y assurance techni-

* The employee has since been terminated from s e r v i c e .

c i a n , who was not r e s p o n s i b l e f o r i n s p e c t i o n s t a t i o n f i e l d a u d i t s ,

informed t h e A s s i s t a n t Bureau Chief of the problem. The Bureau began

complying with i t s r e g u l a t i o n i n April 1982 and since t h e n h a s conducted

v i r t u a l l y a l l required f i e l d a u d i t s of i n s p e c t i o n s t a t i o n s .

Hamilton Test Systems a l s o did not conduct a l l f i e l d a u d i t s required by

i n t e r n a l company p o l i c i e s i n f i s c a l year 1981- 82. U n t i l March 1982, HTS

informal p o l i c i e s c a l l e d f o r one monthly f i e l d a u d i t f o r each t e s t lane.

In April 1982, HTS f i e l d a u d i t p o l i c i e s were revised t o r e q u i r e two such

a u d i t s per month. HTS f a i l e d t o conduct t h e a u d i t s because they were

u n c e r t a i n a s t o t h e s p e c i f i c procedures t o be used. DHS had not supplied

HTS with needed information f o r t e s t i n g and HTS was i n t h e process of

converting from one equipment system t o another. I n a d d i t i o n , some t e s t

r e s u l t s were not being recorded. Like DHS, however, Hamilton has complied

with i t s p o l i c i e s s i n c e A p r i l 1982.

TABLE 5

BVEI AND HTS STATE STATION AUDITS,

FISCAL YEAR 1981- 82

1981

J u l y

August

September

October

Novernbe r

December

1982

January

February

March

April

May

June

Totals

- DHS - HT S

Lanes Audited Lanes Audited

Percent o f Percent of

Number Required Number Required

Additional Controls Are Needed

Although BVEI and HTS have increased the frequency of t h e i r equipment

a u d i t s a t inspection s t a t i o n s , a d d i t i o n a l c o n t r o l s a r e needed. BVEI needs

t o 1) i n s t i t u t e a formal management reporting system and 2) require HTS

by c o n t r a c t t o conduct f i e l d a u d i t s .

The Bureau has not yet e s t a b l i s h e d a formal management reporting system t o

inform top management of f i e l d a u d i t performance. The Bureau Chief does

not know on a timely b a s i s 1) whether a l l required f i e l d a u d i t s have been

conducted and 2) the nature and e x t e n t of t e s t lane equipment f a i l u r e .

Thus, t h e r e is no assurance t h a t the problem which occurred i n f i s c a l year

1981- 82 w i l l not recur.

While Hamilton Test Systems now conducts two f i e l d a u d i t s per t e s t lane

per month, the c o n t r a c t o r is not required t o do so. The DHS contract with

Hamilton does not r e q u i r e r e g u l a r f i e l d a u d i t s of inspection s t a t i o n s nor

reporting of r e s u l t s t o DHS. Requiring HTS by contract t o conduct two

monthly a u d i t s and t o report r e s u l t s t o DHS would ensure t h a t a t o t a l of

four f i e l d a u d i t s a r e performed on each t e s t lane per month. As noted

e a r l i e r , t h i s frequency of f i e l d auditing i s comparable t o the minimum

requirements i n other s t a t e s which have inspection programs s i m i l a r t o

Arizona's.

BVEI Did Not Conduct Timely Audits

of Analyzers i n P r i v a t e F a c i l i t i e s

BVEI a l s o did not conduct timely inspections of f l e e t and r e g i s t e r e d

analyzers i n p r i v a t e f a c i l i t i e s i n f i s c a l year 1981- 82. As a r e s u l t ,

f a u l t y analyzers may have remained i n service longer than necessary.

Although BVEI r u l e s and p o l i c i e s require the Bureau t o inspect f l e e t and

r e g i s t e r e d analyzers every t h r e e months, approximately 3 7 percent of the

i n s p e c t i o n s exceeded the 90- day requirement i n f i s c a l year 1981- 82. As

shown i n Table 6, f l e e t and r e g i s t e r e d a n a l y z e r i n s p e c t i o n s which exceeded

90 days averaged 117 and 124 days, r e s p e c t i v e l y , and ranged as high a s 264

days-- nearly 9 months. These inspect ions were not conducted because

employees f a i l e d t o carry out t h i s f u n c t i o n p r o p e r l y and management did

not take a c t i o n t o address the deficiency.

Because i n s p e c t i o n s have not been timely, f a u l t y analyzers may have

remained i n s e r v i c e longer than necessary. One- fourth of the analyzers i n

our sample f a i l e d t h e q u a l i t y assurance t e s t i n 1981- 82. I f t e s t e d a s

required, f a u l t y analyzers would be taken out of s e r v i c e and adjusted t o

meet BVEI accuracy standards. This would provide more assurance t h a t

vehicles inspected a t f l e e t s t a t i o n s and p r i v a t e garages a r e properly

t e s t e d . A t l e a s t 10 percent of the vehicles inspected annually pass

through these f a c i l i t i e s .

TABLE 6

TIMELINESS OF QUALITY ASSURANCE INSPECTIONS

FOR SAMPLE OF FLEET AND REGISTERED ANALYZERS,

FISCAL YEAR 1981- 82

Greater Than Average Days In

Total Within 90 Days 90 Days Excess of 90

Number Percent Number Percent

F l e e t

S t a t i o n s 167 116 69% 51 31% 11 7

Registered

Analyzer

S t a ti ons - 152 - 84 55 - 68 45 - 124

Total 3- 19 - 200 6- 3% - 119 3- 7% 1- 21

CONCLUSION

Both BVEI and Hamilton Test Systems have increased the frequency of

equipment f i e l d a u d i t s a t inspection s t a t i o n s . However, a d d i t i o n a l

c o n t r o l s a r e needed t o a s s u r e a c c u r a t e and r e l i a b l e emissions t e s t i n g .

DHS a l s o needs t o conduct more timely i n s p e c t i o n s of f l e e t and r e g i s t e r e d

analyzers i n p r i v a t e f a c i l i t i e s .

1. BVEI should e s t a b l i s h a system of twice monthly r e p o r t s t o the Bureau

Chief, noting the number of S t a t e inspection s t a t i o n lanes f i e l d

audited by both Bureau and Hamilton Test Systems personnel and the

a u d i t r e s u l t s .

2. BVEI should require by contract t h a t Hamilton Test Systems conduct two

f i e l d a u d i t s per month and report the r e s u l t s of the a u d i t s t o the

BVEI Quality Assurance Manager.

3. The BVEI Quality Assurance Manager should ensure t h a t inspectors

conduct timely q u a l i t y assurance i n s p e c t i o n s of f l e e t and r e g i s t e r e d

analyzers a s required by Bureau r e g u l a t i o n s and policy. Results of

these i n s p e c t i o n s should b e r e p o r t e d a t l e a s t monthly t o the Bureau

Chief.

OTHER PERTINENT INFORPATION

During the a u d i t , other p e r t i n e n t information was developed regarding

1) vehicle inspection f e e payments and c o l l e c t i o n s and 2) the emissions

t e s t equipment and procedures used i n the program.

Vehicle Inspection Fee

Pavments and Collections

Since January 1981 the Bureau of Vehicular Emissions Inspection ( BVEI) has

paid Hamilton Test Systems ( HTS) $ 575,000 l e s s than the amount b i l l e d by

t h e c o n t r a c t o r . This s h o r t f a l l i n payments occurs f o r two reasons:

1) t h e r e i s a s i g n i f i c a n t time lag between emissions t e s t s and remittance

of funds by the county assessors and the Motor Vehicle Division ( MvD),

and 2) fees are n o t c o l l e c t e d f o r a l l t e s t s performed. Although BVEI has

proposed ways t o address the cash flow problem, the problem of c o l l e c t i n g

f o r a l l t e s t s conducted has not been addressed.

Collection Process - Current s t a t u t e s provide f o r emission t e s t f e e s t o be

c o l l e c t e d by t h e county assessors o r the Motor Vehicle Division of the

Department of Transportation a t the time of v e h i c l e r e g i s t r a t i o n . * These

funds a r e remitted on a monthly b a s i s t o BVEI which then pays HTS f o r

t e s t s performed. Vehicle owners must pay $ 5.44 f o r the f i r s t inspection

and the same amount f o r the t h i r d , f i f t h and seventh subsequent t e s t s .

Each of these paid t e s t s includes a f r e e r e t e s t , i f needed.

S h o r t f a l l i n Payments - As shown i n Table 7 , BVEI has paid HTS

approximately $ 575,000 l e s s than the t o t a l amount b i l l e d by t h e c o n t r a c t o r

since January 1981. This s h o r t f a l l i n payments t o HTS r e p r e s e n t s the

d i f f e r e n c e between the value of emission t e s t s performed by HTS ($ 11.2

m i l l i o n ) and the amount BVEI received from the county assessors and PlVD

( b10.6 million) f o r the period January 1981 through September 1982.

* MVD has taken over the v e h i c l e r e g i s t r a t i o n function i n Pima County.

In a d d i t i o n , some vehicle owners r e g i s t e r t h e i r vehicles a t the PiVD

c e n t r a l headquarters i n Phoenix.

TABLE 7

BILLINGS AND PAYMENTS FOR EMISSIONS INSPECTIONS,

JANUARY 1981 THROUGH SEPTEMBER 1982

Payments Unpaid Balance

Amount Percent of Percent of

Period B i l l e d Amount B i l l i n g Amount B i l l i n g a

January -

December

1981 $ 6,252,344 $ 5,783,680 92.5% $ 468,664 7.5%

January -

September

1982 4,915,808 4,809,115 97.8 106,693 2.2

Total

The s h o r t f a l l grew s u b s t a n t i a l l y during the f i r s t year of the current

c o n t r a c t t o approximately $ 469,000. The s h o r t f a l l grew more slowly i n

1982-- increasing about $ 107,000. BVEI paid HTS 98 percent of the amount

between January and September 1982 compared t o 93 percent i n the f i r s t

year of the new c o n t r a c t . T o t a l payments since January 1981 a r e 95

percent of t o t a l b i l l i n g s .

Causes of S h o r t f a l l - The s h o r t f a l l i n payments t o HTS appears t o occur

f o r two reasons. F i r s t , t h e r e is a s i g n i f i c a n t time lag between emissions

t e s t s and remittance of funds t o BVEI. Second, fees a r e not c o l l e c t e d for

a l l t e s t s performed by HTS.

The amount of time t h a t may elapse between an emissions inspection and the

r e c e i p t of f e e f o r t h a t inspection appears t o be a major c o n t r i b u t o r t o

the d i f f e r e n c e between b i l l i n g s and payments. Although HTS b i l l s f o r

i n s p e c t i o n s a t the end of each month, BVEI may n o t r e c e i v e payment f o r as

long as 135 days a f t e r a vehicle i s inspected. As a r e s u l t , BVEI r e c e i p t s

and payments have been l e s s than HTS b i l l i n g s i n 13 of the 21 months since

the beginning of the c u r r e n t c o n t r a c t .

The time lag between emissions t e s t i n g and remittance of funds is

permitted by DHS r u l e s and regulations. Regulations require emissions

i n s p e c t i o n s 90 days p r i o r t o t h e r e g i s t r a t i o n renewal date. Regulations

a l s o s t i p u l a t e t h a t county assessors o r MVD r e m i t s i n s p e c t i o n f e e s

c o l l e c t e d a t the time of r e g i s t r a t i o n by the 15th of the month following

c o l l e c t i o n . Thus, c o l l e c t i o n procedures c r e a t e a " p i p e l i n e e f f e c t " which

delays r e c e i p t of inspection f e e s up t o 135 days. I f vehicles a r e

r e g i s t e r e d l a t e , t h i s time lag can be even g r e a t e r .

Results of a recent DHS a u d i t confirmed t h a t the time lag e x i s t s between

t e s t i n g and remittance of funds by county assessors and MVD t o BVEI. Fees

f o r 41 p e r c e n t o f emissions t e s t s conducted i n October 1981 were not

c o l l e c t e d u n t i l the following o r subsequent months. In a d d i t i o n , a u d i t o r s

found t h a t some funds were held f o r up t o s i x weeks a f t e r c o l l e c t i o n .

However, such r e t e n t i o n is permitted by BVEI r e g u l a t i o n s and the audit

found no evidence t h a t county assessors and MVD were not properly handling

emissions f e e s or were not complying with applicable regulations.

The s h o r t f a l l i n payments a l s o occurs because f e e s a r e not c o l l e c t e d f o r

a l l t e s t s performed. According t o BVEI analyses, approximately 2.3

percent of HTS b i l l i n g s s i n c e J u l y 1981 include f e e s f o r vehicles which do

not q u a l i f y f o r r e g i s t r a t i o n . These are vehicles which f a i l e d the i n i t i a l

t e s t and have not passed or were waivered a t the time HTS b i l l e d f o r the

i n i t i a l test. The DHS a u d i t i d e n t i f i e d 3.9 percent of October 1981 t e s t s

which were s t i l l uncollected i n March 1982.

BVEI a l s o f e e l s t h a t some motorists do not pay a s required f o r a l l t e s t s

performed on t h e i r vehicles. These a r e motorists who must r e t e s t t h e i r

vehicles a t h i r d , f i f t h or even seventh time and should pay fees f o r these

odd- numbered t e s t s i n a d d i t i o n t o the i n i t i a l t e s t f e e ( even- numbered

t e s t s a r e f r e e r e t e s t s ) . The BVEI- HTS c o n t r a c t r e q u i r e s BVEI t o c o l l e c t

fees f o r more than one t e s t f o r a motorist only when HTS n o t i f i e s the

assessor o r MVD t h a t a d d i t i o n a l f e e s a r e required.

Although HTS has implemented some procedures i n an attempt t o ensure t h a t

the inspection c e r t i f i c a t e s r e f l e c t a l l charges f o r multiple t e s t s ,

m o t o r i s t s can e a s i l y circumvent these procedures. S t a t i o n personnel punch

entry documents ( renewal c e r t i f i c a t e s o r t i t l e s f o r example) t o show t h a t

the v e h i c l e has been t e s t e d . However, motorists can submit vehicles f o r

a d d i t i o n a l t e s t s and avoid charges by switching documents. For example,

Auditor General s t a f f submitted two vehicles f o r eleven paid inspections.

Only one HTS inspector i d e n t i f i e d p r i o r t e s t s t h a t should have been added

t o the inspection c e r t i f i c a t e .

A l t e r n a t i v e s f o r Reducing S h o r t f a l l - Although BVEI has proposed ways t o

address the cash flow problem which c o n t r i b u t e s t o the s h o r t f a l l i n

payments, the problem of c o l l e c t i n g f e e s f o r a l l t e s t s conducted has not

been addressed.

bVEI has i d e n t i f i e d two s o l u t i o n s t o the c o l l e c t i o n s cash flow problem.

F i r s t , f e e s could be c o l l e c t e d by Hamilton a t the time of emissions

t e s t i n g . This method of c o l l e c t i o n s was used p r i o r t o January 1981, but

was inconvenient f o r the public. Hamilton accepted only cash payment and

personal checks were not accepted. Collecting $ 5.44 i n cash or any other

odd amount, a t the time of t e s t i n g , would represent an even g r e a t e r

inconvenience i f Hamilton continued t o r e q u i r e cash payment.

Second, BVEI has proposed t h a t county assessors and MVD remit emissions

t e s t i n g f e e s on a d a i l y b a s i s . T h i s would improve cash flow and i s

c o n s i s t e n t with a s i m i l a r Auditor General recommendation regarding

remittance of other vehicle r e g i s t r a t i o n fees by t h e counties ( s e e Auditor

General Report number 82- 31.

Neither DHS nor HTS have developed procedures t o ensure c o l l e c t i o n of fees

f o r a l l t e s t s performed. This would require improving c o n t r o l s a t

inspection s t a t i o n s t o i d e n t i f y motorists who receive multiple

inspections. Use of s i n g l e e n t r y documents o r unique v e h i c l e

i d e n t i f i c a t i o n numbers may be needed t o allow HTS t o n o t i f y county

a s s e s s o r s and MVD of the proper amount t o be c o l l e c t e d .

Emissions Test Equipment

and Procedures

An important p a r t of the Vehicle Emissions Inspection ( VEI) Program a u d i t

was a review of t e s t r e l i a b i l i t y and accuracy of records. Finding I V

addresses the major r e l i a b i l i t y i s s u e , q u a l i t y assurance. Our research

i n d i c a t e s t h a t t h e equipment used t o inspect v e h i c l e emissions i s a s good

as any a v a i l a b l e . Pioreover, t e s t procedures and data handling a r e , with

two exceptions, adequate t o ensure r e l i a b l e t e s t i n g and b i l l i n g .

Equipment - Competitors and a major c l i e n t a l i k e told Auditor General

s t a f f t h a t the Arizona VEI Program c o n t r a c t o r , Hamilton Test Systems

( HTS), i s a leader i n the emissions t e s t i n g f i e l d . An o f f i c i a l of the

C a l i f o r n i a emissions program described HTS as a pioneer i n t h e f i e l d . A

r e p r e s e n t a t i v e of a company which competed against HTS f o r the Arizona

c o n t r a c t said t h a t he knew of no b e t t e r equipment and t h a t h i s company

used HTS as a standard when making i t s bid. Another competitor described

i t s equipment as superior but added t h a t it designed its equipment t o meet

C a l i f o r n i a standards. HTS designed the o r i g i n a l equipment f o r t h a t

s t a t e ' s program. F i n a l l y , a r e p r e s e n t a t i v e of one of the major automobile

manufacturers described HTS as a leader i n both developing t e s t i n g

equipment and t e s t procedures. Thus, t o the extent t h a t HTS maintains its

equipment, the equipment is a s good as any a v a i l a b l e . As noted i n Finding

I V Y however, the Department of Health Services ( DHS) has not adequately

inspected HTS equipment i n the past t o ensure t h a t it functions up t o i t s

c a p a b i l i t y .

Test Procedures - The VEI Program t e s t equipment is f u l l y automated t o

provide accurate t e s t i n g and recording of t e s t data. A l l t h a t an

inspector must do is e n t e r and v e r i f y t h e v e h i c l e i d e n t i f i c a t i o n d a t a ,

i n s e r t the t a i l p i p e probe and the system automatically samples the

exhaust, analyzes the gas and r e p o r t s the r e s u l t s . According t o HTS

personnel, s t a t i o n i n s p e c t o r s cannot a f f e c t t h e conduct of the t e s t .

Moreover, the system is programmed t o abort t e s t i n g i f major problems

occur t h a t would r e s u l t i n erroneous t e s t r e s u l t s .

However, HTS personnel do not always e n t e r t h e proper engine s i z e

information ( number of engine c y l i n d e r s ) i n t o t h e system, r e s u l t i n g i n the

p o s s i b i l i t y t h a t vehicles w i l l be measured on the wrong standards. In 6

of 14 instances where Auditor General s t a f f submitted vehicles f o r

emissions inspection, s t a t i o n personnel did not ask the number of

c y l i n d e r s or check under the hood. The entry document did not provide

t h i s information. In f i v e of the s i x i n s t a n c e s , s t a t i o n personnel

i n c o r r e c t l y assumed the vehicle had s i x r a t h e r than four c y l i n d e r s . T h i s

e r r o r reduced allowable emissions from 2.5 t o 2.2 percent-- a 12 percent

reduction.

B i l l i n g Procedures - Hamilton Test Systems' control over the b i l l i n g

process is s u f f i c i e n t l y adequate t o ensure t h a t b i l l s submitted t o the

Department of Health Services a r e accurate. Although some weaknesses

e x h i s t i n the recording and processing of data which r e s u l t s i n the HTS

b i l l , the weaknesses do not s i g n i f i c a n t l y a f f e c t b i l l i n g accuracy.

Auditor General EDP s t a f f reviewed the process by which HTS records and

handles t e s t information. This process begins when a vehicle e n t e r s a

t e s t s t a t i o n and ends with the preparation of a b i l l t h a t HTS submits t o

the Department. The process i s r e l i a b l e and accurate. Direct recording

of t e s t information on the s t a t i o n computer, processing of a l l records

with no d e l e t i o n s and separation of programming and systems functions a r e

strong c o n t r o l f e a t u r e s i n the process.

Although EDP s t a f f noted s e v e r a l weaknesses, only two appeared

s i g n i f i c a n t . F i r s t , s t a t i o n i n s p e c t o r s sometimes f a i l t o a c c u r a t e l y e n t e r

the proper number of engine c y l i n d e r s . As noted above, t h i s problem

occurred with vehicles submitted by Auditor General s t a f f f o r inspection

and can r e s u l t i n erroneous r e s u l t s . The second weakness i s t h a t HTS can

change vehicle information. This produces t h e p o t e n t i a l f o r f a l s e

b i l l i n g . However, a spot check of b i l l i n g s did not i n d i c a t e t h a t such

f a l s e b i l l i n g s had occurred.

GLOSSARY

AIR/ FUEL WAIVER: Waiver issued to vehicles f a i l i n g the emissions t e s t for

carbon monoxide only. A motorist may obtain an a i r / f u e l waiver upon

c o r r e c t i n g t h e problem and furnishing s a t i s f a c t o r y evidence of

correction t o the Bureau of Vehicular Emissions Inspection.

AIR SHED: Area i n which a i r pollutants mix but cannot escape in the

absence of a major weather f r o n t . An a i r shed provides the basis for

area- wide a i r quality management.

AMBIENT AIR: The a i r around us.

BUREAU OF VEHICULAR EMISSIONS INSPECTION: The Bureau within the Department

of Health Services that administers the Vehicle Emissions Inspection

Program.

CARBON MONOXIDE ( ~ 0 ) : Odorless, colorless gas emitted by motor vehicles.

Carbon monoxide deprives the blood of oxygen. A large concentration

can k i l l ; a smaller amount can cause d i z z i n e s s , f a t i g u e , headaches and

slower d r i v i n g r e a c t i o n time.

FIELD AUDIT: An inspection of an emissions analyzer using a blend of t e s t

gases i n which the r e l a t i v e proportion of each gas is known. ~ i e l d

audits are conducted to ensure accuracy and r e l i a b i l i t y of emissions

analyzers-

HAMILTON TEST SYSTEMS ( HTS): A private company which operates the S t a t e

emissions inspection s t a t i o n s under a contract with the Department of

Health Services.

HYDROCAKBONS ( HC): Atmospheric p o l l u t a n t caused by unburned gasoline i n

engine exhaust by evaporation l o s s e s from petroleum storage and

handling and by evaporation from organic solvents. Hydrocarbons a r e

one component of the p o l l u t a n t ozone.

MAXICOPA ASSOCIATION OF GOVERNIVIENTS ( MAG): Regional Planning Organization

responsible f o r developing plans f o r reducing automotive a i r p o l l u t i o n

i n Maricopa County.

NITkOUS OXIDES ( NO ): Atmospheric p o l l u t a n t produced where fuel is X

burned a t high temperatures, such a s i n motor vehicles o r power

p l a n t s . This p o l l u t a n t is a component of ozone.

NONATTAINMENT AKEA ( NU): An area where the a i r q u a l i t y standards are

v i o l a t e d f o r a given p o l l u t a n t . In Arizona, NAA boundaries f o r carbon

monoxide a r e a l s o the boundaries i n which the emissions inspection

program is i n e f f e c t .

OZONE: Photochemical oxidants which r e s u l t from r e a c t i o n s i n the

atmosphere between hydrocarbons and n i t r o u s oxides i n the presence of

s u n l i g h t . Ozone is popularly known as smog.

PIMA ASSOCIATION OF GOVERNlYENTS ( PAG): Regional Planning Organization

responsible f o r developing plans t o reduce automotive a i r p o l l u t i o n i n

Pima County.

TIME SERIES ANALYSIS: A s t a t i s t i c a l method of studying movements i n a s e t

of chronologically ordered observations.

TONS PEE DAY: Unit of measurement f o r carbon monoxide and hydrocarbons.

VEHICLE EMISSIONS INSPECTION ( VEI) PROGRAM: A program enacted by the

Arizona L e g i s l a t u r e requiring c e r t a i n motor vehicles t o pass an annual

inspection t o ensure t h a t t h e i r exhaust emissions meet standards

e s t a b l i s h e d by the Department of Health Services. The standards

specify minimum emissions f o r carbon monoxide and hydrocarbons*

WRITTEN RESPONSES TO THE AUDITOR GENERAL'S REPORT

ARIZONA DEPARTMENT OF HEALTH SERVICES

PIPIA ASSOCIATION OF GOVERNPENTS

HAMILTON TEST SYSTEMS, INC.

ARIZONA DEPARTMENT OF HEALTH SERVICES

q Office of the Director f' " -)

@ BRUCE BABBITT, Governor

I

February 1 , 1983

JAMES E SARN, M D , M P H , D~ rector tat- ,+, !- \- \.: , ;"

C ""

Mr. Douglas Norton

Auditor General

111 West Monroe Street

Phoenix, Arizona 85007

Dear Mr. Norton:

This is in response to the performance audit report of the Vehicle Emissions

Inspection Program conducted pursuant to Senate Bill 1220 enacted by the

34th Legislature in 1980.

In our meeting on January 11, 1983 and in our memorandum of January 17, 1983,

we pointed out serious problems with the time series analysis and the resultant

conclusions provided by your consultant, the Center for Informative Eva1 uation.

In spite of our well founded objections which are supported by experts in the

field of Air Quality, the performance audit report continues to be based upon

a study which lacks understanding of air quality complexities and is misleading.

The analysis is flawed to the point of rendering it invalid. The analysis fails

to support the conclusion that " the I/ M program has had no salutary impact on

ambient air quality in Arizona." A more appropriate conclusion of the study

might be that the model, data and consultant utilized were unable to detect

any effect either positive or negative. More importantly the contractor did

not make a sufficient attempt to reconcile his analysis with the significant

empirical evidence which shows the program is effective in reducing emissions.

The following experts support our position and their comments can be found as

Attachments 1, 2 and 3:

John Trijonis, Ph. D., an environmental scientist with extensive

experience in air pollution studies in California, Dr. Trijonis

has also served on several National Academy of Sciences air quality

committees.

Robin Dennis, Ph. D., Staff Scientist for the National Center for

Atmospheric Research, Boulder, Colorado, Dr. Dennis is involved in

the development of stochastic models of ambient CO data in Denver.

Mr. Phil Lorang, Chief, Technical Support Staff, U. S. E. P. A., Ann

Arbor, Michigan, and author or co- author of several technical papers

relating to the effects of vehicle emissions on air quality.

Terry Woodf ield, Ph. D. , Assistant Professor of Mathematics at Arizona State

University, also reviewed the report to evaluate the application of the

methodology. His comments, noting impediments to the use of this Report in

b

The Department of Health Services is An Equal Opportunity Affirmative Action Employer. All qualified men and

women, including the handicapped, are encouraged to participate.

State Health Building 1740 West Adams Street Phoenix, Arizona 85007

Mr. Douglas Norton

Page Two.

decisionmaking, are included as Attachment 4. Our detailed comments relating

to the study are contained in Attachment 6. Additional comments regarding

Finding I and comments on the Summary, Findings 11, 111, IV, and Other Per-tinent

Information are contained in Attachment 5. Attachment 7 is Hamil ton

Test Systems' response to Finding IV. *

We are in the process of initiating a more detailed study of carbon monoxide

air quality in Arizona. This study will be conducted by recognized profes-sionals

in the field of air quality. Because of the considerable expert

opinion which is in disagreement with your consultant's report, we recommend

that you delay the issuance of your report or appropriately modify it until

such time as the detailed study is complete.

We appreciate your concern shown in considering our comments on these drafts.

If you have any questions concerning our comments, please contact R. Fred

Iacobelli, Chief, Bureau of Vehicle Emissions Inspection at 255- 1135.

JES: RFI: db

Attachments

Sincerely,

A+-- J mes E. Sarn, 4 . a ~ ~

* ( The Auditor General has chosen t o include t h i s as a separate response.

Therefore, there is no Attachment 7 . )

SANTAF E RESEARCHC ORPORATION

ROUTE 7 BOX 124K SANTA FE, NEW MEXICO 87501 ( 505) 983- 6568

R. Fred Iacobelli, P. E.

Acting Chief, Bureau of Air Quality

Arizona Department of Health Services

1740 West Adams

Phoenix, Arizona 85007

Dear Mr. Iacobelli:

January 10, 1983

As per your request, I have reviewed the January 1983 report " Time

Series Analysis of the Impact of the Vehicle Emissions Inspection Program

( I/ M) on Ambient Air Qua1 i ty in Phoenix and Tucson." In my opinion, the

analyses and results in this report do not support i t s principal conclusion

that " the I / M program has had no salutary impact on ambient a i r qual ity

in Arizona." More importantly, this report alone certainly cannot be

considered adequate j u s t i f i c a t i o n t o discontinue or modify the current I/ M

program. The report does, however, suggest the need to analyze and resolve

possible inconsistencies between ambient a i r quality trends and calculated

emi ss i on trends.

I find that the aforementioned report contains several important

omissions and defects. Before I discuss the specific deficiencies, however,

I would like t o r a i s e a more general issue. This issue concerns the general

d i f f i c u l t y in interpreting trends of ambient a i r quality data. I have been

involved in numerous a i r quality trend studies performed for federal, s t a t e ,

and local agencies. In some of these studies, I have found that ambient

a i r quality can yield confusing if not wrong implications. Not only are

b a i r quality data continually confounded by meteorological fluctuations, b u t

also unexpl icable aberrations often creep in due to undocumented changes in

monitoring procedures or other unknown factors. I think that i t i s very

important to analyze a i r qual i ty data to make sure that ongoing control pro-grams

are having the expected effect. When the a i r quality trends confirm the

predictions based on emission analyses, the results are reassuring and are

supportive of the control programs. When the ambient data seem to contradict

emission trend data, such as be the case for CO in Arizona, then there i s

a need to analyze the problem further and resolve the discrepancy. However,

considering the vagaries of meteorology and potential errors in ambient

a i r quality data, such a discrepancy should definitely not be immediately

interpreted as a f a u l t in the emission data or a failure of the control program.

Such an interpretation would be particularly inappropriate with regard to the

results of the report in question.

I would now 1 i ke to address some of the major deficiencies i n the report;

these are discussed below:

The main conclusion in the report, " The I/ M program has had no salutary

impact on ambient CO air quality in Arizona: is overstated. Assumingthat

the statistical results are correct, a more reasonable conclusion would

appear t o be that " no evidence can be found that the I/ M program has

had an impact on CO air quality in Arizona." More importantly, depend-ing

on the standard errors of the s t a t i s t i c s ( see next comment), the

conclusion might actually be " no evidence can be found t h a t the I/ M

program has not had the expected 3- 15% improvement in air quality

during the two interventions!

t

A major omission in the report is the lack of standard errors for the

trend statistics. If these standard errors are quite large, the results

of the report may not be inconsistent with a lo%, 20%, or 30% improvement

in CO air qua1 i ty due to I/ M.

0 A ma. ior deficiencv in the statistical a ~ ~ r o a cihs the failure to

expl ici t l y control for traffic growth, federal emission standards, and

meteor01 ogy. A1 1 three of these phenomena can potentially produce

greater effects than the I/ M program. Traffic levels evidently grew by

about 50% over the 8- year study period; new car standards produced a

large decrease in total emissions over the period; and meteorology leads

typically to 10- 20% fluctuations in air quality indices on a year- to-year

basis. I t is a major defect t o search for the impact of I/ M without

explicitly controlling for the other factors, especially when the other

factors can be included in an accurate fashion ( e. g. gasoline sales

adjusted for fuel efficiency to represent traffic, emission models for

the new car standards, and Arizona DHS meteorological normalization

procedures). The authors of the report say that they have " controlled"

for these factors, b u t this is true only t o the extent that meteorological

fluctuations are purely seasonal and that traffic growth and federal

emission reductions represent a constant trend over the period. Neither

of these latter two assumptions is justified. Considering that the three

exogenous factors are likely t o be stronger than the I/ M effect and are

likely to be confounded historically with the I/ M effect, i t is important

t o treat them explicitly in the statistical model.

0 A t several points in the report, the authors erroneously assume that

vehicular CO emissions depend on fuel efficiency. This is not true.

Vehicle emission standards and factors are expressed in gm/ mi, and total

vehicular emissions depend on these factors and traffic levels ( e. g.

miles traveled). There is no consistent relationship between fuel

efficiency and vehicular emissions. Fortunately, I think that this error

does not affect the statistical results in the report. The error does,

however, seem to bear on some of the interpretations and qualifications

made by the authors.

o The functional form used t o represent the I/ M program -- stepped increases 1

spread out over 12 months in 1977 and 1979 -- may be inappropriate. As

I understand i t , the Arizona DHS has expectations and evidence that

there was a voluntary response t o I/ M in 1976 and that there was a

continued improvement in 1978 and 1980 as vehicle owners modified their

behavior in response t o I/ M. I think that, in further analyses, the

appropriate I/ M transition function should be selected in consultation

wi t h Arizona DHS.

The above paragraphs summarize my major conclusions concerning the

report. If necessary, I would be happy to meet with you t o discuss any of

these issues in greater detail. Please contact me if you have any questions

or if you require any further information.

Sincerely,

. -. -

I.'

'\ / d* ;.* .{*+ - " .-- { 4 $~--*, - 3 -

L John Trijonis

ATTACHMENT 2

NATIONAL CENTER FOR ATMOSPHERIC RESEARCH

P. 0. Box 3000 Boulder, Colorado 80307

Telephorze: ( 303) 494- 51.51 TW, Y. YIO- 940- 3245 Trle.~ 4: 5 694 I-' TS: 322- 5151

1 4 January 1983

Fred Acabella

Arizona Department of Health Services

S t a t e Health Building

Phoenix, AZ 85007

Dear Fred:

This l e t t e r is i n response t o your request for our comments on portions

of a report by Richard McCleary on h i s time s e r i e s analysis of the impact

of IM on ambient CO l e v e l s i n Phoenix and Tucson. Currently we are involved

i n an e f f o r t t o develop s t o c h a s t i c models of ambient CO data i n Denver,

t h e r e f o r e we are w e l l aware of the complexity of the problem and current

l i t e r a t u r e on the topic.

It is d i f f i c u l t t o comment on the analysis because so l i t t l e a c t u a l d a t a

is included i n the two appendices we received, so our comments must be gen- '

e r a l . The annual means shown i n Table 3a of Appendix I r a i s e some i n t e r e s t -

ing questions, however. The annual means over a l l of the data sets show

generally the same p a t t e r n of ups and downs by year: a decline 1974- 76,

l e v e l l i n g o f f 1977- 81. The random f l u c t u a t i o n s about t h i s general trend a r e

to be expected due to differences i n meteorology from year to year. Coin-c

i d e n t a l l y , t h e r e a r e increases i n 1979 i n a l l data s e t s , and i n 1977 i n a l l

data s e t s except Phoenix Sunnyslope. Such f l u c t u a t i o n s a r e not unusual but

they i l l u s t r a t e the d i f f i c u l t y of separating i n t e r v e n t i o n e f f e c t s from the

equally abrupt changes which can be caused by changing weather patterns.

Therefore it is e s s e n t i a l t o have some understanding of causal r e l a t i o n s h i p s

and to view the CO data i n r e l a t i o n t o important explanatory v a r i a b l e s . We

cannot accept McCleary's contention t h a t a u n i v a r i a t e ARINA mode1, will ac-c

u r a t e l y s e p a r a t e IM e f f e c t s from a l l other e f f e c t s and t h a t it is not neces-sary

t o understand the reasons f o r observed changes.

The accuracy of any estimated parameters requires t h a t the model be c o r r e c t l y

formulated. There a r e several reasons t o doubt the correctness of McCleary's

ARIMA model: ( 1) As noted above, random f l u c t u a t i o n s due to unexplained

sources a c t u a l l y led to increased CO l e v e l s i n the years the IM e f f e c t s were

expected t o occur, hence changes i n weather and t r a f f i c p a t t e r n s should be

investigated a s p o t e n t i a l causes. ( 2) Emissions changes due to IM predicted

by the MOBILE2 model d i f f e r from the one- year s t e p change used i n McCleary's

model. ( 3) It has been shown t h a t strongly seasonal time s e r i e s based on

d e t e r m i n i s t i c physical processes are l i k e l y to be i n c o r r e c t l y described by a

standard ARIMA model and require, i n s t e a d , a periodic ARMA model.. Points ( 2)

and ( 3) r e q u i r e f u r t h e r elaboration.

The National Center for Atmospheric Research is Operated by the University Corporation

for Armosphenc Research under sponsorship of the National Science Foundation.

An Equal Opporrunity/ Affirmative Action Employer

Fred Acabella

14 January 1983

Page 2

In regard to point ( 2), EPA t e s t e d the e f f e c t s of I M on emissions i n Portland,

Oregon, over s e v e r a l successive inspections and, i f . necessary, readjustments

a t 3 month i n t e r v a l s . The r e s u l t s , incorporated i n MOBILE2, showed t h a t

f u r t h e r reductions i n mean emissions were obtained on successive inspections

( Rutherford, 1982). MOBILE2 p r e d i c t s an i n i t i a l step change due to the

f i r s t inspection and a d d i t i o n a l smaller step changes due to the second and

t h i r d inspections. Also, i n the Arizona program mandatory annual inspections

of motor vehicles began i n January 1976 but r e p a i r of f a i l e d vehicles was

voluntary. W e would assume t h a t some few f a i l e d vehicles w e r e repaired i n

1976. Thus it is q u i t e possible t h a t some IM e f f e c t would be incorporated

i n the trend portion of a u n i v a r i a t e model over several years and would not

appear as a s t e p change.

In regard t o point ( 3), the a c t u a l models used a r e not shown i n e i t h e r of the

two appendices t h a t we have seen. We can only assume t h a t they were standard

u n i v a r i a t e ARIMA models of the type described i n McClearyls book ( Applied

. Time Series Analysis f o r the Social Sciences, 1980). These methods use

seasonal differencing t o accomodate the c y c l i c behavior of seasonal data, and

assume t h a t t h e mean and autocovariance function of a differenced s e r i e s a r e

homogeneous over a l l seasons. However, i n strongly seasonal d a t a t h e assump-t

i o n of homogeneity can be q u i t e inappropriate. Tiao and Grupe ( 1980) have

shown t h a t t h i s can lead t o a misspecified model which looks acceptable, but

is much less accurate than a model which includes periodic means and/ or per-i

o d i c covariances. Cleveland and Tiao ( 1979) demonstrate t h e e f f e c t s of such

an inappropriate model choice on monthly ozone data. The standard ARIMA model

c l e a r l y d i d not produce as good a f i t as the a l t e r n a t i v e , a periodic ARMA model.

This simply means t h a t d i f f e r e n t parameter estimates were needed f o r d i f f e r e n t

months of t h e year to get a good f i t . Similar e f f e c t s can be expected f o r

monthly CO data because, l i k e ozone, it has a s t r o n g p h y s i c a l l y determined

seasonal p a t t e r n .

Lacking controls t h a t can be shown t o account f o r changes i n weather, t r a f f i c ,

and f e d e r a l emission standards, it is possible t h a t McCleary's model is in-capable

of i s o l a t i n g t h e e f f e c t s of IM. In h i s model it is l i k e l y t h a t some

weather e f f e c t s a r e included i n t h e s t e p change component which he a t t r i b u t e s

to IM and t h a t some IM e f f e c t s a r e included i n the trend component which is

supposed to represent only non- IM changes. There is no way of t e l l i n g , since

the model has no explanatory power. Thus McCleary's conclusion t h a t IM had no

impact is overstated. It would be more accurate t o say t h a t no impact could

be detected using t h i s p a r t i c u l a r model.

It is also possible t h a t the Arizona data base f o r 1974- 81 is incapable of

supporting any model of the necessary precision. McCleary's recommendations on

development of a f u t u r e data base a r e e x c e l l e n t and should be implemented.

Fred Acabella

14 January 1983

Page 3

We agree with PlcCleary t h a t the 1979 Arizona DHS study of ambient e f f e c t s of

I M has serious problems which put its conclusions i n doubt. McCleary shows

a lack of understanding of the empirical s t u d i e s of emissions e f f e c t s of IM,

however. Meteorology and climate a r e not relevant i n a controlled FTP t e s t

of t a i l p i p e emissions. Thus the controlled emissions t e s t s o f f e r empirical

evidence i n support of IM. Whether emissions reductions due to IM can be

discerned i n ambient CO measurements remains t o be seen.

References

Cleveland, W. P. and G. C. Tiao ( 1979). Modeling Seasonal Time Series. Revue

Economic Appliquee, 32, pp. 107- 29.

Rutherford, J. ( 1982). Derivation of I / M Benefits f o r Pre- 1981 Light Duty

Vehicles f o r Low Altitude, Non- California Areas. EPA- AA- IMS- 82- 3.

Tiao, G. C. 6 M. R. Grupe ( 1980). Hidden Periodic Autoregressive- Moving Average

Models i n Time Series Data. Biometrika, 67, 2, pp. 365- 73.

Very t r u l y yours,

Robin Dennis

& dZ

Mary Downton

ATTACHMENT 3

Fret 2accbo: ii

Bureau of Vehicle Dission

Ins; 3ecticn

600 SO* 40th Street

? hoenix kZ 85C08

Thgr, k ycc fcr the oppcrtucity to cement on the Janrizry 6,

2,033 draft report t i t l e d ' The Series Analysis of the Iopiiet

of rke Vchicie B. issions Ing2ectiort P , ' Q ~ T ~ ( cI ,&) cn Anbierrt

Air Quality in Phoenix and Tucson.* 1 have already coEr. entad

by on earlier letter on a preuic~ s d r a f t of this report.

Ey rest serious ccaplaint ahaut t h e previou- s & aft applies t o

this report also, mcely that the autttbrs reset tea negative-ly

te their fa. ilt? re t o reject the null hypothesis that I/) I

kas had an affeer cr: CO levels, The authors seen t:, 20

beyond conventional statistical repcrting to perssade the

reaAer that the inability to reject the n u i l hypothesis

prdves that the ~ u l l hypothecis sfiauId he accepted. I ak

l e f t vanc? tritig uhetber the authcr8 have sirply last sight of

the ! inits of hypothesis testing as a netfiod cf positive

procf, or vhether they tldve a pre- existin? eci~ us tovsrds X/ E

ir, particular. e unexplainab1e failure to report the

stcndard er: or of the eatinatea nakes it inpossit- le for the

reaEer t o . judge . how nuch weight to attribute to the icabilftj.

t o reject the n c l l hypcthesis. If as i suspect the stanEzrd

QrrorE are large, then any nuEber of Rore aptinistic

hypotheses about l/ Kts effect could also be imune from

rejectior,. It is true that the point estinates cf I/ Ets

effect are saall, but if the standard errors are t a r ~ e , t here

is ir good probability of Seein9 such result$ by pure chance

even i f I/ E were as effective as believed,

The authors ~ a k esu ch of having used three CO sites and up t o

fcur CO cecsurcs ir! the analysis, for a t o t d l of eight

site- ceescte cotpbinatioas. Pbey t t y t o convince the reader

t h a t if the I/ H p r q r m failed tc show a significant benefit

in . eight ' quasi- experinente the fj)! Frogran has been

positively proven ta be ineffective. In fact, however, sites

in the sane city and different Deasures of CO are not

independent enperinents for testin? ih. The study really

has cnly pivan i/) r tro opportunities to prove itself, cnce

( Corrected 2/ 2/ 83)

f o r Phoenix and once f o r Tucson. Even these may not be

independent, s i n c e r e g i o n a l weather p a t t e r n s could influence

both c i t i e s .

The r e p o r t i n a number of p l a c e s s a y s t h a t emissions decrease

a s f u e l economy i n c r e a s e s . T h i s r e v e a l s a lack of

understanding of Federal emission c o n t r o l r e g u l a t i o n s , which

r e g u l a t e emissions i n grams per mile, not per q u a n t i t y of

f u e l burned.

The r e p o r t r e f u s e s t o acknowledge t h a t r e s u l t s from emissions

t e s t s of in- use v e h i c l e s i n I/ M programs a r e empirical data

and can be used as evidence t h a t I / M reduces CO emissions.

Only a d i s b e l i e v e r i n t h e conservation of mass could argue

t h a t emissions can be reduced without improving a i r q u a l i t y .

Of course, it is p o s s i b l e t o pose arguments t h a t t h e very

l a r g e body of evidence from tests of in- use v h i c l e s i n

P o r t l a n d , Los Angeles, and Phoenix could be misleading f o r

reasons such a s p o t e n t i a l r e c r u i t i n g b i a s , d i f f e r e n c e s

between the FTP and r e a l world d r i v i n g c o n d i t i o n s , e t c .

However, t h e r e p o r t f o r t h e most p a r t chooses t o ignore t h i s

body of evidence r a t h e r than d i s c u s s its merits f a i r l y . T h i s

is u n f o r t u n a t e , s i n c e I b e l i e v e t h e evidence supports a t

l e a s t a t e n t a t i v e conclusion t h a t I / M works, and t h a t it is

t h e r e f o r e u n f a i r t o p r e f e r a n u l l hypothesis t h a t it has zero

e f f e c t .

In t h e main p a r t of t h e a n a l y s i s , t h e a u t h o r s u s e a model

which ignores t h e p o s s i b i l i t y t h a t i r r e g u l a r year- to- year

f l u c t u a t i o n s i n t r a f f i c volume and meteorology could be

obscuring a p o s i t i v e b e n e f i t f o r I/ M. The a u t h o r s ' attempts

t o r a t i o n a l i z e t h i s model a r e s t a t i s t i c a l l y i n c o r r e c t . My

understanding is t h a t t h e ARIMA m ~ d e l ' s N t term c o n t r o l s

f o r l i n e a r and seasonal t r e n d s b u t n o t f o r year- to- year

weather d i f f e r e n c e . The r e p o r t does not adequately inform

t h e reader of t h i s shortcoming.

The r e p o r t includes a s h o r t s e c t i o n on r e s u l t s obtained f o r a

model which uses t r a f f i c volume e s t i m a t e s and a t e r m p e r a t u r e

i n v e r s i o n v a r i a b l e . In p r i n c i p l e , t h e s e two v a r i a b l e s should

e x p l a i n some of t h e v a r i a t i o n i n CO l e v e l s , t h u s reducing the

standard e r r o r s and improving t h e model's power. Since

t r a f f i c and weather may be c o r r e l a t e d w i t h the I / M interven-t

i o n , including them as v a r i a b l e s can change t h e expected

value of t h e r e g r e s s i o n e s t i m a t o r s a l s o . It t u r n s out t h a t

t h e normalized data a r e sometimes more v a r i a b l e than before

normalization and the I / M e f f e c t s a r e still not s i g n i f i c a n t l y

d i f f e r e n t from zero. The f a c t t h a t t h e data become more

v a r i a b l e w i t h the normalization s u g g e s t s t o me t h a t perhaps

t h e p a r t i c u l a r temperature i n v e r s i o n v a r i a b l e is f o r some

reason a poor i n d i c a t o r of CO d i s p e r s i o n p o t e n t i a l , and t h a t

o t h e r meteorological v a r i a b l e s such as wind speed or r e l a t i v e

humidity should have been tried. In any case, the fact that

even with this particular normalization it is impossible to

reject the null hypothesis of no effect from I/ M does not

prove that I/ M has no effect, as the authors attempt to

conclude.

If I can be of further assistance; please call.

Sincerely Ayours,

Phil Lorang, C d ief

Technical Support Staff

,. a;, t2 ST.<>

V'

,--

, :--\ -, .-

a , UNITED ST,.!, TES E>;\/ iR3"\:'. bi;,. j 1 , AIL PzCTEY,::;): .; GF, , C'y ; a,- p 2 '. ; 7n- z . . , ,-. , - . \ 8 - * q-

++ - L., L 2 -+ ,, -,-,- b,,, ?- I.'-,-,.-, T, -.\>, Lz

%. ~\\ d

I.: 2 --?-, 6-

I - L1 I

R. Fred I g c o b e l l i , P. E.

A c t i n g C h i e f , Bureau of A i r Q u a l i t y

Arizona Department of Health S e r v i c e s

i-' 1740 West Adams

P h o e n i x , A r i z o n a 85007

Dear Mr. I a c o b e l l i :

Thank you f o r t h e o p p o r t u r , i t y t o review t h e d r a f t Appendix

t i t l e d " Time S e r i e s A n a l y s i s of t h e Impact of t h e Vehicle

Emissions I c s p e c t i o n Program on Ambient A i r Q u a l i t y i n

Phoenix and Tucson" by Richard McCleary and Barbara C.

N i e n s t e d t . S i n c e I an r e s p o n s i b l e f o r EPA1s a c t i v i t i e s i n

e v a l u a t i o n of i n s p e c t i o n a n d maintenance ( I/ M) programs, I am

i n t e r e s t e d i n a l l such e f f o r t s . As you know, my s t a f f

s u p e r v i s e d t h e s t u d y by G. C. Tiao an6 h i s a s s o c i a t e s of t h e

impact of t h e I / M program i n P o r t l a n d , Oregon on t h a t c i t y ' s

CO a i r q u a l i t y , so I am f a m i l i a r with t h e t y p e of f u n c t i o n a l

models which c a n b e used f o r t h i s purpose. The Tiao s t u d y is

l i s t e d a s Reference 22 i n t h e d r a f t Appendix you s e n t me,

which speaks q u i t e f a v o r a b l y of it.

I have o r g a n i z e d my comments on t h e d r a f t Appendix t o a d d r e s s

i n sequence t h e modeling method, t h e - i n t e r p r e t a t i o n of t h e

r e s u l t s of t h e modeling e f f o r t , t h e recommendations o f f e r e d

a t t h e end of t h e Appendix, and my own c o n c l u s i o n s about t h e

o v e r a l l v a l u e and s i g n i f i c a n c e of t h e s t u d y r e p o r t e d i n t h e

Appendix.

Mozeling Pethod and I s s u e s

F i r s t , t h e Appecdix d e s c r i b e s t h e s t u d y by Tiao i n P o r t l a n d

a s t h e o n l y good p r i o r work on t h e s u b j e c t of e s t i m a t i n g I / M

b e n e f i t s from CO a i r q u a l i t y d a t a . U n f o r t u n a t e l y , t h e

Appendix does n o t use t h e same f u n c t i o n a l form or

s p e c i f i c a t i o n a s Tiao d i d f o r t h e model which r e l a t e s CO

l e v e l s t o t h e v a r i a b l e s t h a t c o n t r i b u t e t o t h o s e l e v e l s .

Tiao fount it b e s t t o e x p r e s s CO c o n c e n t r a t i o n a s t h e p r o d u c t

of a CO e m i s s i o n s / m i l e emission f a c t o r term [ k e B t

( 1 + M i n . T i a o l s s y m b o l s ] , a t r a f f i c volume term

[ TRt], and a m e t e o r o l o g y t e r m [ REt]. This makes s e n s e ,

s i n c e CO e m i s s i o n s r e s u l t from t h e p r o d u c t of a. n emission

r a t e per m i l e times VMT and a r e d i l u t e d depending on

meteorology. In c o n t r a s t , t h e Appendix u s e s s e v e r a l

d i f f e r e n t models i n which t h e t r a v e l v a r i a b l e and meteorology

v a r i a b l e a r e added, not m u l t i p l i e d . The Appendix does not

use any term t h a t is c l e a r l y an emission f a c t o r term. The

Appendix does mention t h a t a " normalized" model was a l s o used

( p . 37) which may be more l i k e the more l o g i c a l Tiao

approach, but t h i s " normalized" model is not very well

described ir. t h e Appendix. A l l e l s e a s i d e , the use of t h e

a d d i t i v e model in the Appendix should s t i l l give meaningful

r e s u l t s but they w i l l be l e s s d i r e c t l y i n d i c a t i v e of e f f e c t s

on a i r qua1 i t y .

Second, the Appendix uses t h e s i n g l e h i g h e s t 8- hour CO

average each month t o d e s c r i b e CO a i r q u a l i t y during t h a t

month. Tiao used the average CO value f o r t h e month. The

s i n g l e h i g h e s t value is n o t o r i o u s l y v a r i a b l e , s i n c e it can be

influenced by t r a f f i c volume and weather during very s h o r t

periods and t h e s e i n t u r n may f l u c t u r a t e g r e a t l y . Since the

model i n the Appendix uses monthly averge f u e l s a l e s a s the

measure of t r a f f i c volume and monthly average r e l a t i v e

humidity a s the meteorology v a r i a b l e , t h e CO s t a t i s t i c should

c l e a r l y be based on t h e monthly average a s w e l l . ( I f the

monthly average is not r e a d i l y a v a i l a b l e , the average of each

day's h i g h e s t 8- hour v a l u e s h o u l d be used.) Using the s i n g l e

monthly high value adds noise t o t h e time s e r i e s and makes it

much harder t o show t h a t i n d i v i d u a l f a c t o r s have a

s t a t i s t i c a l l y s i g n i f i c a n t e f f e c t on CO l e v e l s .

Third, a s - mentioned, t h e Appendix used f u e l s a l e s a s t h e

measure of t r a f f i c volume f o r lack of a c t u a l t r a f f i c counts.

The Appendix p o i n t s out the shortcomings of t h i s approach,

but n e g l e c t s t o mention one shortcoming t h a t could have been

c o r r e c t e d : average v e h i c l e f u e l economy has been i n c r e a s i n g ,

s o t h e r e a l i n c r e a s e in VMT over time has beer, f a s t e r than

i n d i c a t e d by f u e l s a l e s alone. The Appendix ignores t h i s .

A s a r e s u l t , it f a i l s t o recognize p a r t of the reduction i n

CO emissions per mile over time, a reduction t h a t in p a r t may

be a t t r i b u t a b l e t o I / M . I have enclosed some t a b l e s from

standard U. S. Department of T r a n s p o r t a t i o n r e p o r t s which

give e s t i m a t e s of the f l e e t f u e l economy i n each calendar

year.

Fourth, t h e Appendix uses a model which does not e x p l i c i t l y

account f o r t h e b e n e f i c i a l e f f e c t s of the new car emission

standards-- the Federal Motor Vehicle Control Program

( FMVCP). Tiao did recognize t h e FMVCP a s a p o s s i b l e e f f e c t ,

and in f a c t did f i n d from t h e data t h a t t h e FMVCP caused a 5%

p e r y e a r r e d u c t i o n in CO emissions and ambient l e v e l s . It is

i n c o n c e i v a b l e t h a t t h e FMVCP h a s n o t a l s o had an e f f e c t i n

Phoenix and Tucson. The Appendix a c c o u n t s f o r t h e e f f e c t

only a s p a r t of t h e " n o i s e " term. I am n o t f a m i l i a r with how

t h i s n o i s e term is t r e a t e d i n t h e ARIMA model used by t h e

Appendix, but t h e f a i l u r e t o r e c o g n i z e t h e p o s s i b l e e f f e c t of

t h e FMVCP may have n e g a t i v e consequences f o r t h e r e s u l t s of

t h e a n a l y s i s .

F i f t h , t h e Appendix u s e s a v e r y s i m p l e v a r i a b l e form t o

d e s c r i b e t h e i n t e r v e n t i o n due t o I / M . It assumes t h a t I / M

has a c o n s t a n t e f f e c t on ambient l e v e l s ( e x c e p t f o r an

a d d i t i o n a l impact from t h e change i n f a i l u r e r a t e ) once a l l

c a r s have been i n s p e c t e d one t i m e . A b e t t e r v a r i a b l e would

have been t h e number o f v e h i c l e s which had been i n s p e c t e d ,

f a i l e d , and t h e n passed t h e r e t e s t a f t e r r e p a i r . This would

c a p t u r e i n one v a r i a b l e t h e e f f e c t of s t a r t i n g t h e I / M

program i n 1977 and i n c r e a s i n g t h e f a i l u r e r a t e i n 1979. It

would a l s o r e f l e c t any . changes i n waiver r a t e s over time;

presumably c a r s which r e c e i v e waivers do n o t c o n t r i b u t e a s

much t o reducing CO l e v e l s a s do s u c c e s s f u l l y r e p a i r e d

v e h i c l e s . S i n c e CO l e v e l s a r e a f f e c t e d by a l l c a r s on t h e

road and n o t j u s t t h o s e b e i n g i n s p e c t e d each month, it is

n e c e s s a r y t o use a c u m u l a t i v e v e h i c l e count f o r t h i s

v a r i a b l e . To r e f l e c t g r a d u a l d e t e r i o r a t i o n i n CO emissions

back towards non- I/ M l e v e l s f o l l o w i n g i n s p e c t i o n and r e p a i r ,

I recommend T i a o ' s method of d i s c o u n t i n g i n s p e c t i o n s from

p r e v i o u s months by p r o g r e s s i v e l y l a r g e r p e r c e n t a g e s . See

page 24 of T i a o ' s f i n a l r e p o r t t o EPA f o r d e t a i l s . Another

a d v a n t a g e o f t h i s method is t h a t one could t h e n e a s i l y

e s t i m a t e t h e b e n e f i t of r e d u c i n g t h e waiver r a t e through a

h i g h e r c o s t l i m i t , b e t t e r t r a i n i n g f o r mechanics, or more

r i g o r o u s r e v i e w o f waiver a p p l i c a t i o n s .

F i n a l l y , t h e Appendix d i f f e r s from t h e Tiao method of

a c c o u n t i n g f o r s e a s o n a l i t y . I am n o t f a m i l i a r enough with

t h e two methods t o s a y whether t h i s is l i k e l y t o have an

e f f e c t on t h e r e s u l t s .

I n t e r p r e t a t i o n of Modelinq R e s u l t s

The Appendix estimated. t h e e f f e c t of t h e I / M program i n

Phoenix o n ' CO l e v e l s i n s e v e r a l d i f f e r e n t ways and g o t

r e s u l t s r a n g i n g from a r e d u c t i o n of 17% t o a r e d u c t i o n o f

42%. This r a n g e c o r r e s p o n d s r o u g h l y t o what EPA models of

I / M would s u g g e s t . However, t h e Appendix r e p o r t s t h a t none

of t h e e s t i m a t e d b e n e f i t s a r e s t a t i s t i c a l l y s i g n i f i c a n t . The

Appendix concludes t h a t t h e r e is t h e r e f o r e no evidence t h a t

I / M has had an e f f e c t , and even implies t h a t t h e evidence

shows it does n o t have an e f f e c t . This is an

o v e r s i m p l i f i c a t i o n of t h e meaning of t h e s t a t i s t i c