Measurement tools for assessing Motor Vehicle Division port-of-entry performance

              Measurement Tools for Assessing
Motor Vehicle Division Port-of-
Entry Performance
FINAL REPORT 527
Prepared by:
Jason Carey
4304 East Campbell Avenue
Phoenix, AZ 85018
SEPTEMBER 2003
Prepared for:
Arizona Department of Transportation
206 South 17th Avenue
Phoenix, Arizona 85007
in cooperation with
U.S. Department of Transportation
Federal Highway Administration
The contents of the report reflect the views of the authors who are responsible for the
facts and the accuracy of the data presented herein. The contents do not necessarily
reflect the official views or policies of the Arizona Department of Transportation or the
Federal Highway Administration. This report does not constitute a standard,
specification, or regulation. Trade or manufacturers’ names which may appear herein
are cited only because they are considered essential to the objectives of the report.
The U.S. Government and The State of Arizona do not endorse products or
manufacturers.
Technical Report Documentation Page
1. Report No.
FHWA-AZ-03-527
2. Government Accession No. 3. Recipient's Catalog No.
4. Title and Subtitle 5. Report Date
September 2003
Measurement Tools for Assessing Motor Vehicle Division
Port-of-Entry Performance
6. Performing Organization Code
7. Authors
Jason Carey
8. Performing Organization Report No.
9. Performing Organization Name and Address
Jason Carey
10. Work Unit No.
4304 East Campbell Avenue, Phoenix, AZ 85018 11. Contract or Grant No.
SPR-PL-1-(59) 527
12. Sponsoring Agency Name and Address
ARIZONA DEPARTMENT OF TRANSPORTATION
206 S. 17TH AVENUE
13.Type of Report & Period Covered
PHOENIX, ARIZONA 85007
Project Manager: John Semmens
14. Sponsoring Agency Code
15. Supplementary Notes
Prepared in cooperation with the U.S. Department of Transportation, Federal Highway Administration
16. Abstract
The Arizona Port of Entry (POE) Program provides a valuable service to the residents of Arizona, but lacks a
clear means of evaluating the efficiency and effectiveness of its enforcement program. This in turn makes it
more difficult to communicate the achievements of the Port of Entry Program, and to identify potential
improvements in service quality. This research addresses the development of measures of performance for
evaluation of the Arizona Port of Entry Program. By developing specific measures tied to the goals and
objectives of the program, Arizona POE managers will have a better set of tools for decision making, and
increased accountability to Arizona taxpayers.
Measures of performance should communicate the need for improvement in an organization, but should
highlight accomplishments as well. Many of the performance measures discussed in the literature emphasized
quantity of a particular unit of measurement (e.g., trucks weighed), but did not relate that quantity to the
operational conditions under which it was achieved. In contrast, the measures recommended in this report
provide a means of relating measurements to the intended outcome of each activity. Comparing revenues to
truck travel, or overweight traffic to the percentage of traffic weighed, indicates the degree to which enforcement
induces compliance with state regulations. Similarly, illustrating the benefits that accrue to highway users as a
result of port of entry services provides a means of evaluating the overall value of POE services.
Performance measures need to be redefined as the priorities of an organization change, and special care must
be taken when comparisons are made between multiple agencies or time periods. The best assessment of the
needs of the port of entry program will come from port managers, who are most familiar with the goals and
operating conditions that affect the ports of entry. The measurements developed for this study were intended to
provide additional tools from which port of entry performance could be managed, but the ultimate responsibility
for selecting and implementing an appropriate measurement system remains with port of entry administrators.
17. Key Words
ports-of-entry; truck weight enforcement;
truck safety enforcement; commercial fee
collection
18. Distribution Statement
Document is available to the
U.S. public through the
National Technical Information
Service, Springfield, Virginia
22161
23. Registrant's Seal
19. Security Classification
Unclassified
20. Security Classification
Unclassified
21. No. of Pages
117
22. Price
SI* (MODERN METRIC) CONVERSION FACTORS
APPROXIMATE CONVERSIONS TO SI UNITS APPROXIMATE CONVERSIONS FROM SI UNITS
Symbol When You Know Multiply By To Find Symbol Symbol When You Know Multiply By To Find Symbol
LENGTH LENGTH
in inches 25.4 millimeters mm mm millimeters 0.039 inches in
ft feet 0.305 meters m m meters 3.28 feet ft
yd yards 0.914 meters m m meters 1.09 yards yd
mi miles 1.61 kilometers km km kilometers 0.621 miles mi
AREA AREA
in2 square inches 645.2 square millimeters mm2 mm2 Square millimeters 0.0016 square inches in2
ft2 square feet 0.093 square meters m2 m2 Square meters 10.764 square feet ft2
yd2 square yards 0.836 square meters m2 m2 Square meters 1.195 square yards yd2
ac acres 0.405 hectares ha ha hectares 2.47 acres ac
mi2 square miles 2.59 square kilometers km2 km2 Square kilometers 0.386 square miles mi2
VOLUME VOLUME
fl oz fluid ounces 29.57 milliliters mL mL milliliters 0.034 fluid ounces fl oz
gal gallons 3.785 liters L L liters 0.264 gallons gal
ft3 cubic feet 0.028 cubic meters m3 m3 Cubic meters 35.315 cubic feet ft3
yd3 cubic yards 0.765 cubic meters m3 m3 Cubic meters 1.308 cubic yards yd3
NOTE: Volumes greater than 1000L shall be shown in m3.
MASS MASS
oz ounces 28.35 grams g g grams 0.035 ounces oz
lb pounds 0.454 kilograms kg kg kilograms 2.205 pounds lb
T short tons (2000lb) 0.907 megagrams
(or “metric ton”)
mg
(or “t”)
Mg megagrams
(or “metric ton”)
1.102 short tons (2000lb) T
TEMPERATURE (exact) TEMPERATURE (exact)
ºF Fahrenheit
temperature
5(F-32)/9
or (F-32)/1.8
Celsius temperature ºC ºC Celsius temperature 1.8C + 32 Fahrenheit
temperature
ºF
ILLUMINATION ILLUMINATION
fc foot candles 10.76 lux lx lx lux 0.0929 foot-candles fc
fl foot-Lamberts 3.426 candela/m2 cd/m2 cd/m2 candela/m2 0.2919 foot-Lamberts fl
FORCE AND PRESSURE OR STRESS FORCE AND PRESSURE OR STRESS
lbf poundforce 4.45 newtons N N newtons 0.225 poundforce lbf
lbf/in2 poundforce per
square inch
6.89 kilopascals kPa kPa kilopascals 0.145 poundforce per
square inch
lbf/in2
SI is the symbol for the International System of Units. Appropriate rounding should be made to comply with Section 4 of ASTM E380
Table of Contents
SUMMARY OF KEY FINDINGS ...............................................................................................1
I. INTRODUCTION ..................................................................................................................5
Why Measure Performance? ........................................................................................... 7
How Is Performance Measured?...................................................................................... 8
Deciding What To Measure............................................................................................. 9
Measuring Efficiency Versus Measuring Effectiveness .............................................. 9
Establishing Benchmarks And Performance Targets .................................................... 11
II. PORT OF ENTRY PERFORMANCE: STATE FINDINGS........................................................13
State-Specific Performance Evaluations ....................................................................... 14
Comparative Measures Of Port Of Entry Performance................................................. 30
Comparisons Of Weight Enforcement Activity......................................................... 30
Weight Enforcement Measures Of Effectiveness: Weigh-in-motion Data................ 36
State Port Of Entry Performance Survey Results.......................................................... 39
Summary Of Port Of Entry Performance Measures Among States .............................. 46
III. ARIZONA PORT OF ENTRY EXISTING CONDITIONS .......................................................47
Arizona POE Goals And Initiatives............................................................................... 47
Arizona POE Current Conditions .................................................................................. 48
Operational Challenges .............................................................................................. 48
Procedural Challenges................................................................................................ 52
Summary Of Port Of Entry Activity, 1998 To 2002 ..................................................... 53
Selected Operational Measures ..................................................................................... 55
Weight Enforcement .................................................................................................. 55
Safety Inspections ...................................................................................................... 56
Service Contacts......................................................................................................... 56
Financial Measures..................................................................................................... 57
Summary Of Operational Measures........................................................................... 58
IV. ARIZONA PORT OF ENTRY PERFORMANCE MEASUREMENT .........................................59
Existing Measurements Versus Goals And Initiatives .................................................. 60
Recommended Performance Measures ......................................................................... 61
Measures of Efficiency: POE Outputs ....................................................................... 62
Measures of Effectiveness: Enforcement Outcomes.................................................. 72
Simplifying The Performance Analysis ........................................................................ 84
V. CONCLUSIONS................................................................................................................89
REFERENCES .......................................................................................................................91
APPENDIX A: PORT OF ENTRY STATISTICS........................................................................95
APPENDIX B: VEHICLE INSPECTION STANDARDS.............................................................103
APPENDIX C: STATE POE SURVEY INSTRUMENT..............................................................105
APPENDIX D: SAFETY BENEFIT CALCULATIONS ..............................................................109
APPENDIX E: WEIGHT ENFORCEMENT MEASURES OF EFFECTIVENESS. ...........................111
List of Tables
TABLE 1: MINNESOTA OUT-OF-SERVICE RATES BY ENFORCEMENT TYPE........................ 19
TABLE 2: MONTANA MOTOR CARRIER SERVICES PROGRAM INDICATORS........................ 20
TABLE 3: BUTLER COUNTY TRUCK WEIGHT ENFORCEMENT MEASURES ......................... 22
TABLE 4: SELECTED PERFORMANCE MEASURES, FLORIDA DEPARTMENT OF
TRANSPORTATION ....................................................................................................... 24
TABLE 5: MICHIGAN STATE POLICE MOTOR CARRIER DIVISION ENFORCEMENT MEASURES
................................................................................................................................... 28
TABLE 6: COMPARISON OF MEASURES OF STATE TRUCK WEIGHT ENFORCEMENT
ACTIVITY, 1997 .......................................................................................................... 33
TABLE 7: NCHRP MEASURES OF EFFECTIVENESS (M.O.E.S) FOR WEIGHT ENFORCEMENT
................................................................................................................................... 37
TABLE 8: PURPOSES OF POE/ENFORCEMENT DATA COLLECTION .................................... 40
TABLE 9: SUMMARY OF PERFORMANCE MEASURES USED BY SURVEY RESPONDENTS..... 41
TABLE 10: EXISTING MEASURES AND RESPONSIBILITIES.................................................. 43
TABLE 11: EXISTING MEASURES AND EXPECTED OUTCOMES .......................................... 44
TABLE 12: PERCEIVED UTILITY OF EXISTING PERFORMANCE MEASURES ........................ 44
TABLE 13: ARIZONA POE OPERATIONAL STATISTICS, FISCAL 1998 – 2002 .................... 54
TABLE 14: PORTS AND ENFORCEMENT SERVICES PERFORMANCE MEASURES, FY 1997... 59
TABLE 15: MOTOR VEHICLE ENFORCEMENT SERVICES PERFORMANCE MEASURES, FY
2000........................................................................................................................... 60
TABLE 16: MONTHLY SERVICE CONTACTS PER SCHEDULED HOUR OF OPERATIONS, JAN
2000 TO DEC 2000 ...................................................................................................... 65
TABLE 17: MONTHLY SERVICE CONTACTS PER SCHEDULED HOUR OF OPERATIONS, JAN
2001 TO DEC 2001 ...................................................................................................... 66
TABLE 18: PERCENTAGE OF PORT TRAFFIC WAVED THROUGH, JAN 2000 TO DEC 2000... 67
TABLE 19: PERCENTAGE OF PORT TRAFFIC WAVED THROUGH, JAN 2001 TO DEC 2001... 68
TABLE 20: INFLATION-ADJUSTED MONTHLY OPERATING COST PER SERVICE CONTACT,
JAN 2000 TO DEC 2000 ............................................................................................... 69
TABLE 21: INFLATION-ADJUSTED MONTHLY OPERATING COST PER SERVICE CONTACT, JUL
2000 TO JUN 2001....................................................................................................... 70
TABLE 22: VEHICLES PROCESSED AT PORTS OF ENTRY PER MILLION TRUCK MILES OF
TRAVEL....................................................................................................................... 71
TABLE 23: REVENUES COLLECTED PER TRUCK-MILE OF TRAVEL, FISCAL 1997 TO 2001 74
TABLE 24: PORT REVENUES COLLECTED PER DOLLAR OF OPERATING COST ................... 75
TABLE 25: SYSTEMWIDE WIM MEASUREMENTS, CALENDAR 1996 TO 1998 ................... 77
TABLE 26: NOGALES POE MONTHLY ENFORCEMENT LEVEL, 2000................................. 78
TABLE 27: WEIGHT VIOLATION SUMMARY FOR NOGALES ENFORCEMENT SCENARIOS, 2000
................................................................................................................................... 79
TABLE 28: SAMPLE PAVEMENT LIFE BENEFIT CALCULATIONS FOR PORT OF ENTRY
WEIGHT ADJUSTMENT ACTIVITY ................................................................................ 81
TABLE 29: ARIZONA MVES MONTHLY OOS SAFETY BENEFIT ....................................... 82
TABLE 30: ARIZONA PORTS OF ENTRY OUTPUT STATISTICS PER HOUR OF OPERATION ... 85
TABLE 31: PORT OF ENTRY MEASUREMENT CORRELATION, FISCAL 1998 TO 2002 ......... 86
TABLE 32: PORT OF ENTRY EXPENDITURE CORRELATION, FISCAL 1998 TO 2001............ 86
TABLE 33: ARIZONA PORT OF ENTRY LOCATIONS............................................................ 95
TABLE 34A: PORT OF ENTRY TRAFFIC, FISCAL 1997 TO 1999.......................................... 97
TABLE 34B: PORT OF ENTRY TRAFFIC, FISCAL 2000 TO 2002 .......................................... 97
TABLE 35A: VEHICLES PROCESSED BY PORT OF ENTRY, FISCAL 1997 TO 1999 ................ 98
TABLE 35B: VEHICLES PROCESSED BY PORT OF ENTRY, FISCAL 2000 TO 2002 ................ 98
TABLE 36A: VEHICLES WAVED THROUGH BY PORT OF ENTRY, FISCAL 1997 TO 1999 ..... 99
TABLE 36B: VEHICLES WAVED THROUGH BY PORT OF ENTRY, FISCAL 2000 TO 2002 ..... 99
TABLE 37: COMMERCIAL VEHICLE SAFETY INSPECTIONS, FISCAL 1997 TO 2002 .......... 100
TABLE 38: NUMBER OF VEHICLES WEIGHED, FISCAL 1997 TO 2002.............................. 100
TABLE 39: AVERAGE INBOUND TRAFFIC PER HOUR, FISCAL 1997 TO 2002 ................... 101
TABLE 40: SELECTED VIOLATION MEASURES, FISCAL 2002 .......................................... 101
List of Figures
FIGURE 1: ARIZONA PORT OF ENTRY FACILIY LOCATIONS................................................ 96
Glossary of Acronyms
ADA Arizona Department of Agriculture
ADOT Arizona Department of Transportation
ADVMT Average daily vehicle miles traveled
ATR Automated traffic recorder
CA California
CO Colorado
CVISN Commercial Vehicle Information Systems and Networks
CVO Commercial vehicle operator
CVS Commercial vehicle safety
CVSA Commercial Vehicle Safety Alliance
DOT Department of Transportation
DPS Department of Public Safety
ESAL Equivalent single axle load
FHWA Federal Highway Administration
FMCSA Federal Motor Carrier Safety Administration
FTE Full-time-equivalent
FY Fiscal year
GA Georgia
GVW Gross vehicle weight
HM Hazardous material
HURF Highway User Revenue Fund
ID Idaho
IFTA International Fuel Tax Agreement
IL Illinois
IRP International Registration Program
k 1,000
LTPP Long Term Pavement Performance
MAPS Multi-jurisdictional Automated Pre-clearance System
MCD Motor Carrier Division
MCMIS Motor Carrier Management Information System
MCTD Motor Carrier Transportation Division
MD Maryland
MDOT Michigan Department of Transportation
MN Minnesota
MnDOT Minnesota Department of Transportation
MO Missouri
MOE Measure of Effectiveness
MS Mississippi
MT Montana
MVD Motor Vehicle Division
MVES Motor Vehicle Enforcement Services
NCHRP National Cooperative Highway Research Program
No. Number
NY New York
OMC Office of Motor Carriers
OOS Out of service
OR Oregon
OW Over weight
PASS Port-of-entry Advanced Sorting System
PITWS Permanent Intermittent Truck Weigh Sites
POE Port-of-entry
PrePass Electronic credential verification vehicle clearance system
SC South Carolina
SD South Dakota
TWEET Truck Weight Enforcement Effectiveness Tool
USDOT United States Department of Transportation
VMT Vehicle miles traveled
WA Washington
WDT Weight-distance tax
WI Wisconsin
WIM Weigh-in-motion
1
SUMMARY OF KEY FINDINGS
The Arizona Port of Entry (POE) Program provides a valuable service to the residents of
Arizona, but lacks clear means of evaluating that service in terms of the efficiency and
effectiveness with which program activities are carried out. This in turn makes it more
difficult to communicate the achievements of the Program, and to identify potential
improvements in service quality. This research is intended to develop measures of
performance for evaluation of the Arizona Port of Entry program. By developing specific
measures tied to the goals and objectives of the program, Arizona POE managers will
have a better set of tools for decision making, and increased accountability to Arizona
taxpayers.
The mission of the Port of Entry program is to ensure that all commercial vehicles
operating on Arizona highways have proper credentials and are in safe operating
condition, while providing efficient, fair, and friendly treatment to port of entry
customers and residents of the state of Arizona. Measures of performance must therefore
reflect a variety of program activities: enforcement of weight and safety regulations, the
timely collection of revenues, and non-enforcement services such as permit issuance. For
the purposes of this research, port activities were grouped in the broad categories of
weight enforcement, safety inspections, and financial responsibilities. Preliminary
measures of performance were then developed for each group of responsibilities. The
preceding enforcement activities, as well as customer service functions, were also
grouped together under “service contacts,” as an overall measure of all POE activities.
Preliminary measures of performance for each group of activities were divided into two
categories according to the intent of the measurement. The first category, efficiency,
considered the actual performance of POE duties. Measures in this category quantified
the output of performance activity – the functional tasks performed by the POE staff as
compared with a target or baseline measure of performance. The purpose of this type of
measurement is to identify practices, staffing levels, locations, or other scenarios in
which the activities of the POE are performed more quickly and/or accurately or with
lower costs than the target(s). Measures of program efficiency help managers make
decisions about the direction of resources, the identification of more efficient practices,
and the results of different approaches to the agency’s objectives.
Measures of efficiency recommended for evaluating port of entry performance were
intended to be simple, but reflect the broad scope of program activities. The first three
measures comprised a cohesive unit from which more detailed program assessments
might be made.
• Service contacts per hour of operations.
• Unit cost of service contacts.
• Percentage of vehicles waved through.
• Number of trucks processed as a ratio of truck vehicle miles traveled (VMT).
2
The analysis can be done in greater detail by expanding service contacts into multiple
components. These could then be compared on a standardized basis for the program or
for each facility in order to determine the trade-off in efficiency that occurred as different
activities were emphasized. A correlation matrix of several performance measures
revealed unexpected results that emphasized trends in performance identified in the broad
analysis.
The second category of performance measure was concerned with the outcome of port
activities. These measures were intended to reflect the degree to which POE operations
influence driver and vehicle characteristics that are the focus of enforcement activities. In
other words, how well did performance of the POE mission effect external changes
among drivers and vehicles? The most meaningful goals in terms of accountability go
beyond a mere summary of program activities and define the outcomes of those activities,
that is, whether performance is improved. Potential measures of program performance
included the following, listed in order of simplicity and breadth of coverage:
• Port of entry and Highway User Revenue Fund (HURF) motor carrier revenues per
truck-mile of travel.
• Revenue collections per dollar of spending at ports of entry.
• System-wide overweight vehicle travel versus enforcement level.
• Corridor or facility-specific changes in overweight travel versus enforcement.
• Estimated pavement preservation attributable to load adjustment.
• Estimated social cost benefits attributable to safety inspections.
Many researchers prefer outcome measures because they directly relate the agency’s
strategic goals to the results of the activities undertaken to achieve them. But because the
baseline from which effectiveness is measured (e.g., the entire population of vehicles)
can not be known, the effectiveness of POE activities is much more difficult to measure.
Nonetheless, several states have made efforts to estimate the effectiveness of their POE
operations, shifting emphasis from simply output (e.g., “number of trucks weighed”) to
outcome (e.g., “reduction in overweight truck travel’’).
Over time, performance measurement should result in investment decisions that bring
about the outcomes desired by both customers and those charged with system operation
and development.
As with any operation, there will come a point at which investment of additional funds in
enforcement will yield diminishing returns. It is in the interest of taxpayers to receive the
best return on investment in port of entry enforcement. This return may be measured in
terms of productivity for various outputs (e.g., service contacts or vehicles weighed per
dollar spent) and in terms of the effectiveness of the program in inducing safe operations
and regulatory compliance.
Measures of performance should communicate the need for improvement in an
organization, but should highlight accomplishments as well. Many of the performance
measures discussed in the literature emphasized quantity of a particular unit of
3
measurement (e.g., trucks weighed), but did not relate that quantity to the operational
conditions under which it was achieved. In contrast, the measures recommended here
provide a means of relating measurements to the intended outcome of each operational
activity. Comparing revenues to truck travel, or overweight traffic to the percentage of
traffic weighed, indicates the degree to which enforcement induces compliance with state
regulations. Similarly, illustrating the benefits that accrue to highway users as a result of
port of entry services provides a means of evaluating the overall value of POE services.
Performance measures need to be redefined as the priorities of an organization change,
and special care must be taken when comparisons are made between multiple agencies or
time periods. The best assessment of the needs of the port of entry program will come
from port managers, who are most familiar with the goals and operating conditions that
affect the ports of entry. The measurements developed for this study were intended to
provide additional tools from which port of entry performance could be managed, but the
ultimate responsibility for selecting and implementing an appropriate measurement
system remains with port of entry administrators.
4
5
I. INTRODUCTION
Commercial vehicle traffic on the Arizona state highway system increased from 11.4
million average daily vehicle miles of travel (ADVMT) in calendar 1998 to 12.2 million
in calendar 2000. The increased number of heavy trucks traveling on state highways
places an additional burden on the structural integrity of the highway system, and
requires ongoing maintenance and rehabilitation of roads the trucks damage. The state
has imposed a variety of weight restrictions, weight fees and road use taxes on
commercial vehicles in an effort to compensate for these added costs to the highway
network. However, ensuring that motor carriers comply with fees and restrictions
requires ongoing enforcement activity.
In addition, the weight, operating characteristics, and often long periods of continuous
travel raise concerns about the safety of commercial vehicles and their drivers. While
commercial vehicles have generally been among the safest on the nation’s highways, as
measured by relative crash rates, the size of commercial vehicles makes it much more
likely that a crash will result in serious injury, death, or property damage. Large trucks
account for approximately 7% of all motor vehicle travel and only 3% of motor vehicles
involved in police-reported crashes. However, accidents involving large trucks account
for 12% of U.S. traffic fatalities. [State of Florida Legislature 1999] The potentially
high severity (i.e., risk of injury or fatality) of crashes involving commercial vehicles, has
prompted regulations covering such safety-related concerns as the length of time of
continuous operation and vehicle equipment standards.
Safety regulations, as well as weight and dimension regulations, play a very important
role in commercial vehicle safety in all jurisdictions. The enforcement of weights is also
in itself safety enforcement because vehicles are required to operate at a certain
maximum weight to achieve acceptable levels of stability and control. [Middleton and
Ruback 2001] But as with taxes, some carriers have an economic incentive to operate
vehicles under substandard or illegal conditions. These incentives require that state
enforcement personnel be constantly vigilant for non-compliance.
The objectives of most commercial motor carrier laws and regulations are “to keep
people safe from harm and to keep the damage to the roadways to a minimum.” [State of
Colorado 1995, 1] Impediments to the achievement of these objectives occur when
significant numbers of unsafe and overweight trucks are able to operate unchallenged.
Without effective enforcement, including the certainty of penalties and sanctions
sufficient to deter violation, weight limit laws and safety regulations become
meaningless. [Middleton and Ruback 2001] Enforcement activities have been
shown to reduce the amount of overweight traffic [Kishore and Klashinsky 2004]
and thus the premature failure of highways. Strong enforcement has also been shown to
reduce the number of highway crashes by removing unsafe drivers and vehicles from the
highways. [State of Florida Legislature 1999]
In Arizona, commercial ports of entry staffed by the Motor Vehicle Division
Enforcement Services provide the front line of enforcement for commercial vehicle
6
regulations. Staff at Arizona Ports of Entry attempt to screen all commercial traffic
entering the state of Arizona. The ports of entry check commercial vehicles for
compliance with a variety of regulations: registration, motor tax, size and weight
restrictions, commercial drivers license requirements, insurance requirements, and motor
carrier equipment safety requirements. Port enforcement officers issue permits to motor
carriers and collect the required fees, and issue citations or place vehicles and drivers out
of service when regulations are violated. The mission of the Port of Entry program is to
ensure that all commercial vehicles operating on Arizona highways maintain proper
credentials and are in safe operating condition, while providing efficient, fair, and
friendly treatment to port of entry customers and citizens of the state of Arizona.
A large volume of data is typically collected at Arizona port of entry locations. In
addition to measuring the amount of traffic passing through the port, the number of
vehicles weighed, the number of credentials verified and safety inspections performed,
many ports also provide permit services to commercial vehicles and coordinate more
specialized inspections with other agencies. These activities require the recording of
additional data, such as permit type, duration, and revenues. But the mere collection of
data does not necessarily provide the information needed to improve performance. What
is needed is a way to interpret data to evaluate organizational goals and opportunities for
development.
Performance measurement is a practice intended to provide insight into the effectiveness
and efficiency of operational programs, processes, and people. To use performance
measures effectively, an organization must do more than simply collect data. Effective
performance-based management requires that the organization decide on what indicators
it will use to measure its progress in meeting strategic goals and objectives, gather and
analyze performance data, and then use these data to drive improvements in the
organization and successfully translate strategy into action. [Office of the Vice President
of the United States of America 1997]
The Arizona Port of Entry Program provides a valuable service to the citizens of Arizona,
but lacks clear means of evaluating that service in terms of the efficiency and
effectiveness with which enforcement activities are carried out. This in turn makes it
more difficult to communicate the achievements of the Port of Entry Program, and to
identify potential improvements in service quality. This research is intended to address
the interpretation of data by developing measures of performance for evaluation of the
Arizona Port of Entry program. By developing specific measures tied to the goals and
objectives of the program, Arizona POE managers will have a better set of tools for
decision making, and increased accountability to Arizona taxpayers.
The remainder of this section addresses the concepts, procedures, and rationale for
measuring performance in general. Section II examines the methods and measures used
by other states to evaluate performance of various enforcement activities at ports of entry.
Measures used by individual states, as well as comprehensive measures developed for
comparisons between states are discussed. Section III describes the current conditions at
Arizona ports of entry, highlighting some of the operational challenges to effective
7
enforcement. Based on the review of state practices and Arizona conditions, Section IV
presents suggested performance measures for the Arizona ports of entry.
WHY MEASURE PERFORMANCE?
State transportation agencies face a growing need to align their ongoing operations with
public demands for government to become more efficient and service oriented. Funding
for transportation programs has shifted from a more reliable mix of annual grants and fuel
tax revenues to a more variable mix of grants and appropriated funds, user fees and debt
financing. At the same time, the mission of state transportation agencies has grown to
encompass not only the construction and maintenance of extensive infrastructures, but the
operation and improvement of increasingly congested transportation networks.
[Transportation Research Board 1997] Thus, like many other organizations, the
challenge to state transportation agencies is to accomplish more with less.
In many cases, they have responded to this challenge with a performance-based approach
to managing the multiple objectives and priorities inherent in a complex organization.
Public focus on accountability in the public sector has heightened awareness of
performance in government agencies. State transportation agencies have endeavored to
become more flexible and efficient, with added emphasis on the outcomes of programs
and the satisfaction of constituents and customers.
To be held accountable, an agency needs a clear understanding of its purpose and goals,
as well as ways to determine how well current methods lead to achievement of these
goals. [Kassoff 1999] Measuring performance provides managers with a framework
with which to assess current practices within the context of past successes and failures.
This emphasis on performance requires continuous monitoring of existing programs, not
only to determine the operational efficiencies and deficiencies, but also to identify new
possibilities for more effective delivery of services, and to evaluate the changing role of
outdated procedures and functions. These insights in turn provide guidance for future
strategies to improve the organization.
Osborne and Gaebler provide a succinct rationale for measuring performance in
Reinventing Government [Osborne and Gaebler 1992]:
• If you don’t measure results, you can’t tell success from failure
• If you can’t see success, you can’t reward it
• If you can’t see failure, you can’t correct it
Performance measurement, in theory, should be used as a tool to identify the
accomplishment of goals or lack thereof. It should tell the manager where things were
done correctly and where performance is not to expected levels. [Moreno et al. 2000]
Like any tool or instrument, performance measurement can be a powerful force in
bringing about positive change. However, measurement of performance can be a
complex and often controversial endeavor.
8
HOW IS PERFORMANCE MEASURED?
The use of performance measures for decision making is referred to as performance
management. Performance-based management entails selecting the most appropriate
measurements (or performance indicators) for the organization, collecting data that
reflect these indicators, and then analyzing the data to identify potential improvements
that can be made toward meeting organizational goals and objectives. The 1997 National
Performance Review [Office of the Vice President of the United States of America 1997,
6] defines performance measurement as:
“A process of assessing progress toward achieving predetermined goals, including
information on the efficiency with which resources are transformed into goods
and services (outputs), the quality of those outputs (how well they are delivered to
clients and the extent to which clients are satisfied) and outcomes (the results of a
program activity compared to its intended purpose), and the effectiveness of
government operations in terms of their specific contributions to program
objectives.”
In other words, performance measurement is a process that provides organizations with
insight into the effectiveness and efficiency of their operations. Performance measures
are quantitative or qualitative characterizations of performance. For example, they might
be indicators of work performed and/or results achieved. [Kassoff 1999] Successful
performance-based management is therefore dependent on the selection of performance
indicators that provide concrete representation of progress (or lack thereof) in meeting a
specified target level for organizational objectives.
The performance measurement process takes place in four stages: setting of goals,
development of performance measures, collection of data, and analysis and reporting of
results. [Dalton et al. 2001] While these stages might be thought of as a start-to-finish
process, each step has the potential to affect other stages both “upstream” as well
as “downstream.” Therefore the process should be considered one of continuous
feedback and, if necessary, adjustment.
The identification of goals varies by specific situations and is constrained by the
resources available to the organization. Developing performance measures and collecting
data can impact an agency’s goals in that these processes require suitably precise
definition of the intended outcome such that attainment can be measured. Performance
measures need to be redefined as the priorities of an organization change, and special
care must be taken when comparisons are made between multiple agencies or time
periods.
The National Performance Review [Office of the Vice President of the United States of
America 1997] established a set of guiding principles for performance-based
management. These include setting a narrow focus on, and specific identification of, the
processes to be measured. Measurements should be chosen that directly reflect these
processes, and should serve as a means of achieving agency goals, not as an end in
themselves.
9
DECIDING WHAT TO MEASURE
Performance measures, to be communicated clearly and to be applied effectively, should
be as straightforward as possible. The array of possible measurements makes it easy to
fall into the trap of measuring too much. A few basic, well-aligned measures are better
than a large number of complex measures. But at the same time, oversimplification of
measures can lead to applying them ineffectively and counterproductively. [Kassoff
1999] A useful way to begin the process is to ask what it is the performance measure is
intended to address, who is interested in the results, and how the results will be
used. [Office of the Vice President of the United States of America 1997]
Before deciding on specific measures, an organization should identify and understand the
processes to be measured. Each key process should be analyzed to ensure a thorough
understanding of the process and that a measure central to the success of the process is
chosen. Good measures [Office of the Vice President of the United States of America
1997]:
• Are accepted by and meaningful to constituents.
• Are representative of agency goals and objectives.
• Are simple, understandable, logical, and repeatable.
• Are clearly and unambiguously defined, with respect to purpose, data
requirements and calculation methods.
• Allow for economical data collection.
• Are timely, sensitive, and show a trend.
Reliable data, intelligently used and presented, are essential for the successful use of the
type of measures described above. [Dalton et al. 2001] The availability and
character of such data must be considered at each stage of a measure’s development and
use. Relevant and useful data can be gathered if the correct measures were set up in the
first place. [Office of the Vice President of the United States of America 1997] Data
collection should be based on a set of agreed-upon definitions to minimize dissonance
when comparisons are made. Organizations should continually assess whether their
current measures are sufficient or excessive, are proving to be useful in managing the
business, and are driving the organization to the right result. As the goals of the agency
change, so to should the priority of various measures, with emphasis added or lessened as
needed. When measures become obsolete, they should be changed or discarded.
Measuring Efficiency Versus Measuring Effectiveness
In terms of POE operations, it is useful to consider measurements that provide feedback
for two basic categories: efficiency and effectiveness. These categories will be used in
this study to classify potential measures of performance that might be useful to POE
managers.
The first, efficiency, considers the actual performance of POE functions. These measures
may be made in gross terms (e.g., the total number of vehicles weighed) or relative terms
10
(e.g., the percentage of vehicles weighed). In either case, what is being measured is the
output of performance activity – the functional tasks performed by the POE staff as
compared with a target or baseline measure of performance. The purpose of this type of
measurement is to identify practices, staffing levels, locations, or other scenarios in
which the activities of the POE are performed more quickly and/or accurately or with
lower costs than the target(s). The key question being asked is: How well does the POE
internally perform its mission?
Most states collect data related to the efficiency (outputs) of port enforcement activity.
The Federal Highway Administration (FHWA) requires that states submit data related to
weight enforcement as part of the annual certification of enforcement. [Church and
Mergel 2000] However, it has been recognized that the data submitted to the FHWA
comprise direct measures of enforcement activity, and do not reflect the effectiveness of
those activities with respect to outcome. However, measures of program efficiency help
managers make decisions about the direction of resources, the identification of more
useful or cost-effective practices, [Pickrell and Neumann 2001] and the results of
different approaches to the agency’s objectives.
The second measure of performance is concerned with the outcome of port activities.
These measures are intended to reflect the degree to which POE operations have an
influence on driver and vehicle characteristics that are the focus of enforcement activities.
In other words, how well does performance of the POE mission effect external changes
among drivers and vehicles? For example, whereas measures of POE efficiency might
consider the percentage of commercial traffic weighed, the second type of measurement
would consider how effective weight enforcement operations were in deterring
overweight vehicle travel. Similarly, the effectiveness of POE safety inspections might
be measured in terms of reduction in the unsafe vehicle population (e.g., out-of-service
violations) or reductions in associated variables (e.g., rate of truck crashes with recorded
safety violations).
The most meaningful measures go beyond a mere summary of program activities and
define the outcomes of those activities, that is, whether performance is improved.
Outcome measures are preferred by many researchers because they directly relate the
agency’s strategic goals to the results of the activities undertaken to achieve them. But
although outcome measures are generally superior, transportation agencies need to
consider data availability, cost, and validity when developing their system
measures. [Dalton et al. 2001] Because the entire population of vehicles can not be
known, the effectiveness of POE activities is a much more difficult measurement to
make. Nonetheless, several states have made efforts to estimate the effectiveness of their
POE operations, shifting emphasis from simply output (e.g., “number of trucks
weighed”) to outcome (e.g., “reduction in overweight truck travel’’). Over time,
performance measurement should result in investment decisions that bring about the
outcomes desired by both customers and those charged with system operation and
development. [Pickrell and Neumann 2001]
11
For all performance measurement activities, the “garbage in, garbage out” concept
applies to the data used. Highly uncertain data will lead to the drawing of uncertain
conclusions and will have reduced value for managing the agency. For this reason, great
care needs to be taken in data collection. In reality, however, some important things
either cannot be measured accurately or cannot be measured accurately at an acceptable
cost. [Dalton et al. 2001] Transportation agencies need to consider the uncertainty
introduced by inaccurate or incongruous data when taking action based on their system of
performance measures.
ESTABLISHING BENCHMARKS AND PERFORMANCE TARGETS
Once an organization has decided on its performance measures, the next step in the
process is to determine a baseline for each of the measures selected. In its simplest form,
a baseline can be conceived of as the first data collected on a particular
measurement. [Office of the Vice President of the United States of America 1997]
However, virtually all measures will exhibit some variance between time periods. It is
more useful to develop a performance measurement tool that measures performance
changes across time. [Moreno et al. 2000]
Determining appropriate targets for each measure after these baseline data are collected
can be accomplished in several ways. A common practice is to set goals that will force
the organization to try to exceed its past performance. In some cases, targets, minimums,
or maximums are defined for each measure. In others, a range of upper and lower
statistical limits are built around a performance target. [Office of the Vice President of the
United States of America 1997] It should be recognized that variation occurs in most
measures, and that there are both normal and special causes for such variations.
Significant changes in performance should be analyzed prior to making any changes.
Defining an acceptable or desirable level of performance can be tricky. Performance
targets (sometimes called “objectives” or “standards”) must reflect an agency’s
priorities, goals, and resources. It is best to begin with a cycle of objective measurement
to define the agency’s current position and to conduct sufficient analysis to determine
how much improvement might reasonably be expected given current or likely resource
availability before setting numerical targets or objectives. [Pickrell and Neumann
2001]
Perhaps the most important task is to establish benchmarks against which performance
can be measured. [Transportation Research Board 1997] These benchmarks must be
realistic, that is, achievable, and they must be meaningful, that is, related to decision
points. In some cases, benchmarking to the performance level of a group of peer
agencies may help an agency to initially define a reasonable or desirable level of
performance. But it is not useful to compare an agency with a group of agencies that are
not necessarily peers or if the reasons for the differences in peer scores are reported but
not well understood or explained. [Pickrell and Neumann 2001]
By benchmarking measures, an organization can validate the fact that the goals are still
attainable. For example, if peer standards have been at 80 percent customer satisfaction, a
12
goal of 100 percent may not be realistically attainable. Setting a 100 percent goal anyway
can reduce employee morale by giving them an essentially impossible target. [Office of
the Vice President of the United States of America 1997] However, this need not imply
that targets can’t be increased. If some incremental level of improvement is not possible,
the performance measure itself will likely need to be reevaluated. It may be that the time
required for effects to occur limit the agency’s ability to measure performance, and a
change in units of measurement (e.g., short-term outputs and long-term outcomes) are
required. [Pickrell and Neumann 2001] Given the need for continuing
reassessment and revision as experience is gained, the task of establishing benchmarks
and performance targets will be an ongoing process.
13
II. PORT OF ENTRY PERFORMANCE: STATE FINDINGS
This section of the report is subdivided according to the means of assessment used to
evaluate ports of entry. A variety of research literature is summarized in the first two
parts of this section. The first part reviews published performance measures and state
performance audits of specific port functions. These reviews are used to determine
whether some overlap exists both in the evaluation of ports by different states, and in
analyses using different goals or methods. Researchers have attempted to address port of
entry performance in a number of ways. In some states, organizational goals related to
POE performance have been specifically quantified by some of the agencies responsible
for port enforcement, while in others performance has been evaluated via audits
conducted by other branches of government. In the latter case, the performance of only
one POE function, (e.g., weight enforcement), has usually been evaluated.
The second part examines research efforts to synthesize POE measurements across
multiple states in order to make comparative measurements of uniform goals with respect
to efficiency and/or effectiveness. A significant part of the difficulty with measuring
performance in multifaceted operations is that the diversion of resources from one
function to another may also impact tertiary functions. The measurement of performance
at state ports of entry is complicated by the diverse enforcement functions performed at
POEs. The evaluation of one aspect of POE performance may present an incomplete or
distorted picture of the total operation. For example, many audits of weight enforcement
activities have suggested that efforts at ports of entry be shifted to mobile enforcement
crews to better capture the overweight truck population. However, these
recommendations have largely ignored the other components of POE missions, such as
safety and commodity inspections, credential verification, and the collection of taxes and
fees. Several POE administrators have raised this objection in responses to a number of
state audits of specific enforcement functions that did not consider the full range of POE
activities when recommending courses of action.
The final part documents the results of a survey of state agencies made during the course
of this research to determine what measurements were being taken at the operational
level to evaluate ports of entry. When applicable, the survey results are compared to
external measures and goals from various states identified in the preceding literature
reviews to determine whether POE managers are making similar measurements as
external studies. The survey instrument distributed to state agencies is included in
Appendix C of this report.
14
STATE-SPECIFIC PERFORMANCE EVALUATIONS
A number of performance evaluations of motor carrier enforcement functions have been
published by different states. These range among internal reviews developed by the
agency, performance audits by government auditors, and operational evaluations made by
external consultants. While states have made a variety of findings and recommendations,
most studies have been limited to a subcategory of enforcement activity such as weight
enforcement. Few studies address the multiple responsibilities of port of entry
operations. Nonetheless, these provide relevant background material for the evaluation
of the different components of POE operations. This section examines several studies,
organized by state.
Arizona
A 1986 performance audit of the Arizona Weight Enforcement program found that ports
of entry were insufficient for deterring overweight vehicles from traveling on Arizona
highways. Citing several studies, the auditors claimed that between 10 percent and 33
percent of trucks on Arizona highways were exceeding weight limits. [State of Arizona
Auditor General 1986] Furthermore, it was stated that the Motor Vehicle Division did
not place a high priority on intrastate weight enforcement activities. [State of Arizona
Auditor General 1986]
Port operations were found to be lacking in coverage, with a presence on only 13 of the
33 paved roads leading into Arizona from surrounding states and Mexico.1 Weight
enforcement was further weakened because port scales were frequently inoperative.
Thirteen percent of trucks passing through ports in fiscal 1984-85 were not weighed due
to inoperative scales, which were attributed to high port traffic volumes that exceeded the
capacity of the scales. Finally, several bypass routes were identified for different ports of
entry, with between 6 percent and 12 percent of vehicles avoiding the ports via these
alternate routes.
Although these deficiencies might have been offset by adequate mobile enforcement, the
auditors found that officers assigned to interior mobile crews spent less than 50 percent of
their time on weight enforcement. The auditors also noted regulations that required
officers to allow shifting of a load when a vehicle is only over axle weight, not over gross
limits. If the load is shifted to be within legal axle load limits, the driver can not be cited.
As a result, more than 90 percent of Arizona’s weight enforcement violations between
fiscal years 1982 and 1984 could not be cited.
While the Arizona performance audit focused exclusively on weight enforcement, this is
a significant function of ports of entry. The auditors raised important possibilities for
POE performance measurement, including the percentage of traffic that was not weighed
(output), the proportion of the total vehicle population exceeding weight limits
(outcome), and the amount of time spent on enforcement activity (output). But in the
1 Because the performance audit focused on weight enforcement, roads were only considered covered if the
port of entry had operational scales.
15
latter case, the amount of time spent performing other functions was not specifically
accounted for. It is entirely plausible that other agency functions (e.g., credential
verifications, safety inspections) required the time not devoted to weight enforcement.
This drawback will also be observed in many of the following studies; that is, sufficient
data are often not available or not considered for resource allocation among multiple
functions.
Arizona Border Region
An analysis of international ports of entry on the Arizona-Mexico border [Transcore
1997] was performed by Transcore in 1997. The study focused on operations at the
commercial and passenger ports in Nogales. Interestingly, the performance analysis was
limited to the time spent in processing by commercial and passenger vehicles. No
attempt was made to measure port performance in terms of the mandated responsibilities
of port personnel or the outcome of port enforcement activities. Based only on
processing times, the Nogales port was deemed to operate at a relatively high level of
efficiency given prevailing staff resources, infrastructure capacity, and arrival patterns.
The sole recommendation made for improving operations was to increase port capacity.
For these reasons, the results of the Nogales efficiency study were interpreted as
inadequate for the goals of this research. However, it is acknowledged that the amount of
time spent carrying out mandated functions is an acceptable measure of efficiency.
Indeed, one of the principle shortcomings identified at fixed ports of entry has been the
tendency of port traffic to back up as vehicles enter the screening area more quickly than
port officers can clear them. Although some time-based measures are more readily
calculated (e.g., number of vehicles processed per hour), the number of vehicles “waved
through” (i.e., allowed to pass without screening) due to insufficient capacity will also be
a function of immediate traffic volume. A potential measure of efficiency considered for
this research is the percentage of vehicles processed per hour, which incorporates both
processing time and traffic flow.
Georgia
The overall purpose of the Georgia audit was to identify opportunities for improvement
in the Georgia Department of Transportation’s Permits and Enforcement
Program [Hinton 2000]. The Permits and Enforcement Program is responsible for
enforcing state and federal laws governing the weight and dimension of motor vehicles
using Georgia’s roads and highways. The purpose of the Program is to protect the public
from vehicles whose weight or size exceeds safe operating limits and to protect the state’s
roads and bridges from premature deterioration and damage caused by
overweight/oversize vehicles. Due to their size and carrying capacity, multiple-axle
trucks are the primary focus of the Program’s regulatory activities.
Citing the number of enforcement crews (43) patrolling the 98,276 miles of non-interstate
highways in Georgia, the auditors concluded that as it currently functions, the Program
provides only limited assurance that the public and the state’s roads and bridges are being
16
adequately protected from damage caused by overweight and oversize vehicles. The low
probability of being caught by weight enforcement officers, as well as the small dollar
amount of weight-related fines, were considered indicative of the inadequacy of the
state’s weight enforcement program.
Performance measures considered in the performance audit were:
1. Number and percentage of citations issued.
2. Fine and permit revenues.
3. Percentage of weighed vehicles in compliance with weight limits.
4. Number of vehicles weighed per staff hour (mobile crews).
5. Total number of vehicles weighed.
Auditors suggested improving the “overall efficiency and effectiveness of the Permits
and Enforcement Program” by shifting resources from fixed scales at permanent weigh
stations to mobile enforcement crews using portable scales. This recommendation was
based on a number of findings with regard to the number and dollar amount of the
overweight citations issued by the Program:
• Portable scales had a higher violation rate: In fiscal 1999, 1.4 percent of truck traffic
was weighed on portable scales, but these vehicles received 20.5 percent of total
overweight citations
• Portable scales had a higher severity rate: the average dollar amount of the citations
resulting from portable and semi-portable scales was $123.50, or about three times
the average dollar amount of the citations generated by the fixed weigh stations
($44.90)
• Fixed weigh stations were more susceptible to shift-related inefficiencies: the number
of citations written at the weigh stations decreases substantially on Friday nights and
weekends (as well as at shift change).
For fiscal year 2000, the Georgia Program’s weight enforcement goal was to have 99.6
percent of trucks in compliance with the state’s weight limits. While this was a
reasonable measure of program effectiveness, the state auditors found that the method
used to compute this percentage overstated effectiveness and did not provide an accurate
measure of compliance. Compliance was estimated using the number of overweight
trucks cited by permanent, semi-portable and portable scales. However, statistical data
were not adjusted to account for the relatively small percentage of the truck population on
secondary roads weighed on portable scales. Revised estimates prepared by the auditors
suggested that compliance was reduced from 99.3 percent to 97.2 percent after adjusting
for the low capture rate.
In recommending more careful monitoring of the use of portable scales, the Georgia
auditors cited variance in efficiency and effectiveness of mobile weight enforcement. For
the purposes of the audit, efficiency was measured as the “number of trucks weighed per
man-day” and ranged from 5.3 to 31.1 among districts. The effectiveness of enforcement
activity was measured as the percentage of trucks issued citations, and ranged from 6.3
17
percent to 50.2 percent. Both measures were considered jointly in identifying high-performing
districts (i.e., districts with above average rates of weighing and citation).
It should be noted that the Georgia audit found an inverse relationship between measures
of efficiency and effectiveness: “…the data indicate that the number of trucks weighed
tends to decrease with an increase in the number of citations issued, [although] this
relationship is not found in every district.” [Hinton 2000, 22] Oddly, this observation
was sidestepped when making the case for shift-related inefficiencies. The lack of
citations issued on weekends and close to shift changes may have been the result of
ineffectiveness (per the audit definition), or simply a change in the traffic stream, but not
poor “efficiency” as measured in the audit.
The Georgia auditors also found that the citations for exceeding statutory weight limits
were ineffective in discouraging overweight vehicle traffic and routinely went unpaid by
many carriers with little or no consequence. Furthermore, the cost to issue and process
citations was found, in many cases, to exceed the amount of the citation. Motor carrier
program personnel estimated that the cost to issue and process overweight citations was
approximately $21 per citation. In fiscal year 1999, a total of 1,519 citations were issued
for $8 fines, or about $13 less than the cost of issuing the citation. This represented an
aggregate loss to the Georgia DOT of approximately $19,747. Similarly, Georgia Motor
Carrier Program staff estimated that current permit fees only approximate the cost of
issuing the permits. The fees did not generate any additional revenue to cover the cost of
the damage caused by overweight vehicles operating with the permits.
The Georgia Permits and Enforcement Program audit methodology was comprehensive
in terms of measures selected, and provides means of adjusting results for changing
traffic flows and staffing levels. However, the focus on citations issued as a measure of
effectiveness is unreliable: the stated purpose of the program is to protect the state’s roads
and bridges from premature deterioration and damage caused by overweight/oversize
vehicles. Because citations can only be issued when a vehicle is operating illegally, it
can be assumed that the program has not deterred cited vehicles from traveling illegally.
In other words, all other things being equal, citations should decrease if the program is
more effective.
Nonetheless, while the number of citations issued is an inadequate measure of
effectiveness in terms of program goals (reducing the number of overweight vehicles
overall), it does provide a basis for comparison of different types of enforcement (e.g.,
mobile versus fixed scales). If the vehicle population has a given percentage of weight
violators, then the percentage captured (i.e., cited) relative to the percentage screened
represents a good means of identifying the most effective enforcement procedures. The
difficulty is that this presumes a different goal, the capture of the greatest proportion of
violators, as distinct from the reduction in illegal travel. Furthermore, it is possible that
the means used to capture the largest proportion of illegal vehicles might not capture the
largest number of illegal vehicles. In sum, the Georgia audit suffered from a lack of
clarity in establishing the goals of the audit versus the goals of the program, and mixed
measures that may have been appropriate to one at the expense of the other.
18
Minnesota
An audit of the Minnesota Department of Transportation Truck Safety Inspection
Program [State of Minnesota 2004] compared program performance to the enforcement
activities of the Minnesota State Patrol. Both agencies met their 1991 roadside inspection
and safety review commitments made by the enforcement program, qualifying Minnesota
for maximum federal financing. While Minnesota's rate of detecting violations was
found to be slightly below the national average, the auditors noted that federal officials
were “pleased with the way both agencies carry out Minnesota's truck safety program.”
The auditors suggested that the data support a conclusion that the Patrol is more effective
than Minnesota Department of Transportation (MnDOT) in detecting safety violations.
In a concurrent review of the cost of conducting roadside inspections, it appeared that the
Patrol achieved certain efficiencies relative to MnDOT owing to the more extensive
statewide deployment of commercial vehicle inspectors engaged in weigh scale
operations. While this finding indicated that a reduction in travel and lodging costs
would be achieved by shifting enforcement activity from MnDOT to Patrol, the data were
considered inconclusive, and no recommendation was made.
Much of the Minnesota comparison was based on Out of Service (OOS) levels for
commercial vehicles and drivers as a result of enforcement activity. According to the
program audit, the OOS rate “reflects the skill and thoroughness with which the
inspections are conducted, as well as other factors, such as the part of the state where the
inspection occurs and the types of trucks inspected.�� [State of Minnesota 2004, 3]
Auditors noted that the Patrol and MnDOT chose locations and screening procedures that
enhanced the probability of detecting serious safety violations, and concluded that the
out-of-service rate was viable as a general measure of effectiveness.
The national OOS rates were used as a baseline for Minnesota agency performance.
Nationally, between 1984 and 1990, about 36 percent of vehicles and seven percent of
drivers inspected were taken out of service. In the early years of the program, both the
Patrol and MnDOT had vehicle out-of-service rates significantly below the national
average but this gap was considerably reduced by 1989. The Patrol was found to have
a higher vehicle OOS rate than MnDOT between 1984 and 1990. But agency
performance converged in 1991, with MnDOT achieving a vehicle OOS rate of 27
percent compared to 27.7 percent for the Patrol. Both agencies, however, lagged behind
the national average of 33 percent.
Both agencies reported driver OOS rates (the number of drivers placed out of service per
inspection) below the national norm until 1990, when the Patrol exceeded the national
rate, 8.3 percent to 7.0 percent. In 1991, the Patrol's driver OOS rate more than doubled,
to 18.9 percent. MnDOT's rate improved from 3 percent in 1990 to 3.6 percent in 1991,
still below the national average of 7.0 percent and well below the Patrol's rate.
19
Table 1: Minnesota Out-of-Service Rates by Enforcement Type
Measurement 1990 1991 Change
Driver OOS (%)
MnDOT 3.0 3.6 20.0%
Mn State Patrol 8.3 18.9 227.7%
National 7.0 7.0 0.0%
Vehicle OOS (%)
MnDOT NR 27.0 N/A
Mn State Patrol NR 27.7 N/A
National NR 33.0 N/A
Source: Minnesota Office of the Legislative Auditor, 1992. [State
of Minnesota 2004]
The drawbacks to emphasizing one set of performance measures were illustrated in the
agency responses to the Minnesota audit. The State Patrol attributed its improved
performance to increased emphasis on driver-only inspections, and emphasis on
intercepting interstate trucking on interstate highways. MnDOT pointed out that many of
its inspections were done “in the interior of the state where a greater share of the traffic
was local and thus either exempt from rules on how long a driver was allowed to drive
without resting or less likely to be in violation of them than interstate traffic.” [State of
Minnesota 2004, 3] In the first case, the limited performance measures may not have
captured the breadth of agency responsibilities, and in the second, the measurement may
not have been appropriate for the type of enforcement being done. In other words, the
Patrol emphasized activity that increased performance based on this set of measurements,
but it was not clear whether performance of other duties changed as a result. The
MnDOT response illustrated the difficulty of achieving a target when the measurement
was not aligned with the program’s emphasis.
Montana
The Montana Motor Carrier Services Program publishes performance measures related to
the Program’s goals and objectives for each fiscal year. [Montana Department of
Transportation 2004] The stated goal of the program is to protect state and federal
investment in Montana's highway system and assure the safety of the traveling public.
This goal is to be accomplished through “customer service oriented regulation of the
commercial motor carrier industry and enforcement of state and federal commercial
motor carrier laws and regulations.” [Montana Department of Transportation 2004, 1]
Program indicators for the 2002 fiscal year are shown in Table 2 below. The Montana
program uses a gross measure of (1) service and enforcement contacts (i.e., aggregate of
all enforcement-related transactions between officers and drivers), and (2) the number of
trucks weighed. Program goals for fiscal 2003 include improvement of size and weight
compliance and reduction in the number of Montana-based commercial vehicles that
have not received an annual Level 1 safety inspection (see Appendix B for definitions).
However, no specific measures were identified for these goals.
20
Table 2: Montana Motor Carrier Services Program Indicators
Fiscal Year Number of Service and
Enforcement Contacts1.
Number of Trucks
Weighed
1996 60,601 710,299
1997 54,658 657,867
1998 69,424 661,071
1999 70,500 665,000
2000 126,557 599,697
2001 72,500 719,197
Notes: 1) A contact includes issuing oversize/overweight permits, performing
commercial vehicle and driver safety inspections, issuing citations, taking
commercial vehicle fuel samples.
Source: Montana Motor Carrier Services, 2002. [Montana Department of
Transportation 2004]
While the number of trucks weighed is a fairly typical measure of program outputs
among states, the “Service and Enforcement Contacts” measure is an interesting method
of accounting for the wider variety of program responsibilities. This measurement
recognizes that a finite quantity of activity can be accomplished with a given set of
resources, and that time will necessarily be spent fulfilling these duties. But functions are
not prioritized, ostensibly due to the mandate of the agency. In other words, it makes
little sense to focus on a single type of contact if staff are required to conduct a variety of
different contacts as needed.
The Montana measures have several shortcomings. First, the published results only
measure program outputs. Second, these outputs are not defined in relation to the volume
of traffic. For example, the program showed a 1.5 percent increase in contacts and a 0.5
percent increase in number of vehicles weighed from fiscal year 1998 to fiscal year 1999.
However, this would not necessarily be considered an improvement in output if traffic
increased by three percent over the same period. Finally, the rationale for setting
performance targets is not spelled out. No mention is made of target levels of
achievement for the two measures, nor of the expected effect of an increase in one
measure on the other. Was year 2000 a trade-off of fewer trucks weighed for more
service and enforcement contacts? If so, why the drop in both for 1997? Although
changes in traffic, staffing levels, capital spending or enforcement priorities might
explain year-to-year differences, the use of gross measures precludes any firm conclusion
about program efficiency.
Colorado
The Colorado Department of Revenue completed an audit of the Colorado Port of Entry
Division in 1995 [State of Colorado 1995]. At the time of the audit, the POE Division
had operated the same 11 fixed ports since 1980 and had not conducted a comprehensive
study of its fixed port operations, locations, and traffic volume for a number of years.
21
The 1995 audit identified several areas in which port of entry performance could be
improved, and suggested that all ports be evaluated for closure based on productivity. In
this sense, the measures used by Colorado auditors were more focused on the cost of port
operations, the revenues generated (or lost) due to enforcement procedures, and the cost
of enforcement to the trucking industry. The Division has a statutory responsibility to
enforce all laws concerning commercial motor carriers and the owners and operators of
motor vehicles. Lax enforcement of certain regulations was determined to be a
significant impediment to the efficiency and effectiveness of POE operations.
The auditors recommended that the POE Division lower the costs of fixed port operations
and ensure compliance with statutes by reducing the number of fixed ports to the smallest
number needed to fulfill its regulatory activities. They also said the Division should
evaluate the productivity, traffic patterns, and enforcement activities of each and
recommend any statute change needed to allow fewer full-time fixed ports. Auditors
suggested that the POE Division might be operating some fixed ports that are no longer
productive or needed. If some of these ports were eliminated, workload could be
absorbed by existing fixed and mobile operations at lower cost. As an example, auditors
estimated that elimination of two fixed ports (Fort Garland and Platteville) could result in
cost savings of $328,000 the first year and total one-time and recurring savings of about
$1 million over five years.
According to auditors, a significant procedural shortcoming of the Colorado POE
operations was the lack of enforcement of a statutory requirement that mandates a unique
identification number on the side of each commercial motor vehicle over 16,000 pounds.
The lack of enforcement was considered costly both to the state and the trucking industry,
as trucks processed through ports despite lack of proper identification took about five
times longer to clear. Using a fiscal year 1994 estimate of 261,000 incidents when trucks
did not have the required identification, representing 15,812 unmarked individual trucks
clearing ports on an average of 16.5 times annually, the auditors calculated the time cost
to the industry as $392,000 and a loss to the state of at least $790,600 in statutory fines
that went uncollected. Further, it was estimated that about 10,900 hours, or $209,300, of
port officer time were needed to clear vehicles without proper identifications. This time,
which is equivalent to about 5.25 full-time-equivalent (FTE) employees, could have been
better spent on other activities, such as safety inspections or mobile port operations.
Another shortcoming was identified as the inadequate enforcement of sanctions against
all trucks that illegally avoided a fixed port (i.e., “port runners”). According to its
reports, the POE Division cited 55 percent of the trucks that were caught trying to
illegally bypass a fixed port in fiscal year (FY) 1994. The Division could not explain
why it issued citations to only a little more than half the port runners who were caught.
However, a port runner violation, like the unique identification citation, requires a costly
court appearance by the port officer and the driver. According to management, some of
the port officers use an inappropriate statute that does not require a court appearance to
cite port runners.
22
Finally, the predictability of enforcement activity was identified as an impediment to
effective enforcement. Fixed ports of entry were not operated 24 hours a day in both
directions, and generally followed the same operating schedule even during off-peak
enforcement. The mobile scale teams and safety inspectors did not change locations or
operate during higher-risk, off-peak hours with the frequency required by the POE
Division's regulations and policies. Citing FHWA, Colorado and Virginia researchers,
the audit found that predictable enforcement methods were inadequate for deterrence of
illegal behavior by truck operators.
The Colorado approach illustrates a useful means of evaluating agency operations in
terms of costs and revenue generation. Although specific measurements were not
specified, the audit implied use of operating costs as a means of normalizing comparisons
among ports of entry. Such a measure could be combined with gross measures of service
contacts, weighings or traffic processed to identify the most efficient port operations.
Furthermore, the Colorado audit also suggests that the outcome of particular activities
might be best evaluated in terms of savings to the state or the trucking industry.
Ohio
On a smaller scale, the Truck Weight Limit Enforcement Program of Butler County,
Ohio, has published measures of performance from 1991 to 1998 [Butler County, Ohio
2004]. The county program consists of two full-time deputies who patrol the county and
check suspect vehicles for load limit violations by utilizing portable truck scales.
Vehicles are screened for enforcement action based on visual criteria such as visible type
of load, material being dropped on the roadway, tires deformed by axle weight, and
handling characteristics of suspect vehicles.
As in the case of Montana, the Butler County program relies on just two measures. Like
the Georgia program, the measures focus on the frequency of weight citations and the
severity of the violation. In this case, severity is measured in pounds overweight rather
than the amount of the citation. Butler County program measures are shown in Table 3.
During 1991, the truck enforcement program's first year, 487 overweight vehicles were
cited for an average overweight of 12,800 pounds. Seventy eight of the citations (16
percent) were for more than 10 tons overweight. Overweight citations for January 1,
1998 through June 2, 1998 totaled 176 for an average of 10,434 pounds per truck.
Table 3: Butler County Truck Weight Enforcement Measures
Measurement 1991 1998a. 1991 – 1998
Overweight citations (number) 487 422 4,440
Average overweight (pounds) 12,800 10,434 9,972
Notes: a.) Annualized basis from January 1 to June 2, 1998.
Source: Butler County Truck Weight Limit Enforcement Program, Ohio
An assessment of efficiency can not be made due to the lack of measures of traffic, cost
or time spent on enforcement. However, the Butler County results present a mixed
23
picture of program effectiveness. Assuming a constant level of traffic and time spent on
enforcement, the number of violations appears to have increased. However, the average
severity of each violation appears to have decreased. The net effect of these changes
might be estimated by comparing the total overweight load in pounds for each year. For
1991, 487 citations at 12,800 pounds on average represented a total illegal weight of 6.2
million pounds. In 1998, after adjusting for only 5 months of data, the total illegal weight
was 10.6 million pounds. But while it appears that the program has not reduced the
number of overloaded trucks, or the total amount of weight, it is important to remember
that (1) changes in traffic volume could offset the increase on a per vehicle basis, and (2)
the nonlinear impact of vehicle weight on pavement may result in less pavement damage
in 1998 due to the reduced load per vehicle. Thus, without a clear statement of program
goals, it is difficult to draw any conclusions from the Butler County measures.
Florida
From 1999 to 2001, the State of Florida published two performance audits and a set of
operating standards for the Florida Motor Carrier Compliance Program [State of Florida
2001; State of Florida Legislature 2001; State of Florida Legislature 1999]. The
primary purposes of the program are to protect highway system pavement and structures
from excessive damage due to overweight and oversize vehicles and to reduce the
number and severity of crashes involving commercial vehicles. Specified Motor Carrier
Compliance Program objectives are to reduce occurrences of overweight commercial
motor vehicles and eliminate hazards caused by defective or unsafe commercial motor
vehicles.
Inspectors weigh trucks and check registration and fuel tax compliance at fixed scale
locations along major highways. The program’s law enforcement officers patrol the
state’s highways and use portable scales to weigh trucks that do not pass through fixed
scale stations. Officers also enforce commercial motor vehicle safety regulations by
performing safety inspections and enforcing traffic laws. Commercial vehicle safety
inspections include examination of vehicle parts such as brakes, lights, and safety
equipment and, if carried onboard, the packaging and labeling of hazardous materials.
Officers also determine whether commercial drivers are appropriately licensed, have
maintained required logbooks of their hours of service, and are operating their vehicles in
a safe manner (e.g., not speeding or operating under the influence of drugs or alcohol).
Citing driver fatigue as one of the top commercial motor vehicle safety concerns, with
commercial vehicle crashes more likely to be caused by driver error than by mechanical
failure [State of Florida Legislature 2001], the Florida audits placed a relatively high
importance on the safety-related enforcement procedures.
Agency performance in Florida is measured in terms of both program outputs and
program outcomes. For 1999, measures of output were defined as the number of vehicles
weighed (fixed and mobile scales) and number of safety inspections performed.
Outcome measures were defined as the percentages of trucks that were found overweight
on fixed scales and on mobile scales. Program staff did not report outcome measures for
safety enforcement, due to problems of definition [State of Florida Legislature 1999].
24
Performance measures for the Florida Highway Operations and Motor Carrier
Compliance programs are shown in Table 4.
Table 4: Selected Performance Measures, Florida Department of Transportation
Jurisdiction and Performance Measure Type of Measure FY 2001-2002
Standard
FDOT Highway Operations Program
Maintenance condition3 of state highways Outcome 80
Percent of fixed scale weighings overweight Outcome 0.3%
Percent of portable scale weighings overweight Outcome 44%
Number of commercial vehicles weighed Output 11,000,000
Number of CVS inspections performed Output 50,000
Number of portable scale weighings performed Output 35,000
Highway Safety and Motor Vehicles: Motor Carrier Compliance
Ratio of IRP1. and IFTA2. taxes collected to
cost of collection Outcome 1.75 : 1
Number of IFTA Use Tax and IRP Plans
audited Output 309
Number of Motor Carriers audited per auditor4. Output 22 : 14
Notes: 1) International Registration Program. 2) International Fuel Tax Agreement. 3) Measurement based
on internal standard of condition. 4) Second number represents the total number of auditors.
Source: State of Florida. Department of Transportation and Department of Highway
Safety and Motor Vehicles (2001). Approved Agency Performance Measures and
Standards for Fiscal Year 2001-2002.
In the 1999 review, auditors highlighted several deficiencies in the state’s performance
measures for motor carrier enforcement. Differences between mobile and fixed scale
(i.e., port of entry) outcomes were considered ambiguous, not clearly reflecting the level
of overweight traffic on state highways. Auditors recommended using weigh-in-motion
(WIM) station measurements to evaluate the outcome of weight enforcement, and
suggested several measures related to safety. These were the number of safety
inspections performed (output), the percentage of safety inspections resulting in driver
and/or vehicle being placed out-of-service (outcome), and the number of crashes caused
by commercial vehicles (outcome).
However, this last measure was questioned by the Motor Carrier Compliance Office. In a
rebuttal, management stated that, since the program has “limited resources and a limited
enforcement role,” it would be misleading to establish a measure of the number of
crashes caused by commercial motor vehicles or drivers. They noted that many things
outside the control of the program influence the number of crashes.
The Florida measures provide a balance between gross and proportional measures of
performance, with the former generally representative of operational efficiency, and the
latter targeting the outcome of enforcement. However, as with various other states,
measures of output are not controlled for staffing levels or port traffic. Therefore, any
25
increases in output are interpreted as improvement, despite the possibility that a smaller
percentage of vehicles were screened or the results were achieved at a greater unit cost.
As a measure of outcome, the percentage of vehicles overweight appears to be a
misleading measure. If this percentage keeps increasing, it suggests that weight
enforcement activities are not deterring illegal vehicles; in fact, the opposite could be
said. However, it was not clear from the audit whether an increase or a decrease in this
percentage was considered an improvement.
The ratio of taxes collected to the cost of collection might better describe the efficiency
(i.e., output) of collection rather than its effectiveness. But as a measure of efficiency,
this measure provides a useful snapshot of the return on tax enforcement. Furthermore, a
similar measure might be expanded to represent a benefit-cost ratio of enforcement
activity. However, this type of analysis would likely be complicated and controversial, as
savings benefits to one group of constituents might be construed as costs for another
group.2
Oregon
A January, 2002 state audit of Oregon Motor Carrier Transportation Division [Oregon
Office of the Secretary of State 2002] focused on the division’s weight enforcement
responsibilities. The purpose of the audit was to determine if the division was deploying
its resources in the most cost-effective manner to protect roads and bridges from damage
by overweight trucks. Truck weight enforcement activities at 87 permanent scales, as
well as several mobile enforcement units, were evaluated based on a variety of criteria.
Auditors reviewed the following measures of performance:
• Number of trucks weighed at different locations.
• Percentage of weighings to total through traffic.
• Weight violation rates factored by the percentage of weighings and hours of
operation.
• Proportion of trucks required to off-load as a measure of overload severity.
The auditors also undertook a comparison of inbound and outbound weight enforcement
with site-specific WIM measurements to illustrate the impact of non-enforcement on
loads.
The review pointed out a common problem among port of entry operations: that despite
the large amounts of data collected at scale sites, the division had not developed specific
2 For example, the Colorado audit (page 19) identified a time/ cost savings to the trucking industry of
$392,000 if enforcement standards were improved. However the same procedural change was estimated to
generate an additional $790,000 or more in tax revenue to the state. These state revenues would be paid by
the trucking industry. Therefore, from the trucking industry perspective, the change in procedures would
result in a net loss of $398,000. Making things more complicated, it could be argued that the position of
the industry as a whole would be improved because the costs would be borne by illegal operators who
previously enjoyed an unfair competitive advantage. This example simply illustrates the potential for
problems that can arise when enforcement revenues are considered measures of performance.
26
goals related to those data. The auditors recommended several outcome-specific
measurements for future performance evaluations: monitoring the percentage of
overweight trucks on the highway (presumably using WIM, although this was not
specifically mentioned), the severity of overweight violations, average axle weights, and
excess axle weights. However, these procedures did not appear to be used in the current
audit.
The Oregon auditors concluded that the resources devoted to POE operations could be
more effectively deployed at other sites around the state. Allocation of additional staff
resources was specifically recommended for mobile enforcement on secondary state
highways. More variable hours at POEs were also recommended, in order to reduce the
predictability of enforcement efforts. Finally, the use of technology (e.g., portable WIM)
was recommended to identify secondary routes that should be targeted for increased
enforcement.
While generally receptive to the recommendations, the Oregon Motor Carrier
Transportation Division (MCTD) pointed out shortcomings within the audit. The most
significant limitation of the audit results was the consideration of only one function for
which the division was responsible. While the re-deployment of staff from fixed POE to
mobile units might augment weight enforcement efforts, such a move would diminish the
capacity for credential and safety inspections, as well as fee collections related to state
motor carrier taxes. This is a common shortcoming among the performance audits for a
number of states.
The division response also pointed out the problem inherent in using violation rates to
measure performance. Whereas the auditors suggested that higher violation rates at light
enforcement locations were an indication that additional resources should be deployed to
these sites, the motor carrier division countered that the lower violation rates at heavy
enforcement locations were a direct result (i.e., success) of the additional enforcement.
This observation is in direct contrast to the measures identified by other states (e.g., Ohio,
Florida), but is a valid criticism of the use of overweight ratios as a positive measure.
The auditors recommended a shift in resources from ports of entry to mobile operations
based on the success of the latter in capturing overweight vehicles. However, the audit
did not consider the impact of the volume of transactions as a deterrent, not only for
weight enforcement, but also in terms of tax, safety, and other regulatory avoidance. In
addition, the cost differential between fixed location weighings ($0.48 to $2.24 per
weighing) and mobile enforcement ($22.40 per weighing) was identified by MCTD staff
as a limiting factor for allocation of spending.3
3 Although specific measurements were not published in the audit, a potential alternative to the cost per
truck weighed might be the cost per overweight truck identified. This would isolate the relative agency
cost of identifying overweight vehicles based on traffic volume (ports) versus selective enforcement
(mobile). However, such a measure would still be problematic in that a higher frequency of overweights
would be construed as a positive change in the overall measurement.
27
Michigan
A year 2000 legislative report on commercial vehicle enforcement activities of the
Michigan Department of Transportation (MDOT) [State of Michigan 2000] was
followed by a state audit of the Michigan State Police Motor Carrier Division [Michigan
Auditor General 2001] in 2001. MDOT and State Police Motor Carrier Division (MCD)
conducted a comprehensive analysis of relevant issues related to truck law enforcement
to develop cost effective strategies to improve enforcement of truck size and weight laws
and enhance enforcement of truck laws in general. These two reports provide a
framework [State of Michigan 2000] and performance analysis [Michigan Auditor
General 2001] for the development of enforcement-related goals and activities across
multiple agencies.
The Weight Enforcement and Safety Inspection Implementation Plan prepared by MDOT
and MCD, approved by the MDOT Highway Steering Committee in May 1992, estimated
that overweight vehicles cause over $54 million worth of damage to Michigan’s federal-aid
highways annually. Pilot studies from 1997 to 1999, indicated that enforcement
could be improved by placing greater emphasis on mobile enforcement and less emphasis
on scale house enforcement at interior weigh stations. The added emphasis on road
patrol was suggested to lessen the predictability of enforcement and expand enforcement
coverage area, thereby improving enforcement efficiency and effectiveness.
The State Police MCD and MDOT identified six interior weigh stations to consider for
conversion to road based patrol operations use only, with no traditional scale house
operations. MCD began investigating methods for improving efficiency in road patrol
truck weighing procedures such as carrying additional portable scales and weighing one
side of the truck at a time. MDOT funded a research effort for design and construction of
Permanent Intermittent Truck Weigh Sites (PITWS) for use with portable scales. Finally,
both agencies advocated increased usage of WIM sensors to screen truck weights and
plan enforcement action.
To evaluate the utility of these recommendations and “assess the effectiveness and
efficiency of MCD in meeting its mission to provide the public with a safe motoring
environment and protect the highway infrastructure by promoting compliance with
commercial vehicle laws through education and enforcement,” [Michigan Auditor
General 2001, 10] Michigan conducted a performance audit of the State Police Motor
Carrier Division.
The audit examined program activity data and methodology for assigning weigh station,
road patrol, and Specialized Transportation Enforcement Team staff. Auditors reviewed
weighing and inspection activities at the weigh stations for three fiscal years. The
measures of performance used to compare port of entry and road patrol operations are
shown in Table 5.
28
Table 5: Michigan State Police Motor Carrier Division Enforcement Measures
Enforcement Type and Measures Fiscal Year 1996-97 1997-98 1998-99
Weigh Stations
Total Vehicles Weighed 3,268,424 2,867,892 2,337,649
Overweight Violations 1,438 1,309 1,545
Total Violations 8,056 7,752 8,176
Road Patrol
Total Vehicles Stopped 37,249 30,809 32,349
Total Vehicles Weighed 3,969 3,904 3,638
Overweight Violations 1,857 1,778 1,873
Total Violations 22,842 19,490 22,278
Source: Michigan State Police Motor Carrier Division, 2001. [Michigan Auditor
General 2001]
The Michigan auditors compared the measures shown in Table 5 to the output from 21
WIM sensors on Michigan highways.4 Weigh-in-motion data from June 2000 identified
181,000 trucks with 6 or more axles, of which 69,000 (38 percent) were overweight. The
percentage of overweight trucks was considerably lower for those with 5 axles and
below. In comparison, auditors noted that MCD issued a total of 361 citations for trucks
being overweight during the same month. Of these, 140 were issued from the permanent
weigh stations and 221 by road patrol cars.
The 361 citations issued by MCD represented 0.2 percent of the total truck traffic as
measured by the WIM units. However, the implication that enforcement activity was
lacking because WIM identified a greater percentage of overweight vehicles is
misleading. First, the auditors did not identify the number of vehicles passing through
weigh stations or mobile enforcement sites. The violation rates for the fiscal years shown
in Table 5 were 0.2 – 0.3 percent at fixed ports and 5.0 – 5.8 percent for mobile patrols,
suggesting that all 181,000 vehicles did not pass through enforcement areas in the June
2000 sample. Second, the threshold for issuing citations was not specified. It is common
practice among weight enforcement officers, particularly at high volume locations, to
allow vehicles “slightly overweight” to pass without being cited.5 Third, the margin of
error cited for WIM measurements was not used to develop multiple estimates of the
percentage of weight violators.
The State Police MCD responded that it had attempted to utilize mainline WIM sites to
detect overweight vehicles, but those attempts had minimal success because the WIM
sites were frequently non-operational and, when operational, WIM equipment were not
accurate or reliable. Manufacturer claims of accuracy rates of 80 percent and 90 – 95
4 Auditors noted that the accuracy of WIM systems varied by type and installation, ranging from 80 percent
to 90 – 95 percent for the units installed by the Michigan Department of Transportation.
5 In Arizona, the fine for gross weight violations under 1,000 pounds is $1, an amount below the cost of
issuing the citation. Michigan does not fine for gross weight violations, so there may be some leeway for
axle weight violations as well.
29
percent accuracy were dependent upon proper installation, maintenance, and calibration.
It should be noted that the criteria for “success” were not defined by the State Police.
The auditors identified several additional shortcomings with the MCD enforcement
program. First, the MCD had not established specific goals and objectives with
quantified outcomes for motor carrier size and weight enforcement and hazardous
materials inspections and follow-up. This again raises the suspicion that the resistance to
WIM-based enforcement was not based on concrete measures of success. Second, MCD
had not developed an information system to gather output and outcome data. For
example, MCD did not accumulate data from its PITWSs to evaluate effectiveness and
efficiency. Although the use of PITWSs reduced the time it takes to weigh a vehicle,
MCD did not determine whether there was a corresponding increase in weight
enforcement effectiveness and efficiency. Last, MCD had not conducted a comparison of
actual outcome data with desired outcomes. For example, MCD scheduled weekend
enforcement coverage at a lower level than weekday coverage, but had not determined
whether there was a relationship between limited enforcement coverage on weekends and
the number of overweight vehicles or other traffic enforcement on the highways during
weekends.
The Michigan State Police audit was unique in that customers of the Motor Carrier
Division were invited to participate in the performance evaluation. Mail surveys were
distributed to commercial carriers soliciting feedback about the division’s activities. The
survey results suggested that at least some carriers perceived enforcement activities as a
deterrent to illegal travel.
Of the 55 respondents to the commercial carrier survey, eight (14.5 percent) indicated
that at some point during travel on Michigan highways they had been found to be
overweight on certain axles and required to adjust the load before being allowed to
proceed. Six (10.9 percent) indicated that they had been found to be overweight on
certain axles but were not required to adjust their load before being allowed to proceed.
Interestingly, while 14 respondents answered affirmatively to the two previous questions
regarding load shifting, when asked how often a citation had been issued when their load
required adjustment, 29 respondents (52.7 percent) responded to the question. This raises
the possibility that, either the questions were not worded properly in order to exclude
non-violators, or some overweight operators were reluctant to provide truthful responses
to previous questions.
When asked a hypothetical question regarding the probability of detection, 14
respondents (25.5 percent) thought it “unlikely” or “very unlikely” that they would be
detected if traveling overweight in Michigan. Fifteen respondents (27.3 percent)
answered that detection was “likely�� or “very likely” and 21 respondents were neutral.
Five respondents didn’t answer.
In a follow-up question to the probability of detection, respondents were asked to choose
the type(s) of enforcement activity most likely to detect overweight violators. The most
30
frequently chosen activity was highway patrol cars (32), followed by permanent weigh
stations (20) and temporary weigh stations (10). These results suggest that the perceived
enforcement “threat” posed to illegal operators by mobile weight enforcement crews (i.e.,
temporary scales) is of less consequence than the benefit frequently cited in performance
audits.
The use of weigh-in-motion data for evaluation of outcomes and planning enforcement
activity has been the cause of some controversy between researchers and enforcement
agencies. While the theoretical benefits of WIM measurements are undisputed, the
Michigan State Police response illustrates the common complaint that such systems are
far less reliable in practice. This problem is exacerbated by the lack of attention to WIM
measurement variance when preparing estimates. For example, in the case of a perfectly
calibrated WIM sensor with 88 percent accuracy for axle weight measurements, an axle
10 percent overweight stands a 20 percent chance of registering as legal [Bergan et
al. 1998]. However, WIM can be interpreted conservatively as a broad measure of
existing conditions. No other existing measurement provides as comprehensive a picture
of traffic characteristics and weights. Further discussion of WIM measurements as
indicators of effectiveness is included in the next section.
COMPARATIVE MEASURES OF PORT OF ENTRY PERFORMANCE
Several large-scale studies have been conducted at the national level in order to compare
the efforts and achievements of multiple states. As in the case of state performance
audits, these studies have tended to focus on the weight enforcement function carried out
at state ports of entry. Two of the most comprehensive are discussed at length in the
following section. The first examines the level of enforcement activity among nine states
and makes comparisons based on the output (i.e., efficiency) of enforcement programs.
In contrast, the second details the development of measures of effectiveness for assessing
the outcomes (i.e., effectiveness) of weight enforcement activity. The use and validity of
various measures of performance are compared among four states from disparate
geographical areas.
Comparisons of Weight Enforcement Activity
An analysis of the effectiveness of violator penalties for ensuring compliance with truck
weight limits was conducted for the US Department of Transportation Special Programs
Administration in September, 2000 [Church and Mergel 2000]. The approach used
for this study was to conduct discussions with enforcement officials in nine states,
diversified by geography, fine severity, roadside enforcement practice and adjudication
system, on whether their penalty imposition was considered to be effective. The results
of these discussions included measures of weight enforcement activity for the states
surveyed, as shown in Table 6.
The states studied for purposes of this report were: California (CA), Georgia (GA),
Minnesota (MN), Mississippi (MS), Missouri (MO), Montana (MT), New York (NY),
31
South Dakota (SD), and Washington (WA). Researchers chose this group to represent
states with various operational, statutory and procedural differences. These included
three states using the PrePass electronic pre-clearance program (CA, MS and MT) plus
two from NorPass (WA, originally from its MAPS component, and GA, originally from
the Advantage I-75 corridor program); the original "shipper liability" state (MN); a state
that no longer uses fixed weigh stations at all (NY); plus at least one relatively high-fine
(SD) and one relatively low-fine state (either MT or GA) and some geographical
dispersion.
The basic question of whether state penalties were satisfactorily inducing operator
compliance could not be answered definitively based on available data. However, the
state authorities generally indicated that there are persistent compliance problems on
secondary roads and in local bulk trucking. This suggested the potential value of
enforcement efforts targeted at these sectors, which were not subject to economical
surveillance by permanent, fixed-site weigh stations. The researchers noted that a
promising approach appeared to be expansion of the practice of analyzing data
from "non-enforcement" weigh-in-motion equipment so as to efficiently deploy available
mobile truck weight enforcement personnel. This practice was noted as already being
underway in three of the nine study states.
Table 6 shows a comparison of parameters of the weight enforcement activity carried out
in these nine states as reported to FHWA for 1997. In order to compensate for size
differences between states, enforcement activity measures were normalized by estimated
total heavy truck mileage on major rural roads6 within each state, and also by the mileage
of these roads. While acknowledging the large breadth of this estimate as single
normalizing factor, with some states having a larger proportion of their road network
within urbanized areas than do others, the rural roads were justified as the locations for
which states typically had the most opportunity to detain certain trucks without creating
an unsafe condition for, or grossly delaying, other traffic.
According to the researchers, there was no available measurement of weight limit
compliance sufficiently comprehensive to permit determination of actual penalty
effectiveness within different states. In other words, the available measures were
insufficient for evaluating the outcome of penalty enforcement. However, the 1997 data
in Table 6 show some distinct differences between the Study states in the pattern of their
enforcement practices and in the extent of enforcement in relation to size.
Surveillance of truck weights, as indicated by their reported total of static and WIM
screening weighings, varied by a maximum factor of about four among all but one of the
study states when that total was expressed as a relationship to their major rural road truck
traffic. The exception was New York, one of three study states that employed no WIM
screening at all in 1997, where surveillance was vastly lower than in the rest of the group.
Mississippi was highest, followed closely by Georgia. Georgia also had the highest
citation rate in relation to major rural road truck traffic.
6 Interstate and other arterial roads outside of urban areas.
32
New York reported a much higher total number of citations in relation to total weighings
(4.6 percent) than did other states. All other study states showed 1997 citation-to-total-weighing
rates under 1 percent, with Georgia the highest (0.9 percent) despite its very
high volume of WIM screening weighings. However, over 30 percent of the reported
1997 citations, but fewer than 2 percent of the weighings, were generated by the
authorities of the two counties located on Long Island and of New York City, rather than
by the State Police, which performs weight enforcement in the rest of New York. New
York City, which has its own weight limit regime, alone accounted for over a quarter of
the reported citations, which actually outnumbered weighings due to repeat issuance for
multiple types of weight violations by the same vehicle, a practice not generally followed
by the New York State Police. The ratio of citations to weighings for the New York State
Police was 3.2 percent. Given the exclusive use in New York of semi-portable or
completely portable scales, this ratio was expected to be higher than that in other states.
The deployment of mobile scales may be easily altered so as to concentrate on sites or
areas where there are thought most likely to be actual violations, and the lower
throughput capacity of mobile scales encourages the exclusion of empty trucks, and
concentration on the most likely potential violators among loaded trucks.
33
Table 6: Comparison of Measures of State Truck Weight Enforcement Activity, 1997
Measures of Weight
Enforcement Activity
California Georgia Minnesota Mississippi Missouri Montana New York South
Dakota
Washington National
Average 1
Fines/ 10k GVW violation $1,500 $318 $715 $1,000 $1,000 $250 $700 $2,625 $890 $726
Total Enforcement Activity
Static Weighings 12,260,295 1,768,909 701,898 5,684,389 2,756,503 858,158 167,468 592,123 2,362,044 2,079,566
WIM Screenings 4,187,162 11,787,811 950,000 2,569,819 0 20,116 0 0 1,300,000 1,278,133
Citations 44,777 122,901 3,438 13,900 8,799 1,846 7,757 3,349 11,433 12,469
Load Shifts / Off-loadings 46,368 7,876 640 21,660 13,239 14,703 149 2,109 13,797 10,329
Structure of Enforcement Activity
Citations per Static or
WIM Weighing
.003 .009 .002 .002 .003 .002 .046 .006 .003 .004
Ratio of Load Shifts or
Off-loads to Citations
1.04 0.06 0.19 1.56 1.50 7.96 0.02 0.63 1.21 0.83
Percent of Static
Weighings on Fixed
Scales
99.94 77.88 90.07 99.46 99.88 99.39 0 93.84 98.24 98.33
Ratio of WIM Screenings
to Static Weighings
0.32 6.66 1.35 0.45 0 0.02 0 0 0.55 0.61
Enforcement Activity Compared to Estimated Truck Mileage on Major Rural Roads2.
VMT Per No. of Static or
WIM Weighings
339 304 816 284 1,136 908 13,458 1,161 342 1,105
VMT Per No. of Citations 122,618 33,542 392,256 168,513 355,875 432,161 290,543 205,235 109,436 153,601
VMT Per No. of Load
Shifts or Off-loadings
118,411 523,408 2,107,148 108,141 236,524 54,259 15,125,779 325,904 90,685 185,425
Enforcement Activity Compared to Mileage of Major Rural Roads2.
Static or WIM Weighings
Per Road Mile
1,354.6 1,477.7 155.2 1,309.2 379.8 130.2 24.2 91.0 802.7 648.9
Citations Per Road Mile 3.8 13.4 0.3 2.2 1.2 0.3 1.1 0.5 2.5 2.4
Load Shifts or Off-loadings
Per Road Mile
3.2 0.5 0.02 2.4 1.4 1.7 0.02 0.3 2.7 1.4
Sources: (1) Enforcement activity, Federal Highway Administration records (2) Vehicle and road miles - Federal Highway Administration statistics. Definitions:
“Major” rural roads defined as the functional classes Rural Interstate, Rural Other Principal Arterial, and Rural Minor Arterial. Total length of such roads taken
from Table HM-20 of Federal Highway Administration’s Highway Statistics.
33
34
New York and Georgia (and to a lesser extent, Minnesota) required load adjustments
much less frequently than the other study states in relation to the number of citations
issued. Montana, by contrast, required them much more frequently. Variation among
states’ level of truck weight surveillance was greater when total weighings were
expressed as a relationship to major rural road mileage than when expressed as a
relationship to truck traffic. Again, New York was by far the lowest, followed by South
Dakota, which also reported no WIM screening at all.
Only one state (Montana) expressed the firm opinion that penalizing weight limit
violators was having a significant positive impact on general trucker compliance
behavior. In New York, the establishment of a graduated penalty schedule was believed
to have been followed by at least one industry shifting from what appeared to be virtually
universal non-compliance up to an informally-estimated 90% compliance rate. But this
favorable change in local bulk trucking compliance was also attributed in part to the
concurrent establishment of a general annual permit system allowing axle, axle spacing
and gross vehicle weights significantly above federal Interstate Highway standard limits.
In Minnesota, it was thought that general compliance behavior had improved over the
long period since introduction of their “relevant evidence” enforcement system, during
which enforcement surveillance had also been increased and there had been somewhat
more rigorous prosecution of violations in court.
Four other states offered contrasting views of the effectiveness of penalties. In
Washington, the FHWA was told that a recent fine increase had not been accompanied by
discernible improvement in general compliance. In South Dakota, a campaign to raise
already-high fines and legislate greater enforcement powers implied past ineffectiveness
of violator penalties to generate an acceptable level of compliance. This was attributed to
general inattention on the part of truckers to weight requirements, as well as some
acceptance of fines as an “expected cost of doing business.” Missouri’s enforcement
effort was thought to have little effect on compliance in certain sectors of short-haul,
secondary-road trucking, principally because of a low apprehension rate (i.e., probable
penalty) relative to the potential additional earnings available from an overload. A
similar observation was offered by Georgia, where the civil penalty being employed for
overweight offenses was believed to be less effective in the local bulk than in the long-distance
general trucking sector, with the possibility that in the former some intentional
overloading was occurring.
Representatives of the two other States (California and Mississippi) did not have what
they considered to be an adequate basis for evaluating the impact of the enforcement and
penalty system on compliance behavior. However, California representatives allowed the
possibility that their state’s extensive network of permanent weight and safety inspection
stations, many of which are kept open continuously, were deterring some potential
violators.
The problems associated with using the percentage of weight violations as a measure of
compliance, as noted in the previous section, were specifically identified by this study.
Researchers pointed out that the rates at which overloads are detected at fixed-site weigh
35
stations, especially those located only on major through routes or open only at certain
times of the day, were inadequate as a measure of overall weight limit compliance.
These measures at fixed ports typically overstate compliance, whereas detection rates
from mobile weight enforcement units could understate compliance because of the units’
targeting trucks with high violation potential. Citation-to-weighing rates that were
volunteered by five study states for their deployed portable scales varied from 3 percent
to over 58 percent, such vast differences presumably being due both to differences in
actual violation rates in the deployment areas chosen and in the extent to which only
likely violators were being selected for weighing. A representative from Missouri
hypothesized that the compliance rate among trucks on secondary roads carrying two
problem commodities (grain and gravel) might be in the 10-20 percent range.
Representatives from two states volunteered informal, unofficial estimates of overall
weight limit compliance within the whole state. Montana estimated 85 percent
compliance, based on general observation, and California estimated 94-95 percent
compliance based on data output from weigh-in-motion installations primarily used for
highway planning. The enforcement authorities surveyed typically viewed secondary
roads and local bulk trucking as the sectors where their state’s violator detection and
penalization system had an insufficient effect. Local trucking was less likely than
interstate/interregional trucking to be exposed to surveillance by high-volume weigh
stations set up at fixed sites to intercept a state’s major truck traffic flows (often for
purposes of simultaneously carrying out safety and tax/registration document checks).
Also, to the extent that the rate of citation for serious overweights is greater for local
truckers when subjected to enforcement, they presumably benefit more from any
significant reduction of overweight fines during adjudication by local criminal courts,
which was cited as an enforcement problem by some authorities.
Despite the lack of reliable data for measuring the outcome of enforcement and
associated penalties, the comparisons yielded some potentially useful measures for
evaluating productivity. The use of vehicle miles of travel, in particular, eliminates the
variance that occurs when port traffic flows are considered as a normalizing factor for
traffic. Although VMT estimates are in themselves subject to considerable variance, the
temporary closure of a port (and thus non-measurement of traffic) would not enhance
traffic-based measures by reducing the denominator of the equation.7 Ratios of WIM
screenings to static weighings, and load shifts to citations, could be used to respectively
illustrate the effects of technology improvements on traffic processing and the relative
severity of penalties assessed at different locations. However, these measures would not
be indicators of overall productivity, and would be best used in conjunction with other
measures to explain or test variations in practices.
7 If the measure in question is the ratio of trucks weighed to port traffic, performance (efficiency) could be
artificially enhanced by closing ports periodically and thus reducing traffic. Such a practice could
especially skew results at peak operating times when a port is more subject to backlogs and forced wave-throughs.
Estimated vehicle miles of travel, though variable, are not within the influence of POE staff.
36
Weight Enforcement Measures of Effectiveness: Weigh-in-motion Data
In a 1998 report for the National Cooperative Highway Research Program (NCHRP)
[Hanscom 1998a], a different approach to performance measurement
was taken by researchers. Rather than focus on the outputs achieved by state
enforcement programs, the NCHRP study focused exclusively on measuring the outcome
of agency activities. The rationale for this approach was based on the following goals of
truck weight enforcement activities:
• Deter operation of overweight trucks and/or trucks with inappropriate axle spacing.
• Control pavement and bridge damage from overweight trucks.
• Protect the public from safety risks associated with overweight trucks.
• Protect law-abiding truck operators from illegal competition.
The authors noted that benefits of weight enforcement activity “must be recognized in
terms of some, or all, of these objectives.” [Hanscom 1998a, 4] In other
words, a study to evaluate the outcome of truck weight enforcement must be based on
measures that reflect goals of the weight enforcement program, such as changes in
compliance, (e.g., instances and severity of overweight violations), and whether any
enforcement benefit is achieved in terms of reduced pavement wear. A Measure of
Effectiveness (MOE) of weight enforcement activity was defined as a “determinable
quantity of what is achieved as a result of weight enforcement activity,” used to quantify
the contribution that a particular activity makes toward achievement of one or more of the
weight enforcement goals described above.
Using weigh-in-motion data, several MOEs were developed and tested in four states, in
order to determine the statistical validity of each and to make comparisons among states
in terms of the outcome of enforcement activity. The four states used in the study were
California, Georgia, Idaho and Minnesota. The measures of effectiveness and their
definitions are shown in Table 7.
Sampling guidelines were developed to estimate the number of observation sites and
truck sample sizes required for valid measurement of enforcement effects. These
guidelines were provided for specified roadway classification and truck percentage
conditions. Separate observation levels for sampling truck-weight violations were
devised in order to meet the varied types of truck weight enforcement operations: (1)
statewide or regional, (2) highway corridor or local level, and (3) spot or location-specific.
37
Table 7: NCHRP Measures of Effectiveness (M.O.E.s) for Weight Enforcement
Measure Type Definition
Gross Weight
Violation
Proportion The fraction (or percentage) of the total observed truck
sample that exceeds the legal gross weight limit.
Gross Weight
Violation
Severity The extent to which average measured gross weights for
the observed sub-sample of gross weight violators
exceeds the legal gross weight limit.
Single-axle
Weight Violation
Proportion The fraction (or percentage) of the total observed truck
sample with one or more axles that exceeds the legal
single-axle weight limit.
Single-axle
Weight Violation
Severity The extent to which average measured single-axle
weights for the observed sub-sample of single-axle
weight violators exceeds the applicable legal limit.
Tandem-axle
Weight Violation
Proportion The fraction (or percentage) of the total observed truck
sample with one or more tandems that exceeds the legal
tandem-axle weight limit.
Tandem-axle
Weight Violation
Severity The extent to which average measured tandem-axle
weights for the observed sub-sample of tandem-axle
weight violators exceeds the applicable legal limit.
Bridge Formula
Violation
Proportion The fraction (or percentage) of the total observed truck
sample that exceeds the legal Bridge Formula weight.
Bridge Formula
Violation
Severity The extent to which average measured bridge formula
weights for the observed sub-sample of bridge formula
violators exceeds the legal weight.
Excess ESALs Proportion The fraction (or percentage) of the total observed truck
sample exhibiting excess ESALs (equivalent single axle
loads); i.e., ESALs attributable to the illegal portion the
individual single or tandem axle group.
Excess ESALs Severity The average value of excess ESALs observed for the
truck sub-sample exhibiting excess ESALs.
Source: Hanscom, F. R.. Transportation Research Corporation. NCHRP Web Doc 13
Developing Measures of Effectiveness for Truck Weight Enforcement Activities: Final
Report. NCHRP, Transportation Research Board (Mar 1998).
The types of measurements, MOEs used, and results varied among the states in the study.
California
The California Department of Transportation provided output from a WIM scale located
on I-5. An analysis of 3,678 truck combinations exhibited lower gross weights with a
smaller proportion of overweight axles during the time when the weigh station was open.
Data on a sub-sample of 2,370 tractor-semitrailer combinations was further analyzed to
determine MOE sensitivity to enforcement activity. Results confirmed the validity of the
following MOEs: Tandem-Axle-Weight Violation Severity, Bridge Formula Violation
Proportion, and Excess ESAL Severity.
38
Georgia
Mobile truck weight enforcement operations, using a portable roadside weigh scale, were
conducted at a rural interstate location. An analysis of WIM data gathered on 483
combination trucks revealed a number of valid MOE effects associated with observed
axle and tandem weights. Under conditions of observable, and unexpected, mobile
enforcement operations, the observed truck sample exhibited lower steering axle weights,
lower rear-axle weights, and lower rear tandem weights. During the surprise enforcement
operation, a number of overweight trucks were observed to either park alongside the
roadway or divert to alternate routes. Results validated the following MOEs: Single Axle
Weight Violation Proportion, Tandem Axle Weight and Excess ESAL Severity.
Idaho
WIM data gathered on 29,000 commercial vehicles, were provided by the Idaho DOT. A
comparison of baseline versus enforcement conditions during three different weekdays
produced several significant findings. While no day-of-week effects were readily evident
to indicate on which days enforcement effort would more likely be effective, all of the
tested operational measures were shown to be sensitive to enforcement activity.
Measures of Effectiveness most consistently demonstrating sensitivity to enforcement
activity were: Gross Weight Violation Proportion, Single Axle Weight Proportion,
Tandem Axle Weight Proportion, and Excess ESAL Proportion.
Minnesota
Data sets representing two weeks of continuous traffic monitoring were provided by the
Minnesota DOT. Bending plate WIM data were collected approximately five miles from
a permanent truck weight enforcement scale during times when the scale was both open
and closed. The Minnesota results were generally weaker than other study sites, but one
WIM data set did exhibit a smaller proportion of gross weight and tandem axle
violations, along with a tendency for less severe ESALs.
All of the tested Measures of Effectiveness were shown to be sensitive to actual weight
enforcement activities, but validated measures varied from state to state. A number of
factors were seen to affect MOE sensitivity to enforcement procedures, including actual
truck weight/configuration characteristics, shipping commodity demands, observed truck
sample size, and WIM equipment variables.
The authors asserted that proper quantification of effectiveness required measures which
showed benefits in terms of: 1) compliance with operational weight and axle-spacing
regulations, 2) pavement and bridge preservation, or 3) minimization of crashes, deaths,
injuries and property damage. However, as shown in Table 7, only the first and second
classes of benefits were considered for inclusion in the study. The omission of safety-related
MOEs was due to the type of data being evaluated (WIM) and the relative
difficulty in ascribing causal factors to crashes.
Also notable in the NCHRP study was that the authors specifically identified the need to
measure enforcement compliance in the context of actual truck exposure (e.g., total truck
volume), in order to ensure that the sample(s) observed adequately characterized the overall
39
truck population. In contrast with many of the state-specific performance measures, the
NCHRP measures of effectiveness did not consider gross measurements, instead relying on
the proportion of violators and the average severity of violations. This practice facilitates
comparisons between states despite variances in truck traffic and commodities.
However, the NCHRP results served as tests to validate the proposed measures, and thus
did not consider the cost of additional enforcement activity. Given that most states are
constrained by budgetary limitations, the added cost of enforcement is a relevant
consideration for evaluation of enforcement procedures. Furthermore, some states have
expressed concerns that WIM data are not reliable enough for enforcement planning
[State of Arizona Auditor General 1986; Oregon Office of the Secretary of State 2002;
Arizona Department of Transportation 2001; Michigan Auditor General 2001]. While
WIM measurements appear useful as a measure of program effectiveness, the variety of
measures selected by different states in the study, as well as reservations about the utility
of these data, indica