Snowplow simulator training evaluation

              SNOWPLOW SIMULATOR TRAINING EVALUATION

Final Report 585

Prepared by: Mary Kihl, Ph.D. Donald Herring Peter Wolf Stephanie McVey Vamshee Kovuru College of Design Arizona State University Tempe, AZ 85287-2005



November 2006

Prepared for: Arizona Department of Transportation th 206 South 17 Avenue Phoenix, Arizona 85007 In cooperation with US Department of Transportation Federal Highway Administration



The contents of this report reflect the views of the authors who are responsible for the facts and the accuracy of the data presented herein. The contents do not necessarily reflect the official views or policies of the Arizona Department of Transportation or the Federal Highway Administration. This report does not constitute a standard, specification, or regulation. Trade or manufacturers' names which may appear herein are cited only because they are considered essential to the objectives of the report. The U.S. Government and the State of Arizona do not endorse products or manufacturers.



ATRC reports are available on the Arizona Department of Transportation's Internet site.



Technical Report Documentation Page

1. Report No. 2. Government Accession No. 3. Recipient's Catalog No. 5. Report Date



FHWA-AZ-06-585

4. Title and Subtitle



Snowplow Simulator Training Evaluation



November 2006

6. Performing Organization Code



7. Author



8. Performing Organization Report No.



Mary Kihl, PhD w/ Donald Herring, Peter Wolf, Stephanie McVey and Vamshee Kovuru

9. Performing Organization Name and Address 10. Work Unit No.



Arizona State University School of Planning College of Architecture and Environmental Design Tempe, AZ 85287-2005

12. Sponsoring Agency Name and Address



11. Contract or Grant No. R0585 17P / JPA 05-010T SPR-PL-1(67) -585

13.Type of Report & Period Covered



ARIZONA DEPARTMENT OF TRANSPORTATION th 206 S. 17 Avenue, Phoenix, Arizona 85007 ADOT Project Manager: Stephen R. Owen, P.E.



FINAL REPORTNovember 2004 to November 2006

14. Sponsoring Agency Code



15. Supplementary Notes



Prepared in cooperation with the U.S. Department of Transportation, Federal Highway Administration 16. Abstract ? Snowplow drivers must operate $200,000 units of equipment in blinding snowstorms and demanding traffic conditions. Yet traditional training for new drivers, with limited funding and staff, may be only two or three storm shifts with a partner-trainer. For this level of responsibility, training needs to be enhanced, to improve driver safety and reduce risk. The Arizona Department of Transportation (ADOT) outsourced simulator training for snowplow operators in rural Arizona in late 2004. A mobile simulator classroom visited five ADOT districts: Globe, Flagstaff, Holbrook, Kingman, and Safford, to deliver a half-day introductory course with both classroom and simulator training segments. This Year One (2004-05 winter) trainee group included 149 snowplow drivers. In Winter Two (2005-06), more in-depth training was given on a dedicated driving simulator unit, purchased for ADOT's Globe Maintenance District. All 61 of Globe's snowplow drivers took two courses: situational awareness training in the fall, and then fuel management and shifting skills in the spring. All Year Two trainers were experienced ADOT snowplow operators from the Globe District. An interdisciplinary team from Arizona State University (ASU) evaluated the effectiveness of simulator-based training for snowplow drivers as a new dimension in ADOT's winter maintenance training program. The primary focus was on driver response to simulator training, and the effectiveness of that training in terms of public safety and potential cost savings. Clear quantitative results on this small scale have been limited, but two years of experience with simulatortrained snowplow operators in Arizona has resulted in optimism about the potential of simulators as an integral part of a comprehensive winter maintenance and driver skill training program. Based on the Year Two results from Globe and new personnel training needs, ADOT procured two more simulators for Holbrook and Flagstaff Districts in mid-2006. A Working Group was formed of field trainers from the three simulator districts to refine and focus the training courses. A new third-year study will expand on this analysis, with a focus on results of training in proper gear shifting (a control-level skill) to improve fuel efficiency and to reduce repair costs. As the study proceeds, it will continue to evaluate the simulators' effectiveness, providing quantitative documentation to reinforce the qualitative results and to define broader benefits of the driving simulator for heavy equipment operations.

17. Key Words 18. Distribution Statement 23. Registrant's Seal



Driving Simulators, Snowplow Training, Winter Maintenance



Document is available through: ADOT Research Center (ATRC), TH 206 S. 17 Avenue (MD-075R) Phoenix Arizona, 85007

21. No. of Pages 22. Price



19. Security Classification



20. Security Classification



Unclassified



Unclassified



140



SI* (MODERN METRIC) CONVERSION FACTORS

APPROXIMATE CONVERSIONS TO SI UNITS

Symbol in ft yd mi in2 ft2 yd2 ac mi2 fl oz gal ft3 yd3 When You Know inches feet yards miles square inches square feet square yards acres square miles fluid ounces gallons cubic feet cubic yards Multiply By To Find millimeters meters meters kilometers square millimeters square meters square meters hectares square kilometers milliliters liters cubic meters cubic meters Symbol mm m m km mm2 m2 m2 ha km2 mL L m3 m3 Symbol mm m m km mm2 m2 m2 ha km2 mL L m3 m3



APPROXIMATE CONVERSIONS FROM SI UNITS

When You Know millimeters meters meters kilometers square millimeters square meters square meters hectares square kilometers milliliters liters cubic meters cubic meters Multiply By To Find inches feet yards miles square inches square feet square yards acres square miles fluid ounces gallons cubic feet cubic yards Symbol in ft yd mi in2 ft2 yd2 ac mi2 fl oz gal ft3 yd3



LENGTH

25.4 0.305 0.914 1.61



LENGTH

0.039 3.28 1.09 0.621



AREA

645.2 0.093 0.836 0.405 2.59



AREA

0.0016 10.764 1.195 2.47 0.386



VOLUME

29.57 3.785 0.028 0.765



VOLUME

0.034 0.264 35.315 1.308



NOTE: Volumes greater than 1000L shall be shown in m3.



MASS

oz lb T ounces pounds short tons (2000lb) 28.35 0.454 0.907 grams kilograms megagrams (or "metric ton") Celsius temperature g kg mg (or "t")

?



MASS

g kg Mg grams kilograms megagrams (or "metric ton") Celsius temperature 0.035 2.205 1.102 ounces pounds short tons (2000lb) oz lb T



?



TEMPERATURE (exact)

Fahrenheit temperature foot-candles foot-Lamberts poundforce poundforce per square inch 5(F-32)/9 or (F-32)/1.8



F



C



?



TEMPERATURE (exact)

1.8C + 32 Fahrenheit temperature foot-candles foot-Lamberts poundforce poundforce per square inch



C



?



F



ILLUMINATION

fc fl lbf lbf/in2 10.76 3.426 4.45 6.89 lux candela/m2 Newtons kilopascals lx cd/m2 N KPa lx cd/m2 N kPa lux candela/m2 Newtons kilopascals



ILLUMINATION

0.0929 0.2919 0.225 0.145 fc fl lbf lbf/in2



FORCE AND PRESSURE OR STRESS



FORCE AND PRESSURE OR STRESS



SI is the symbol for the International System of Units. Appropriate rounding should be made to comply with Section 4 of ASTM E380



TABLE OF CONTENTS

EXECUTIVE SUMMARY..............................................................................1 I. II. III. IV. V. VI. PROJECT INTRODUCTION..........................................................................7 LITERATURE REVIEW...............................................................................11 ARIZONA SNOWPLOW SIMULATOR TRAINING: YEAR ONE .......... 21 SIMULATOR TRAINING ? GLOBE DISTRICT: YEAR TWO................ 41 QUANTITATIVE ASSESSMENT................................................................55 BROADER IMPLICATIONS OF WINTER MAINTENANCE...................65



VII. FUEL MANAGEMENT TRAINING........................................................... 71 VIII. CONCLUSIONS ............................................................................................77 IX. RECOMMENDATIONS ...............................................................................85



APPENDIX A: MID-SEASON TRAINEE SURVEY QUESTIONNAIRES (TWO YEARS)..............................................................................91 APPENDIX B: MOST CHALLENGING MANEUVERS FOR SNOWPLOW OPERATORS (YEAR ONE)......................101 APPENDIX C: SNOWPLOW DRIVER TRAINING SIMULATOR: FOCUS GROUP QUESTIONS: YEAR ONE/YEAR TWO ......105 APPENDIX D: MONTHLY SNOWFALL AMOUNTS: 1999-2000 TO 2005-06................................................................113 APPENDIX E: SNOWPLOW OPERATIONAL LOSS COSTS: BY WINTER SEASON, BY DISTRICT ....................................117 APPENDIX F: SNOWPLOW OPERATIONAL LOSS COSTS BY DISTRICT, BY WINTER: 1999-2006 ................................121 APPENDIX G: MEASURES OF EXPOSURE RELATED TO OPERATIONAL LOSS COSTS........................125 REFERENCES .......................................................................................................129



LIST OF FIGURES

Figure 1. L-3 Simulator Cab Perspective -- December 2004 Training ................21 Figure 2. Two of Four L-3 Simulators -- December 2004....................................22 Figure 3. Age Categories of Year One Drivers ......................................................26 Figure 4. Years of Experience with Driving Snowplows ......................................27 Figure 5. Drivers Finding Training Demanding.....................................................28 Figure 6. Drivers Feeling Successful/Unsuccessful in Completing Training ........29 Figure 7. Globe Simulator -- Fall 2005 Training ..................................................41 Figure 8. Simulator in Use -- Hands-on Training .................................................42 Figure 9. Globe `Train-the-Trainer' Session -- August 2005 ...............................43 Figure 10. Years of Experience with Driving Snowplows -- Year Two, Globe District........................................................................45 Figure 11. Age Categories of Drivers -- Globe District..........................................46 Figure 12. Challenges Facing Year Two Snowplow Drivers...................................47 Figure 13. Training Concepts Applied on the Job by Year Two Globe Drivers......48 Figure 14. Snowbound Trucks in an I-40 Storm Closure.........................................66 Figure 15. Pre- and Post-Test MPG Results of Fuel Management Training ...........74



LIST OF TABLES

Table 1. Table 2. Table 3. Table 4. Table 5. Table 6. Table 7. Table 8. Table 9. Responses to the Multiple District Surveys ...........................................25 Drivers Finding Simulator Training Met Their Needs ...........................30 Driver Recall of December 2004 Simulator Training -- as Reflected in 2006 Follow-Up Survey.................................................31 Year One Simulator Training Applied on the Job in Holbrook, Flagstaff, and Kingman .....................................................32 Year One Focus Group/Interview Participants .......................................34 Snowplow Operator Activities and Michon's Driver Behavior Model...........................................................40 Historical Snowfall Totals by Winter Season.........................................57 Operational Loss Costs by Winter Season 1999-2006: Project Districts ...................................................................58 Measures of Exposure Related to Operational Loss Costs in Globe ................................................................................60



Table 10. Year One Snowplow Equipment Repair Costs: 2004-05 Winter ? Initial Study Districts.................................................62 Table 11. Year Two Snowplow Equipment Repair Costs: 2005-06 Winter - Study and Control Districts........................................62 Table 12. Statewide Calendar-Year Crashes Related to Snow, Slush, and Ice-Covered Surfaces .......................................................................68



ACKNOWLEDGEMENTS

The ASU project team would like to thank the members of the Technical Advisory Committee (TAC), who have been most helpful in providing data, shaping the project, and critiquing reports prepared in association with this study. The TAC included a broad range of Arizona Department of Transportation personnel, including representatives from ITD Technical Training, Equipment Services, Risk Management, Safety and Health, Central Maintenance Planning, and district offices in Globe, Flagstaff, Kingman, Safford, and Holbrook. The TAC also included a representative of the Federal Highway Administration. Members of the TAC included: Alan Hansen, Annie Parris, Carl Eyrich, Cindy Eiserman, Daniel Russell, David Sikes, Dell Jenkins, Dennis Halachoff, Dennis Johnson, Diane Minton, Erika Blankenship, George Garcia Jr., Jerry Massie, Jo Ann Noriega, John Harper, Randy Routhier, Richard Powers, Sue Olson, and William Kohn. The ASU research team has benefited considerably from the active involvement and diligence of Project Manager Steve Owen of the ADOT Transportation Research Center, who was invaluable in identifying sources of data, keeping the project moving along, and identifying ways of strengthening product. Members of the ASU interdisciplinary team in Year One were: Mary Kihl, Professor, School of Planning, principal investigator; Donald Herring, Clinical Professor, Department of Industrial Design, co-principal investigator; Peter Wolf, Faculty Research Associate, Department of Industrial Design, investigator; and Stephanie McVey, graduate assistant, School of Planning. In Year Two, Vamshee Kovuru from the School of Planning was the graduate assistant.



ACRONYMS & ABBREVIATIONS



ADOA ADOT ADT ASU ATRC ATRI BTW CAC CBT CDL CTRE DOT DPS FHWA FMCSA HMT ITD ITD-TECH L-3 MnDOT MPG MPH NOAA NWS OJT ORG



Arizona Department of Administration Arizona Department of Transportation Average Daily Traffic Arizona State University ADOT's Arizona Transportation Research Center American Transportation Research Institution Behind-the-Wheel Central Arizona College (Casa Grande, AZ) Computer-Based Training Commercial Driver's License Center for Transportation Research and Education (Iowa State University) Department of Transportation Arizona Department of Public Safety (Highway Patrol) Federal Highway Administration Federal Motor Carrier Safety Administration Highway Maintenance Technician (ADOT entry-level crew) Intermodal Transportation Division of ADOT ADOT's Technical Training section of ITD L3 Communications, - parent company of MPRI Ship Analytics, the supplier of TranSim VS III simulators and outsourced training to ADOT during 2004-06 Minnesota Department of Transportation Miles per Gallon Miles per Hour National Oceanic and Atmospheric Administration National Weather Service On-the-Job Training The primary local-level ADOT highway maintenance unit (yard/camp/office)



ACRONYMS & ABBREVIATIONS

"PeCoS" or "PECOS" (PErformance COntrolled System) is ADOT's proprietary highway maintenance activity records system. Pennsylvania Department of Transportation Port of Entry Pennsylvania State Truck Driving Simulator Revolutions per Minute Defensive-driving training model presented by L-3 * (Search, Identify, Predict, Decide and Execute) Strategic, Tactical, Control Matrix ADOT Simulator Working Group ?pilot districts of Globe, Flagstaff, Holbrook Utah Department of Transportation University of Michigan Transportation Research Institute Western Region Climate Center in Reno, NV, administered by NOAA, which archives and distributes date from the National Weather Service



PECOS PennDOT POE PSTDS RPM SIPDE STC Matrix SWG UDOT UMTRI WRCC



EXECUTIVE SUMMARY

BACKGROUND Snowplow drivers typically must operate $200,000 pieces of equipment in long, stressful shifts, during blinding snowstorms under demanding traffic conditions. Yet traditional training, with limited funding and staff, can result in new drivers being sent out alone after only two or three storm shifts with a partner-trainer. For this level of responsibility, training needs to be enhanced, to improve driver safety and morale. In response to this need, the Arizona Department of Transportation (ADOT) Technical Training Group (ITD-Tech) contracted with L-3 Communications - MPRI Ship Analytics to give third-party simulator training to snowplow operators in rural Arizona. In late 2004, the L-3 mobile simulator classroom visited five ADOT districts: Globe, Flagstaff, Holbrook, Kingman, and Safford. L-3 instructors delivered a 2-1/2-hour curriculum with both classroom and simulator training segments. The Year One trainee group (the 2004-05 snow season) included 149 snowplow drivers. ADOT procured its own L-3 simulator for Year Two, to be assigned to ADOT's Globe Maintenance District. In Year Two (the 2005-06 snow season), extensive in-depth training could be provided on this new L-3 TranSim VSIII simulator. All 61 Globe snowplow operators were trained, in two four-hour courses: situational awareness training in the fall, and then fuel management and shifting skills in the spring. All Year Two trainers were experienced ADOT snowplow operators from the Globe District. In late 2004, an interdisciplinary team from Arizona State University (ASU) was engaged to evaluate the effectiveness of driving simulator-based training for snowplow drivers as a new dimension in ADOT's winter maintenance training program. The study was conducted for ADOT's Arizona Transportation Research Center (ATRC) and the Technical Training Group. The primary focus was on driver response to simulator training, and effectiveness of that training in terms of both public safety and potential ADOT cost savings. RESEARCH The university team evaluated the effectiveness of simulator training through quantitative and qualitative assessments of driver response to the training. In Year One, the trainee snowplow drivers were surveyed on the training they had received in the simulator, followed by a series of focus groups at the end of the snowplow season. Interviews with maintenance supervisors and a ride-along task analysis also provided useful qualitative information. A parallel assessment in Year Two provided a comparative evaluation. The ASU team also held four post-winter focus groups in the Globe District, and a fifth focus group involved the supervisors from all seven maintenance yards in the district. Training snowplow drivers via simulators is a relatively new concept, although driving simulators have been widely used for human factor research and automobile driver training 1



for more than 30 years. Simulators offer a safe environment to practice infrequent, dangerous driving scenarios (e.g., a tire blowout). A driver who has over-learned the proper skills in a simulator may be better equipped to manage an actual blowout in real life. By incorporating "active error training," a process in which trainees learn by making errors, driving simulators can be effective tools for what is called "analogical transfer." Through repetitive practice of specific skills, drivers develop expertise at skills similar to those being taught. Simulators are also well suited to training for "adaptive transfer," using one's existing knowledge base to change a learned procedure, or develop a new solution to a problem. ASU Mid-Season Survey Results In the Year One survey in early 2005, over 44 percent of the trainees said the course had fully related to challenges they faced, and another 40 percent felt that it related to some of their concerns. In Year Two, 49 percent of the trainees felt that it related to their specific challenges, but 41 percent said it had not sufficiently addressed issues of visibility, traffic, roadway hazards, and actual plow operations. As to further training, the majority in Year One wanted scenarios relating more closely to local conditions, and this was still an issue in Year Two. Most of the drivers in Year Two were satisfied with the fidelity of the simulator. Still, 65 percent of experienced drivers and 35 percent of less experienced drivers called for more local scenarios in Year Two. In Year Two, drivers were also asked which of the concepts they learned in the simulator had been used on the job. Not surprisingly, 26 percent of respondents made observations related to awareness, which was the primary focus of the course. Another 9 percent made comments relating to hazards on the road. A number of other points were also noted. Driver Focus Groups and Field Staff Interviews At the end of the Year One snow season, in spring 2005, ASU held focus groups in Globe, Kingman, Flagstaff, Holbrook, and Safford to get longer-term perspectives from snowplow drivers on the L-3 training program. What emerged was a wealth of information on the December 2004 simulator course, as well as a fuller understanding of the multi-task aspects of driving, and the challenging conditions facing snowplow drivers. In June 2006, in four focus groups in the Globe District, Year Two drivers again conveyed their enthusiasm for the potential of the simulator-training program. The topics discussed included the "driver awareness" training offered in the fall, and the "fuel management and shifting" training offered in the spring. In terms of driver awareness training, there was a striking difference between attitudes of the newer and more experienced drivers. Newer drivers were enthusiastic about the chance for a jump-start on the season, and said the simulator training had helped them though some "white knuckle" plowing challenges. The experienced drivers said they learned little that was new, and without operational controls on the simulator, they could not practice the more challenging multi-tasking aspects of plow operation.



2



The spring 2006 fuel management/shifting training was well received by all drivers operating manual transmission vehicles, who immediately put their training into practice to see how much fuel they could save. Those driving trucks with automatic transmissions found it not particularly useful. The simulator can report on each trainee's driving performance in such areas as riding the clutch, riding the brake, and grinding the gears. Training in these areas might reduce the amount of maintenance and keep the full fleet operational in a snowstorm. In Year One, the ASU team also visited with district maintenance managers about initial perceptions of the simulator training. Most were optimistic about the potential benefits; their comments on the need for greater realism echoed those of the drivers. A Year Two focus group with Globe District supervisors reinforced their enthusiasm for a system that can provide new drivers with a jump on the snow season, and also give more experienced drivers a refresher before the start of the winter. Quantitative Assessment A parallel quantitative study was launched to determine benefits and costs of snowplow simulator training. The study involved assessments of historical data on snowplowing accidents, liability and insurance claims, and repair records of ADOT snowplows over five winter seasons (1999-2000 through 2003-04). This established a baseline for measuring the effectiveness of simulator training to reduce repair costs to snowplows, to reduce plowingrelated accidents, and to improve roadway driving conditions to reduce accident rates on Arizona highways. ADOT equipment repair records for the 2004-05 winter season showed that six of the 149 drivers with initial simulator training were involved in accidents, resulting in $9,968 in repair costs. By contrast, nine of the 145 snowplow drivers who were not simulator trained had accidents that caused $15,973 in repairs to ADOT equipment. These findings are not statistically significant, but they may indicate a trend. For Year Two, repair figures for Globe were compared to the other four Year One districts, as well as for Prescott, which had no simulator training. Results were inconclusive; Globe's Year Two figures were similar to other districts, and in some cases, higher. Given the small number of accidents in any snow season, a single event is likely to skew reports of repair costs, however. And, accident avoidance is very difficult to quantify. Nevertheless, when repair costs and liability costs are related to exposure (measured in terms of miles plowed or hours spent in plowing or in snowfall inches) the performance in Globe improved on all three measures after the intensive simulator training in Year Two. Public Safety Another indicator of snowplow training effectiveness relates to overall public safety. The stated goal of ADOT snow-management planning is "to provide safe and reliable surfaces for public vehicular use in transporting persons and products." The proportion of injuryrelated and fatal accidents associated with snow and ice are relatively small in Arizona, generally less than one percent of such accidents in the state in any given year. Still, the cost impact to Arizona of 335 personal injuries and 10 fatalities on snow, slush, and ice-



3



covered roadways in 2005 is estimated as $18,012,940.1 Training snowplow operators to maneuver plows more efficiently and safely is expected to result in fewer snowplow accidents, and also reduce accidents among private vehicles. Commercial Shipping Delays Minimizing delay costs for commercial freight shipments in winter is another potential long-term benefit of simulator training. Arizona commercial vehicle operators estimate that a one-day delay costs $700 per truck, and a one-hour delay costs $65. On average, 5,177 trucks cross Interstate 40 daily in the snow season. Using Arizona figures, just a one-hour delay can cost freight operators more than $335,505. If all those trucks had to make the full 355-mile trip on snowy and icy roads across the state at 40 mph, rather than the typical 60 mph, the resulting three-hour delay would cost them more than $1 million. Efficient, effective snow removal is essential to keeping the roads open. The simulator is essentially an investment in sharpening the skills and effectiveness of ADOT snowplow operators, helping to assure that priority routes stay open. Transfer of Training In order to evaluate the effectiveness of the ADOT simulator-training program, the ASU research team focused on transfer of training, the ability to apply what is learned in one context to another. In the current study, this refers to the ability of snowplow operators to apply what they have learned in their simulator training to on-the-road driving practice. To better understand the key driving skills required, the ASU team rode in plow trucks and held focus groups with operators. From this, they sorted driving activities into five categories: Inspecting, Communicating, Driving, Plowing and Spreading. Michon's (1985) driving model served as the framework for this activity model. Three levels of activity describe the set of tasks that comprise driving -- strategic, tactical, and control. Strategic tasks focus on the purpose of the trip and the driver's overall goals. Tactical tasks focus on the choice of maneuvers and immediate goals in getting to a destination. Control tasks focus on the moment-to-moment operation of the vehicle. Driving Skills and Transfer of Training The surveys, focus groups, and performance reports recorded by the simulator all suggest that L-3's SIPDE (Search, Identify, Predict, Decide and Execute) Driver Awareness course was relatively successful at training tactical skills, but less so for control skills. The fuel management/shifting program, on the other hand, seems better designed for teaching control skills. While the SIPDE-Driver Awareness program has a broad focus, the fuel management and shifting training is more narrowly focused on proper gear shifting and related clutch usage. Drivers reported that they quickly applied what they had learned, and saw positive results. Although not statistically significant, the results do suggest positive transfer of training of tactical skills from Driver Awareness training, and control skills from Fuel Management.

1



From ADOT Motor Vehicle Crash Facts, 1999-2005, and National Safety Council.



4



Summary Observations Based on the Year Two experience in Globe, and the need for consistent new-hire training, ADOT made a policy decision to procure two more L-3 simulators in mid-2006, to expand this training program into more of its critical snow-country districts. With three units now deployed to the Globe, Flagstaff, and Holbrook Districts, the following points, as regards sound planning and consistent training course development, will be crucial. 1. New and experienced snowplow operators seem to want different things from the L-3 simulator training. How well each group of drivers will respond to simulator training may depend on the driving skills being taught. For states like Arizona, with high rates of driver turnover, the current simulators are quite useful for training tactical-level driving skills for inexperienced drivers and enhancing safety -- the primary concern for all Department of Transportation agencies (DOTs). 2. It may be easier to quantify transfer of control-level skills than transfer of tactical-level skills. Tactical skills are more "big picture" skills, and therefore are more complex to study and measure. It is relatively easy, however, to determine if drivers are shifting gears more efficiently (e.g., by way of fuel consumption, reduced clutch maintenance, etc.). 3. How a training program is presented to trainees is critical to its success. The first step in designing or purchasing a training program, then, ought to be asking what driving skills are needed and how is the course "marketed" to trainees? ADOT's new Simulator Working Group (SWG) includes the Globe, Flagstaff, and Holbrook Districts, each with an L-3 unit. This team of plow operator-field trainers will be critical in defining desired outcomes of the simulator training, and in shaping the way in which it is marketed to trainees. 4. Globe trainees unanimously praised the ADOT trainers - all veteran snowplow operators. In fact, the trainees reported that they learned a great deal from the `low-tech' storytelling aspects of their training sessions, as well as from the `high-tech' simulator itself. RECOMMENDATIONS The following specific recommendations are drawn from the research team's two-year assessment: ? ? Offer consistent programs in all three districts with simulators in 2006-07, and maximize the Globe successes in using experienced local drivers as trainers. Challenge the new multi-district Working Group to identify specific training issues, and to refine simulator programs to address those concerns. Market the courses with titles that clearly inform drivers and underscore course objectives. For example, winter SIPDE classes could be called Driver Safety or Driver Awareness Training, and the spring fuel management course might be called Training in Driving Techniques. Enhance content of the courses so that they relate to challenges faced in the real world, and allow drivers to practice using scenarios to address those challenges. Enhance driving technique courses with training of key functions for all participants. In a course on manual shifting techniques, for example, add relevant lessons for drivers of automatic transmission vehicles. 5



? ?



? ? ?



Offer all drivers documented feedback on performance, and the opportunity to practice in their areas of concern. Separate experienced drivers from less experienced or new drivers in SIPDE/Driver Awareness courses. Offer the more experienced drivers an advanced class on tactical issues that are challenging for all drivers, such as dealing with motorists, visibility, and hazards, in as realistic a setting as possible. Provide more independent practice time for less experienced drivers so that they can better integrate their simulator and their on-the-job training. Enhance the fuel management/shifting course with more focus on reports provided by the simulator. Criteria can be set to reflect desired driving policies of each district, and ADOT in general. Incorporate references to the de-icing training by highlighting the timing for applying the chemicals, and encouraging the driver to regularly check the (imaginary) temperature gauge.



? ?



?



SUMMARY Two years of experience with simulator training for snowplow operators in Arizona leaves an optimistic feeling about the potential of simulators as an integral part of comprehensive winter maintenance and driver-skill training programs. Further research has been initiated for a third year, with a focus on proper gear shifting (a control-level skill) to improve fuel efficiency and to reduce repair costs. As the study proceeds, it will continue to evaluate the simulator's effectiveness, providing quantitative documentation to reinforce the qualitative results and to define broader benefits of the driving simulator for heavy equipment operations.



6



I. PROJECT INTRODUCTION

Driving simulators have been widely used for human factors research and automobile driver training and retraining for more than 30 years (Linck, Richter, & Schmidt, 1973). Commercial trucking companies are increasingly using them to train drivers and to sharpen their skills. They have not, however, been widely used by Departments of Transportation for training heavy equipment operators. The Arizona Department of Transportation (ADOT) has been a leader among state DOTs, by providing simulator training to snowplow operators since late 2004. The Utah Department of Transportation (UDOT) helped to initiate this program of simulator training in 2003 for a limited number of its plow operators, and has outsourced this course on a broader scale in subsequent snow seasons. And most recently, the Iowa DOT has initiated a simulator training program very similar to ADOT's, beginning in 2005. Snowplow operators are a crucial group of employees in Arizona, a state that sees irregular snowfalls in the rural, mountainous districts of the northern and eastern parts of the state. Some snow seasons have several snowstorms, while others may have only one or two storms -- but they are heavy enough to challenge even the most experienced drivers. Nevertheless, drivers operating equipment valued at up to $200,000 are expected to perform efficiently in clearing the roads, and safely in regard to hazards along the road or motorists who are inexperienced in driving in major snowstorms. Simulator training can offer refresher courses to get experienced drivers ready for the snow season, and give recent hires much-needed pre-season instruction and practice. Given heavy turnover rates, the expectation is that training new hires on the simulator can help to reduce -- but certainly not eliminate -- time needed in on-the job ride-alongs. For the 2004-05 snow season, ADOT's Intermodal Transportation Division (ITD) contracted with the MRI-Ship Analytics unit of L-3 Communications to introduce simulator training to Arizona snowplow operators. Four L-3 TransSim VS III simulators, mounted in a mobile classroom, visited five rural ADOT districts (Globe, Flagstaff, Holbrook, Kingman, and Safford). Training was provided to ADOT drivers by L-3 trainers. In late 2005, ADOT commissioned an L-3 simulator of its own, located in the Globe Maintenance District, initiating a far more extensive pilot training program there for some 60 snowplow drivers. In-house volunteer trainers -- each of whom is an experienced snowplow operator -- were selected, and went through L-3's "Train the Trainer" program. Two additional L-3 simulators have since been purchased for the Holbrook and Flagstaff districts, further expanding the training program for 2006-07. The following report was prepared by an interdisciplinary team from Arizona State University that was asked to evaluate the effectiveness of simulator-based training for snowplow drivers, in cooperation with ADOT's Arizona Transportation Research Center (ATRC). The focus of the study was on driver response to simulator training, and the transfer of training to the real world of winter storm operations. 7



The study is grounded in theory related to transfer of training and assessments of internal validity (relating to consistency and accuracy of reporting on driver performance on the simulator) and external validity (in terms of application to the real world.) Chapter II of this study summarizes literature related to key theories as well as a discussion of related applications of simulators. The chapter ends with a perspective on expectations of simulators in the training process. Those concepts are further elaborated in the rest of the study and applied to an assessment of the application of simulator-based snowplow driver training in Arizona. Chapter III offers a qualitative assessment of the driver simulator training in Year One (2004-05), when an introductory simulator-training course was offered to snowplow drivers in the Globe, Flagstaff, Holbrook, Kingman, and Safford districts. The chapter highlights responses gained from multiple approaches of qualitative data gathering, including site visits, surveys, focus groups, and discussions with supervisors about the training offered in 2004-05. As the discussion in the chapter points out, drivers were enthusiastic about the simulator. In addition, a driving behavior model, introduced by J.A. Michon, offers a comprehensive approach to synthesizing both expectations and contributions of training on the current simulators. Chapter IV offers a parallel qualitative assessment of driver similar training in Globe in Year Two (2005-06). The training program in Globe involved a consistent 4-hour session for small groups of drivers, offered by experienced snowplow operators from the Globe District. The chapter discusses surveys and focus groups with drivers who participated in the Year Two study. An additional focus group involved maintenance supervisors from each of the ORGs in the Globe District. The drivers in the Globe District had a much more comprehensive exposure to simulator training than did the drivers in Year One. Consequently, they were more circumspect in terms what they had learned from the simulator training courses, what was possible to incorporate into subsequent courses, and what they could incorporate into their own snowplow operations. Chapter V offers a quantitative analysis of the value of simulator training in terms of expected reductions in accidents, repairs, and liabilities. A five-year baseline of data was established for each of the districts, and figures for 2004-05 and 2005-06 were compared with the baseline. In order to equalize exposure levels among the five districts where drivers had had some interaction with simulators, the miles plowed in each district was related to operational losses faced by ADOT. Snowfall data provided by the state climatologist was also provided for each district. By way of comparison, data was also provided for the Prescott district, where drivers had no exposure to simulator training. As was anticipated, the data provided limited information regarding the impact of snowplow driver training. Quantitative data cannot show figures for crashes avoided, although anecdotal reports indicate that the awareness training did help some drivers cope with challenges faced in the real world. Another factor is the point that, in general, ADOT snowplow drivers are careful drivers and there are few crashes reported.



8



Chapter VI offers a broader perspective on the potential impact of efficient snow removal on cross-country shipping, and other economic factors that reach far beyond the bounds of Arizona. Chapter VII focuses on the Fuel Management training program, as offered in Globe in spring 2006. That training, unlike the Driver Awareness training in the fall, does focus on operational steps that drivers can take to improve their performance. Similarly, the benefits of that training in terms of fuel saved, and the minimizing of routine repairs to clutch components, brakes, and transmissions, can be traced much more directly to this simulator training of operational skills. Chapter VIII summarizes the conclusions drawn from the findings of this study. Chapter IX offers recommendations to both the Globe District and the two other districts that are about to start simulator training, as well as more general recommendations to other agencies considering truck driving simulator training.



9



10



II. LITERATURE REVIEW

SIMULATORS AS TRAINING TOOLS Training snowplow drivers via simulators is a relatively new concept, although driving simulators have been widely used for human factors research and automobile driver training and retraining for more than 30 years. (Linck et al., 1973, as cited in Reed & Green, 1999). "Operator-in-the-loop" simulators were first developed to train military pilots (Wiener & Nagel, 1988), but have since been used to train locomotive engineers, and ship helmsmen (Emery, Robin, Knipling, Finn, & Fleger, 1999, p. 4). Driving simulators are being used for a wide variety of vehicle applications, including cars, large trucks (Hoskins, El-Gindy, Vance, Hiller, & Goodhart, 2002), buses (Brock, Jacobs, van Cott, McCauley, & Norstrom, 2001), off-road equipment ("Painless Haul-Truck Crashes," 2000), and cranes (Angelo, 2001), among others. Recently, driving simulators have been used to train snowplow operators in Pennsylvania (Vance, El-Gindy, Hoskins, Hiller, & Tallon, 2002) and Utah (Strayer, Drews, & Burns, 2004). Today's state-of-the-art driving simulators generally feature high-resolution graphics, rapid refresh rates, and nearly-180? fields of view. The University of Michigan's Transportation Research Institute (UMTRI) purchased a new driving simulator in 2002. Although the UMTRI simulator is more sophisticated than most, its description (RossFlanigan, 2002) could easily be used to describe many of the driving simulators on the market today (including the L-3 TranSim VS III model used by ADOT): The driving simulator consists of a modified, full-sized vehicle console in a room with wall-sized screens. Computer-generated images simulate views of a roadway as seen through the windshield and in the rear-view mirror. The vehicle is equipped to sound and feel real as it cruises down the highway or drives along city streets. (Ross-Flanigan, 2002, p. 1) Driving simulators are generally used for one of three primary purposes: research, engineering, or training (Emery et al., 1999), although they sometimes serve a combination of these purposes. Research simulators are often used for human factors and cognitive psychology experiments to study various elements of driving behavior (for examples, see Kemeny & Panerai, 2003; Reed & Green, 1999; Sidaway & Fairweather, 1996). Engineering simulators are generally used by automotive manufacturers and suppliers to develop and evaluate vehicle components and systems (for example, see Nagiri, Amano, Fukui, & Doi, n.d.). Training simulators are used by public and private agencies to teach and evaluate various driving skills (for examples, see Emery et al., 1999; Strayer & Drews, 2003; Strayer et al., 2004; Vance et al., 2002). Driving simulators may be categorized as either fixed-base or motion-base simulators. Fixed-base models range from simple, desktop computer models (for example, see Lee, Lee, & Cameron, 2003), to those utilizing head-mounted display with head tracking technology (Liu, Miyazaki, & Watson, 1999), to units that include partial (Ross-Flanigan, 2002) or full vehicle cockpits (for examples, see 2002; Roenker, Cissell, & Ball, 2003). 11



Motion-base simulators are generally more sophisticated than fixed-base models, and feature motion cues that mimic the roll, pitch, and yaw of actual vehicle dynamics. The Iowa Driving Simulator, located in the University of Iowa's Center for Computer-Aided Design is one of the most sophisticated motion-base driving simulators (Kuhl, Evans, Papelis, Romano, & Watson, 1995). Although fixed-base simulators have the obvious advantage of cost, their lack of motion cues may alter "the perceived motion variables that serve as inputs to [one's driving] strategy" (Reymond, Kemeny, Droulez, & Berthoz, 2001, p. 493). This becomes especially important during the low-friction conditions associated with snowplow operation. In this case, even small motion cues (e.g., one to two inches) make a significant difference in how realistic the simulation experience feels to users (P.A. Green, personal communication, February 28, 2005). Lacking these motion cues, fixedbase simulators may demonstrate less internal and external validity compared to motionbase models (as discussed below). Whether used as research, engineering, or training tools, driving simulators offer several advantages over real-world driving. Safety is a primary advantage, as simulators can be used to expose drivers to driving conditions too dangerous to consider for real-world driving (Liu et al., 1999; Reed & Green, 1999). As a training tool, simulators allow trainees to practice driving and develop confidence before taking a road test (Liu et al., 1999, p. 5). Most simulators also have the ability to record and play back training sessions, meaning that evaluation can be objectively assessed, although this advantage is "rarely exploited" (Hoskins et al., 2002). Populations studied have included older drivers (Hakamies-Blomqvist, ֳtlund, Henriksson, & Heikkinen, 1995; Lee et al., 2003), teens (Deery & Fildes, 1999), "headinjured" (Liu et al., 1999) and brain-damaged drivers (van Zomeren, Brouwer, & Minderhoud, 1987), and over-the-road truck operators (Hoskins et al., 2002; Manger, 2003; Pierowicz et al., 2002). VALIDITY OF DRIVING SIMULATORS Despite the increasing popularity of driving simulators, the correlation to real-world driving behavior is unclear. As Hoskins, et al. (2002) note in their report, Truck Driving Simulator Performance Effectiveness, "The most significant disadvantages of driving simulators are a lack of consistent experimental support for simulator training and knowledge transfer. On the whole, surprisingly little work had been done to evaluate the advantages claimed for simulator training" (p. 52). "The correlation to real-world driving behavior" is often referred to as a simulator's validity. According to Emery et al.: Validation can be judged by the extent to which the real environment and simulator evoke similar driver response and behavior. Furthermore, changes in tasks should evoke corresponding changes in driver response and behavior... In general, measures of response and behavior useful in validation include driver 12



control actions and response, vehicle motion response, driver plus vehicle response and performance, and subjective ratings and commentary (Emery et al 1999, p. 6). Vance et al. (2002) don't use the term validity at all; rather they use fidelity to describe "the extent to which it can simulate the real-life situation" (Vance et al. 2002 p. 11). According to these researchers, there are two types of fidelity: physical fidelity refers to "the extent to which the simulator looks like the real situation," while functional (or psychological) fidelity refers to the "extent to which the operator, equipment, and simulated environment interact in the same way as a real-life situation" (Vance et al., 2002, p 12). Although one might reasonably expect there to be a strong relationship between simulator validity (or fidelity) and knowledge transfer, this is not necessarily so. According to Vance et al., the fidelity required of a particular simulator depends upon the training to be conducted, and they note that "certain tasks and skills can be learned even in very crude simulators" (Vance et al., 2002, p. 13): Reasoning or cognitive ability tasks do not require high physical fidelity levels. The skills in these settings are generalizable to many different areas, not only truck driving, and the physical layout need not be exact. High physical fidelity is necessary when the training involves learning perceptual-motor skills, or the interaction of the trainee with the layout of the equipment. An example of where high fidelity is needed is when the goal is to practice tasks that cannot be practiced in the field because they are too dangerous, such as simulated spinouts on ice (Vance et al., 2002, p. 13). Demonstration of internal validity is often considered adequate for engineering and research simulation. "The validation of simulation, however, for the training of a particular skill is most appropriately addressed through an assessment of whether that training actually transfers to the environment in such a way as to encourage skill proficiency and safe operating practices" (Emery et al., 1999, p. 7). As has been shown, little research has been done to support the external validity of simulator training. The ability to apply what is learned in one context to another context is generally called transfer of training. In the case of the current project, this refers to the ability of snowplow operators to apply what they have learned in the simulator training course to on-the-road driving practice. If drivers trained in the simulator perform better on the road than those drivers not trained in the simulator, then it could be concluded that positive transfer has occurred. Conversely, if those trained in the simulator perform worse than their conventionally trained counterparts, it would be considered negative transfer. If there is no difference, then zero transfer has occurred (Goldstein, 1986). Simulators offer a safe environment in which drivers can practice infrequent, dangerous driving scenarios (e.g., a tire blowout). As such, they are well-suited to what is called "over-learning," the "rehearsal of a response past a minimally acceptable performance 13



level [that] serves to maintain proper performance in stressful situations" (Emery et al., 1999, p. 70). So it is thought that a driver who has over-learned the proper skills in a simulator would be better equipped to manage an actual tire blowout in the real world. This would be an example of analogical transfer, which "involves using a familiar problem to solve a problem of the same type" (Reeves & Weisberg, 1994, as cited in Ivancic & Hesketh, 2000, p. 1967). By incorporating "active error training" (Ivancic & Hesketh, 2000), in which drivers are permitted to make their own errors, simulators can be effective tools for analogical transfer. However, it has been shown that "interventions designed to teach specific driving skills (e.g., skid training) have often produced weak or inconclusive results" (Katila et al. 1996, as cited in Ivancic & Hesketh, 2000, p. 1966). Furthermore, there is some concern that "increasing expertise leads to less adaptable skills" (Hesketh, 1997), thus reducing transfer of more generalizable skills. It would be useful, for example, if learning how to manage a tire blowout in the simulator would also improve a driver's ability to safely manage a skid caused by icy road conditions. This would be an example of adaptable transfer, which "involves using one's existing knowledge base to change a learned procedure, or to generate a solution to a completely new problem" (Smith et al. 1997, as cited in Ivancic & Hesketh, 2000, p. 1968). Simulators are well-suited to training for adaptive transfer as well. TRANSFER OF TRAINING AND "SMILE SHEETS" Post-training questionnaires -- or "smile sheets" (Hesketh, 1997, p. 328) -- are often used to support the general idea of knowledge transfer; however, the literature indicates the many shortcomings of such questionnaires for this purpose. The primary criticism is that short-term evaluation tools (e.g., post-training questionnaires) place too much emphasis on immediate trainee satisfaction, and too little emphasis on long-term training transfer. As Hesketh argues, "those methods used during training that are most effective at the end of training, are not necessarily best for transfer and vice versa" (Hesketh, 1997, p. 325). This is especially true for those components of training that, while "effective in developing skills that transfer," are also found to be "more effortful and may adversely affect self-efficacy for training and expectancies for success" (Hesketh, 1997, p. 328). Because of the immediate "fix" associated with these types of training courses -- often at the expense of long-term benefits -- Hesketh refers to them as "cigarette courses," adding that their addictive quality often makes them "popular with trainers, managers, and the trainees" (Hesketh, 1997, p. 382). Simply put, the evidence that drivers trained in simulators perform better on the road is ambiguous, and the lack of validation is "a problematic trend" (Emery et al., 1999). A 1999 study by the Federal Motor Carrier Safety Administration (FMCSA) investigated validation of simulator training for over-the-road truck drivers. The purpose of the study was to "examine how simulator technology, as compared to conventional methods, may facilitate and enhance tractor-trailer driver performance" (Emery et al., 1999). The study was designed to train a control group of novice tractor-trailer drivers using purely "conventional" behind-the-wheel (BTW) methods, while an experimental group received a combination of simulator training and BTW training. After their training, the students 14



would take the Commercial Driver's License (CDL) examination. To evaluate transfer of training, the performance of each group on the CDL exam was to be compared. In addition, a longitudinal study was planned, in which drivers' 3-month and 12-month driving records (number of crashes, number of citations, supervisory ratings, etc.) would be examined. This part of the study was aimed at addressing issues of training retention (Validation of Simulation Technology in the Training, Testing, and Licensing of TractorTrailer Drivers, 2000). Part 1 (Emery et al., 1999) and Part 2 (Pierowicz et al., 2002) final reports have been published, and are discussed in detail below. The report for Part 3 (the longitudinal study) was apparently not published. A similar longitudinal study, by the American Transportation Research Institute, is currently in the planning stages (ATRI, n.d.). OTHER STATES' EXPERIENCES WITH SIMULATORS Arizona is not the first state to use driving simulators to train snowplow operators. Other states, including Pennsylvania, Utah, and Iowa have done work in this area. What follows is a summary of the investigation and development work that these other state DOTs have done with driving simulators. Pennsylvania DOT The Pennsylvania Department of Transportation (PennDOT) has also considered using simulators to train its snowplow operators. A study was conducted in 2001 to investigate upgrading the Penn State Truck Driving Simulator (PSTDS) with software appropriate to snowplow scenarios. The PSTDS, located at Pennsylvania State University, is a motionbase unit used primarily as a research simulator (Vance et al., 2002). The study also investigated PennDOT's training procedures for operators of large vehicles, and how simulators might be used to supplement this training. Four research questions were posed, as follows: 1. Can targeted vehicle operation skills be improved through simulator experience? 2. Does simulator training with instruction enhance vehicle operation skills compared to simulator training without instruction? 3. Do simulator-recorded measurements of vehicle operation skills agree with instructor evaluations of the same skills? 4. Is initial training academy performance enhanced by simulator training? Question 1 The results of the study generally supported the idea that targeted vehicle operation skills can be improved through simulator experience, although to varying degrees. In this case, both subjective data (instructor ratings) and objective data (simulator measures) indicated that "driving performance improved with practice for each scenario" (Vance et al., 2002, p. 66). However, performance in some driving scenarios seemed to benefit from practice more than in other scenarios. According to the researchers, "it is likely that some driving skills are more easily learned in simulator training than others" (Vance et al., 2002, p. 66). It is important to note that at this stage of the study, no on-the-road driving was 15



conducted; all work was conducted in the simulator. The positive results, therefore, suggest good internal validity, but say nothing of external validity. Question 2 The question of whether simulator training with instruction enhances vehicle operation skills, compared to simulator training without instruction, "received overwhelming support" (Vance et al., 2002, p. 66). Study participants who received simulator training with instruction "consistently performed better" than participants who received simulator training without support, and "their learning curves were typically steeper" (Emery et al., 1999, p. 66). As with the first research question, this question addresses issues of internal validity only. Question 3 Results indicated that simulator-recorded measurements of vehicle operation skills do agree with instructor evaluations of the same skills. This is essentially a question of how well subjective data (instructor ratings) correlate with objective data (simulator measures). These findings may offer some support for the use of simulators as training tools. Rather than monitor students continuously, instructors could be used primarily for initial instruction. Following initial instruction, "the simulator measurement charts... could be monitored periodically, with instruction provided as needed until acceptable levels are achieved" (Vance et al., 2002, p. 66). Once again, this question suggests good internal validity, but does not address issues of external validity. Question 4 In order to investigate external validity of the driving simulator, researchers studied initial training academy performance by those participants who received simulator training. Based on an assumed positive transfer of training from the simulator, the researchers expected that study participants who received simulator training should have outperformed those who had not received simulator training in actual driving tasks at PennDOT. However, this was not the case. Indeed, at one of the two training academies, drivers who had received simulator training actually rated lower than drivers who had not received simulator training. On the surface, then, this would appear to suggest poor external validity (and negative transfer of training) for the simulator. However, a number of unforeseen factors contributed to these findings. Average daily ratings (from academy instructors), for example, do not necessarily reflect individual driver performance, or individual skills. As such, they may be insufficient measures of "driver success" at the academies. Also, the researchers found differences in ratings between the two training academies. The single greatest factor contributing to these surprising results is that different measures were used during simulator training and in-vehicle training at the academies. Therefore, no direct comparison was possible. As the researchers note, the "demonstration of transfer of learning of these skills required comparable measures of performance in the training and job contexts" (Vance et al., 2002, p. 67).



16



Implications of the PennDOT Study Despite the limitations of the PennDOT study with relation to external validity and knowledge transfer, the researchers' recommendations are worth consideration. The researchers concluded that both novice and experienced PennDOT drivers could likely benefit from simulator training, including the training of "advanced driving skills... such as snow plowing" (Vance et al., 2002, p. 68). Because of rapidly changing simulator technology, "and the variety of potential uses of this technology in actual training," the researchers suggest "it seems reasonable to invest in more than one type of driving simulator and to spread these acquisitions out over time" (Vance et al., 2002, p. 68). While the researchers are optimistic about the potential benefits of simulator training, they are equally cautious: Implementing simulators involves a reconsideration of the entire system, and may not be possible considering the cost, upkeep, and required effectiveness of the simulator. Many options and alternatives must be weighed before deciding to use simulators. Detailed and thoughtful design of training will raise the likelihood that a simulator is effective. This process begins with a training needs/task analysis, followed by an analysis of opportunities and limitations of the specific simulator. A careful consideration of how the simulator is to be used as part of a training program is critical, followed by a decision about what to measure and how to measure it. (Vance et al., 2002, p. 16) Utah DOT The Utah Department of Transportation (UDOT) began using simulators to train snowplow operators in the 2003-04 winter season. The project began as a collaborative effort between UDOT, the University of Utah's Dr. David Strayer, and General Electric Driver Development to "determine the feasibility of using high-fidelity simulator training to improve the performance of UDOT maintenance operators (i.e., snowplow drivers)" (Strayer et al., 2004). Tasks included an initial needs analysis, development of relevant simulator driving scenarios, actual driver training, and comparison of driving performance for both the simulator-trained and control groups for six months following the training. The authors of the final report indicate positive results of their study: "Overall, the snowplow simulator training program offers a number of attractive benefits for UDOT, including a reduction in the frequency of accidents, a decrease in the cost associated with each accident, and an increase in fuel efficiency" (Strayer et al., 2004, p. 22). Despite the optimistic findings of this study, it offers little in the way of external validity or transfer of training. The number of accidents during the period in question was relatively low, at seven. Of those seven accidents, three involved drivers from the study group -- but in two cases "the trained driver was determined by UDOT to be not responsible for the accident" (Strayer et al., 2004, p. 16). It is therefore very difficult to draw any clear conclusions regarding the effect of simulator training on accident prevention.



17



The difficulty in getting verifiable data regarding the cost savings associated with reduced accidents caused the Utah team to focus more on savings that would be more easily quantified, like fuel savings (D.L. Strayer, personal communication, May 2006). UDOT is not the first to use simulator training to improve drivers' shifting skills -- and therefore improve fuel economy. Indeed, the authors of the UDOT report have themselves conducted a Fuel Management simulation study to quantify the improvement in fuel efficiency for CDL truck drivers" (Strayer & Drews, 2003). The driving patterns of typical CDL truck drivers and snowplow drivers vary considerably, however, and fuel consumption will vary accordingly. Although fuel consumption data were not available through UDOT, the Utah study team noted that a commercial company operating mining equipment did save a considerable amount of fuel after their drivers were trained in proper shifting techniques on the simulator. That point attracted interest in simulator-based training for snowplow operators (Strayer, 2006). Many of the factors that can be controlled in a study of CDL truck drivers simply cannot be controlled in a study of snowplow operators. Indeed, the authors concede that "neither the maintenance data nor the fuel data are of sufficient quality to afford a precise comparison between the study and control groups," and that "the data that were included in the analysis still have unknown levels of noise" (Strayer et al., 2004, p. 19-20). Nevertheless, they suggest "there is every reason to expect that the benefits of training observed on the commercial side will be similar for UDOT drivers" (Strayer et al., 2004, p. 21). A follow-up study by the Utah research team in 2005 determined that it would be more cost effective for UDOT to engage L-3 as a third-party training vendor. L-3 is providing simulators in a self-contained trailer that can be transported to field locations, and is also offering the training to UDOT drivers -- much as was done in Arizona in Year One (Strayer, 2006). Unlike the Arizona simulator training approach, the UDOT maintenance districts chose not to undertake in-house training programs that would be focused on district-level concerns and taught by experienced snowplow operators from that district. Implications of the UDOT Study For the past three years, UDOT has been struggling with the same quantitative measures that have challenged ADOT and the ASU research team for the past two years. Their most recent study is moving away from quantitative measures, in favor of qualitative evaluations. As both Arizona and Utah move forward in their respective training programs, it will be useful to continue to "compare notes" along the way. Iowa DOT In 2006, the Iowa Department of Transportation implemented a snowplow simulator driver training program that in many ways parallels the ADOT program. The DOT is using an L-3 simulator, focusing on driver awareness (via L-3's SIPDE curriculum), space management, and fuel management. The Iowa DOT commissioned a study through Iowa State University's Center for Transportation Research and Education (CTRE) that was designed to: 1) assess the use of 18



this simulator as a training tool, and 2) examine personality and other characteristics associated with being an experienced snowplow operator. The research component is conducted by psychology faculty members Professor Derrick Parkhurst and Professor Veronica Dark. The Iowa simulator is transported in a trailer among the six maintenance districts in the state. Each maintenance district has the simulator for about three weeks, and the DOT intends to eventually offer simulator training for all 1,144 drivers in the districts. As of September 2006, 250 drivers, as well as 200 during the initial study, had been trained in three-hour sessions. The drivers are trained in groups of two by local trainers who were brought together for a common train-the-trainer program. All trainers are experienced snowplow drivers selected by their maintenance districts. Since the Iowa DOT is moving to automatic transmission trucks and shifting is not as much of a factor, the fuel management program focuses on reducing idling of all trucks. In the training program a hypothetical situation is presented. If all 900 of Iowa DOT's snowplow trucks would idle for two hours, at a cost of about $5.90 per truck, the resulting cost to the department would be $5,310. If all trucks reduced idling by one hour, expectations are that as much as 1,000 gallons of fuel could be saved. At $2.95 per gallon, this could add up to as much as $2,950 savings for the state. Other measures are also being taken to reduce fuel consumption. The Iowa team has not yet completed their interim report on the simulator training as of this writing, but the response from its operators has been enthusiastic -- particularly among recent hires. Snowplow operations are supplemented with part-time operators and other DOT field staff via a shared worker program. These part-time drivers, in particular, may benefit from the simulator-training program. The past training programs at the Iowa DOT for their snowplow operators have consisted of training videos on DVD, and training conducted by the local garages. The plow drivers are also required to take a defensive driving course every three years provided by the Iowa Highway Patrol. Experienced drivers expressed concern that the simulator-training program does not resemble the real world because it does not have plow or sander controls. One of the project coordinators from the Iowa DOT, Jim Dowd, emphasizes to trainees that it is just a simulator, and is not intended to mirror the real world. "In the simulator we focus on what the drivers need to be aware of while operating a snowplow such as traffic, lane position and road side obstructions. The other facets of operating a snowplow can be learned while riding with an experienced operator during a winter storm." He and others felt that all could benefit from the space management and the SIPDE training programs (J. Dowd, personal communication, August 2006). Implications of the Iowa DOT Study Since the training program is so similar to the Arizona study, future feedback from Iowa may prove valuable to ADOT (although direct comparisons between the two programs will likely be few, since the Iowa study does not include the same metrics as the current Arizona study). The research looking into the physical responses of drivers will provide 19



useful information that is not being collected by the Arizona study. In addition, the relative success of the program in reducing idling will also be valuable information for trainers and Equipment Services in ADOT. Summary of Other DOT Studies All three of the studies outlined in this chapter are optimistic about the potential of simulator training as a part of an overall department of transportation driver training program. Although none of the studies demonstrates external validity, this may be more a result of the research design than the actual validity of the driving simulator. It is also interesting to note that both the PennDOT and the first UDOT studies use strictly quantitative approaches, whereas the current Arizona study combined quantitative and qualitative approaches (an approach adopted by UDOT for their most recent study). This "mixed method" approach has helped to shed some much-needed light on the external validity of the simulator (and the knowledge transfer associated with the simulator training program), and the impact of the simulator on the overall ADOT training program.



20



III. ARIZONA SNOWPLOW SIMULATOR TRAINING: YEAR ONE

L-3 SIMULATOR TRAINING The snowplow driver training programs offered in Arizona in the two snow seasons, 2004-05 and 2005-06, offered very different opportunities to observe and assess the use of a simulator as part of a training program. As noted previously, the training in the first year was provided by subcontractor MPRI-Ship Analytics, a subsidiary of L-3 Communications, using their TranSim VS III model simulators. From December 2 to 21, 2004, a total of 149 snowplow drivers in five Arizona DOT districts (Globe, Flagstaff, Holbrook, Kingman, and Safford) participated in snowplow simulator training. The training was conducted by L-3 trainers on four fixed-base simulators housed in a mobile classroom trailer. Figures 1 and 2 show the trainee's perspective of the L-3 simulator.



Figure 1. L-3 Simulator Cab Perspective -- December 2004 Training In the second year, ADOT purchased one VSIII fixed-based simulator and based it in the Globe District. A pilot training program was focused solely on snowplow operators. The simulator training modules and a "train-the trainer" course was provided by L-3, and the actual training of approximately 60 drivers was conducted by four of the Globe District's senior snowplow operators. For clarity and comparison purposes, this chapter will reflect primarily on the experience within the five districts in the Year One, 2004-05 in the winter snow season. Chapter IV will focus primarily on the Year Two effort in Globe. In the 2004-05 winter snow season, the planned simulator training was adversely affected by a number of factors. Since the 53-foot cargo trailer housing the four simulators was 21



newly-built and moved from site to site, troublesome network and power system delays occurred. To further complicate matters, some of the simulators weren't continually operational at all five of the districts' training sites due to hardware and software issues. As a result, the various drivers experienced a range of from 15 minutes to 45 minutes of "seat time" in the simulator. The basic Year One Driver Awareness training took place in December 2004, but the northern Arizona snow season had arrived early, and as a result of storm activity, some plow drivers selected for the training simulator training class were unable to attend. As a result, the 2004-05 simulator training had less impact than expected as a "refresher course" that would help to increase awareness of snowplowing issues, even for the most experienced drivers. Nevertheless, despite the many challenges, simulator training was conducted in the five study districts, as planned, and all participating drivers received 2-1/2 hours of training, including a combination of "seat time" and classroom training.



Figure 2. Two of Four L-3 Simulators -- December 2004 The ASU team members observed the simulator portion of the Year One training program, as well several full training sessions in Globe in Year Two.



22



In both years the classroom portion of the fall snowplow driver training program emphasized a defensive driving model called SIPDE (Search, Identify, Predict, Decide, and Execute). Each element of SIPDE was explained in some detail, and examples were used to illustrate each point. As indicated, simulator scenarios were used to reinforce the elements of the curriculum. For example, during an "in-town" scenario, trainees were required to search for pedestrians (behind parked cars, in some cases), identify most significant hazards in a particular situation (the school bus in front vs. the motorist speeding past on the left), and so forth. In addition to teaching the SIPDE model, the classroom presentations also included sections on space management, speed management and stopping distance, and crew communications. To add greater realism, the trainers in the Globe District had the option of adding whiteout and/or nighttime conditions. The following simulator scenarios were observed:1 ? Snow-covered freeway -- trainees begin driving on a snow-covered portion of freeway, and stop driving after a couple of minutes. The purpose is to get trainees familiar with the simulator's look and feel. Mountain pass with tunnel -- trainees begin driving down a steep mountain grade at night, and come upon a tunnel. The purpose is to raise awareness of speed control and space management. Parking lot -- trainees drive from a parking lot out onto a road, and contend with a garbage truck attempting to pull out onto the same road (possibly in front of the trainee). The purpose is to get trainees thinking about other drivers on the road. High country driving -- trainees drive along a snow-covered highway, while deer move close to the highway. The purpose is to raise trainees' awareness of potential distractions/dangers. In-town driving -- trainees navigate through a downtown area, contending with school buses, pedestrians, and other motorists. The purpose is to raise trainees' awareness of potential distractions/dangers, as well as particular policy issues.



?



?



?



?



In Year One, although most drivers were intrigued by the simulator, their brief exposure would have had a limited effect on their driving performance. The overwhelming proportion of participating drivers wanted more time in the simulator. A year later, a follow-up survey with multiple districts indicated that 55 percent of drivers who participated in Year One training -- but not Year Two -- were still anxious for more simulator training. ASSESSMENT APPROACH In assessing the effectiveness of the snowplow simulator training in Arizona, the primary group involved obviously was the snowplow drivers. The most important factors are: ?

1



Driver response to the simulator training approach.



This list is based on the recollection of ASU observers--it is not meant to be all-inclusive, but representative of the scenarios experienced by drivers during the training sessions.



23



? ? ?



Driver perspective on the simulator's value as a training tool. The content of the program in augmenting the ride-along training approach. The integration and reinforcement of other training programs with the simulator training. Reviewed the end-of-session survey administered by the L-3 trainers in 2004 Conducted more in-depth surveys of participant drivers in February 2005. In 2006, an additional survey was sent to the districts that had participated in the Year One training to help to gage recall of the simulator experience. Conducted a series of driver-trainee focus groups at the end of the 2004-05 winter season in all five participating districts. Interviewed district maintenance supervisors in 2004-05 to learn about their perspective on the success of a training program.



To assess this perspective, in Year One the research team: ? ?



? ?



Based on an assessment of findings generated by the focus groups, and subsequent interviews in 2005, team members prepared a Driver Behavior Model (see Table 6) that was refined in Year Two. RESULTS L-3 Survey Response Immediately following the training sessions conducted in each of the five participating districts in 2004, the L-3 team offered a brief survey questionnaire asking whether the participants thought the training was worthwhile, and requesting their comments on the training process. Overall, 88 percent of the respondents thought that their training was worthwhile. In one district, 97 percent of the respondents checked the positive response, "agree," on all questions. Comments included "Simulator was great;" "Great class, will be of great benefit;" "Setup was great;" "Class has a lot of good information." The overwhelming proportion of those respondents (83 percent) felt, however, that more simulator time was needed. This level of enthusiasm was anticipated, since the drivers had participated in a new form of training and met in small groups, giving them the opportunity to interact with each other and one-on-one with professional trainers. As note earlier, Hesketh (1997, p. 328) calls these types of surveys "smile sheets," arguing that they provide short-term positive impressions rather than an indication of potential application of training to the real world. Follow-Up Survey Responses In mid-snow-season, in February 2005, the ASU team launched another survey of all drivers who had participated in the 2004-05 snow season training. The objective was to gain more information on specifics of the training program from the perspective of the drivers (that survey questionnaire is included in Appendix A). Of the 149 drivers who participated in the L-3 training in December 2004, 109 returned these surveys, for a 74 percent response rate. 24



A year later, at the end of the 2006 snow season, a follow-up questionnaire was sent to all drivers from the districts that had participated in the training in Year One, except Globe (A separate mid-season survey was sent to Year Two participants in Globe; those results will be discussed in Chapter IV). Only those drivers who had participated in Year One were asked to complete the survey. This follow-up survey netted a much smaller response rate, reflecting, in part, the driver turnover in the various districts, and, in part, the limited recall of the simulator experience after a period of one year. The follow-up surveys were not distributed in Safford. Table 1 indicates the number of respondents from each district to the multi-district surveys in both Years One and Two. Table 1. Responses to the Multiple District Surveys District Globe Flagstaff Holbrook Kingman Safford Total Number of Trainees in Year One 55 15 26 28 25 149 Number of Respondents, Year One 42 14 20 9 24 109 Number of Respondents, Year Two * 4 19 6 ** 29



* The Globe District participated in an alternative survey. ** The Safford District was unable to distribute these survey questionnaires.



25



Year One Mid-Season Survey Respondents Drivers in all age groups participated in the Year One survey, with the largest proportion of the respondents (37 percent) in the 36-45 age group, as is shown in Figure 3.



50% 45% 40% 35% 30% 25% 20% 15% 10% 5.5% 5% 0% < 26 26-35 36-45 Age Category 46-55 56-65 20.2% 13.8% 22.9% 37.6%



Figure 3. Age Categories of Year One Drivers The survey participants in Year One also reflected a full range of experience in driving snowplows as is indicated in Figure 4.



26



Twenty-one percent had been driving for less than one year, 16 percent for one to two years, 21 percent for three to five years, and 25 percent six to ten years. There were also very experienced drivers. Among those reporting, 17 percent of drivers had been driving snowplows for more than 11 years.



50% 45% 40% 35% 30% 25%



25.0% 21.3% 21.3% 15.7%



20% 15% 10% 5% 0% <1 1-2 3-5 6-10 11-15 16-20 > 20 Drivers' Experience Level (years)



7.4% 3.7%



5.6%



Figure 4. Years of Experience with Driving Snowplows Even the recently-hired snowplow drivers had considerable experience driving other pieces of heavy equipment. There was only one respondent who was new to snowplows, and who also had limited experience with heavy equipment.



27



Drivers in the Year One survey were asked several basic questions about their experience with the simulator.



50% 45% 40% 35% 30% 25% 20%



31.2% 24.8% 20.2% 15.6%



15% 10% 5% 0% Easy Rather Easy Moderate Somewhat Demanding Demanding



8.3%



Level of Training "Demand"



Figure 5. Drivers Finding Training Demanding As the graph in Figure 5 shows, more than 28 percent of the drivers felt it was rather demanding to moderately demanding, while 40 percent felt that the simulator training was rather easy or very easy. The rest found the training moderately demanding. This seems to indicate the potential for adding more challenges to the scenarios included in the simulator-training program. In the Year One survey, 48 percent of the drivers rated themselves as "successful" in the simulator training and an additional 22 percent rated themselves "very successful" in the simulator training, as Figure 6 points out.



28



Figure 6 also indicates that only a small proportion of the drivers (less than 3 percent) felt that they were not successful in completing the training.

50% 45% 40% 35% 30% 25% 20% 15% 10% 5% 0% Not Successful Not Too Successful Moderately Successful Successful Very Successful 2.8% 6.5% 20.4% 22.2% 48.1%



Perceived Training Success



Figure 6. Drivers Feeling Successful/Unsuccessful in Completing Training Challenges Facing Snowplow Drivers Year One participants were asked to help to gage the relationship of the simulator training to their real world experience. They were asked to record up to three maneuvers associated with driving a snowplow that they felt were most challenging. There were a variety of maneuvers noted, but the largest number of drivers (44) felt that avoiding traffic that was crowding the plow was very challenging. Problems with limited visibility with snow/slush and whiteouts were underscored by 41 respondents. The SIPDE approach used in the L-3 simulator training did emphasize space awareness, particularly related to vehicles crowding the plow. Among survey respondents, 66 percent felt that the L-3 program focused an adequate amount of attention on responding to traffic crowding, while 34 percent felt more attention was needed. Only 33 percent of respondents felt that the L-3 program focused sufficiently on limited visibility and whiteout driving. In Year One, a major challenge pointed out by 17 of the drivers was dealing with obstacles, including disabled automobiles on the side of the road -- but 80 percent of them felt that the subject was adequately handled in the Driver Awareness (SIPDE) 29



training program. Of the 16 drivers who found city driving a major challenge, 66 percent felt that the subject had been adequately handled in the Driver Awareness training. Challenges that more than 75 percent of the driver respondents felt were not adequately addressed in the simulator training included real-world challenges, such as plowing snow-packed roads, plowing mountain roads with ice, negotiating switchbacks, and plowing more than 10 inches of snow. Reversing the plow, determining appropriate speed, and driving along shoulders were also were underscored by 50 percent of the drivers as needing more attention in the simulator training program. A variety of other issues were also identified in the 109 surveys returned in spring 2005. Many of these related to specific issues associated with controls on the plow, such as those used for wing plow and spreader operation. These operational issues were not covered in the basic SIPDE - Driver Awareness simulator training. Drivers were asked on the Year One survey whether the simulator training had responded to their needs. The following table documents their responses. Table 2. Drivers Finding Simulator Training Met Their Needs Years Driving Plow < 1 year 1-2 years 3-5 years 6-10 years 11-15 years 16-20 years > 20 years Total Perceived Responsiveness of Simulator Training (%) Yes Partially No 41.0 35.3 61.0 37.5 37.5 50.0 40.0 43.6 (45) 31.8 41.1 26.0 54.2 50.0 25.0 60.0 39.9 (41) 27.2 23.6 13.0 8.4 12.5 25.0 0.0 15.5 (17) Total 100 (22) 100 (17) 100 (23) 100 (24) 100 (8) 100 (4) 100 (5) 100 (103)



Note: Figures in parentheses are raw numbers of survey respondents in each category (not all drivers answered all questions).



Drivers' Recollections Of Training The primary recollections of 17 percent of drivers responding to the ASU Year One survey -- administered six weeks after the simulator training -- was that drivers were to be aware of surroundings when driving, and they were to choose the right speed when driving. Both subjects were heavily emphasized in the Driver Awareness training 30



program. In fact, increasing space awareness was the fundamental point made in the Driver Awareness training program. Eleven percent of the drivers recalled an emphasis on safety concerns, and 13 percent reported that they were instructed to drive a safe distance from other vehicles. There were other responses noted by a small group of respondents, and a substantial number of drivers left the question unanswered. Differences Between New And Experienced Drivers There were differences between the attitudes of experienced and less experienced drivers toward the simulator. Among the new drivers, 41 percent felt that the Driver Awareness simulator training addressed their needs, and 32 percent felt that it had partially responded to their needs. Another 27 percent of the new drivers, however, felt that the simulator training did not meet their needs. Like their more experienced colleagues, they underscored visibility and whiteout issues as their primary concern, but they were also concerned about safe braking distance, avoiding conflicts with tractor-trailers, avoiding guardrails, turning the plow, controlling the plow, stopping the plow, and staying in their lane with snow-pack conditions. Some were concerned about driving on mountain roads with switchbacks. Other issues involved plow lighting and wing plow operation. About 45 percent of the 109 respondents to the Year One mid-year survey felt the simulator training should focus on less experienced drivers, but 55 percent said that it should be directed to all drivers. Among those drivers hired by ADOT in the last two years, 60 percent said that the training should focus on all drivers, and only 40 percent of them felt that it should focus on new hires. Among drivers with three to five years experience, 74 percent agreed that the simulator training should be for all drivers. Conversely, 60 percent of the more experienced drivers (with more than five years of experience) felt that the Driver Awareness simulator training should be focused on less experienced drivers. In Year Two, drivers from Flagstaff, Kingman and Holbrook Districts were asked what they recalled from the simulator training the year before. As Table 3 shows, of the 19 respondents who indicated that they recalled the simulator-training course, nine recalled a focus on safety, six recalled the emphasis on being alert and aware, and four recalled an emphasis on speed related issues. Table 3. Driver Recall of 2004 Simulator Training as Reflected in 2006 Follow-Up Survey Training Issue Awareness Safety Speed Total Number of Respondents 6 9 4 19 31 Percentage of Total Respondents (%) 32 47 21 100



A more telling question in the Year Two multi-district survey asked what training drivers had used on the job. Driver respondents filled in a number of different points; several wrote in more than one response. As Table 4 shows, nine responses noted the awareness training, five responses indicated using the safety training, and four responses noted using the training on appropriate speed. Other drivers indicated they had used a variety of different aspects. Seven responses indicated not using any of the training on the job. Table 4. Year One Simulator Training Applied on the Job in Holbrook, Flagstaff, and Kingman Training Issue Awareness Safety Speed Knowledge of Route Visibility Use Mirrors Everything Avoid Fatigue Did Not Use Anything Total Number of Responses 9 5 4 2 2 2 2 1 7 34



Were they interested in more simulator training? An overwhelming 86 percent said they were. More than half of the respondents (54%) to this Year Two survey thought that there should be a program specifically for new drivers, up from 45% the year before. Clearly, there is continued interest in simulator training among snowplow drivers. Drivers in two more districts, Holbrook and Flagstaff, will have the opportunity to experience the more in-depth training in the 2006-07 snow season. FOCUS GROUP ACTIVITY The observations noted in the Year One multi-district survey were further developed in a series of focus groups conducted in spring 2005 with drivers in each of the districts participating in the fall 2004 simulator training program. Focus groups are regarded as an effective means of gaining in-depth observations reflecting attitudes regarding a product or policy. Used initially to improve the quality of products in production, the method is now widely used to stimulate citizen participation in making public policy. Sessions typically involve between five and nine individuals selected to reflect a range of perspectives on an issue. Focus groups are most effective in gaining information about how people think or feel about a topic, and why they hold certain opinions. They have been used effectively to improve the planning and design of 32



new programs and evaluating existing programs. They can also be used to determine whether a program is responding to individual or group needs, and what might help to make it more effective in reaching those needs (Krueger, 1994; Marcazak & Sewell, 2002; Morgan, 1988). The small group format enables all participants to be heard, and encourages all to share their ideas. Group dynamics contribute to the level and range of discussion as participants build upon the ideas presented by others. An objective facilitator encourages all to participate actively, keeps the discussion on target, and moves the discussion through an established agenda. Recorders note all observations. Discussions are taped and transcripts are made available for later assessment. Focus Group Sessions With Snowplow Drivers During the period of April 26 to May 24, 2005, the ASU research team conducted five focus group discussions with those who had participated in the first simulator training in December 2004 in Globe, Kingman, Flagstaff, Holbrook, and Safford. The objective was to gain the perspective of snowplow drivers on the L-3 simulator-training program. What emerged from the focus groups was a wealth of information not just on the specific simulator training, but also a fuller understanding of the multi-tasking aspect of driving and the challenging weather environment faced by snowplow drivers. Since drivers were also asked in the focus groups to design a simulator program that would meet their needs, a clearer idea of needs and expectations also emerged. In Year One, parallel focus group discussions with drivers who had not been simulator trained also proved to be valuable. These "control groups" helped to gain a clearer idea of the traditional training process and issues that they felt should be addressed in simulator training. In Year One, follow-up discussions were conducted with ADOT maintenance engineers, maintenance superintendents, and ORG supervisors, where possible. In Year Two, all ORG supervisors in the Globe District participated in a separate focus group and offered their expectations and assessment of the simulator programs.



33



Year One Focus Groups Table 5 summarizes the focus group participation and numbers of interviews with supervisors at each location in Year One. Each focus group with those who took the simulator training lasted about one hour, during which time 10 questions were asked (see Appendix C). Results of these focus groups are described below, along with comments from supervisors and engineers. Table 5. Year One Focus Group/Interview Participants Number of Participants 4 13 6 6 9 10 Plowing Experience (years) 0.5-15 0.5-16 0.5-11.5 0.5?17 0.5?35 0.5 -18 Number of Supervisors or Engineers Interviewed 1 2 1 1 2



District Globe Kingman Flagstaff Holbrook Safford



Date April 26, 2005 May 17, 2005 May 18, 2005 May 19, 2005 May 24, 2005



General Comments From The Year One Focus Groups Although the results of the surveys administered by this team indicated widespread support for the simulator training, the focus groups voiced more mixed results. Drivers overwhelmingly agreed that they received too little time on the simulator in December 2004. The L-3 training program called for 2-1/2 hours of training with an interspersed classroom and simulator experience. Each topic was to be presented in the classroom and then reinforced by driving in the simulator. Although the basic L-3 training course on the SIPDE approach was to include 30 to 45 minutes of driving in the simulator with 105 to 120 minutes of class room experience, technical difficulties and weather conditions led to reduced exposure to the simulator in several settings, as previously described. Some drivers who participated in the focus groups said that they spent no more than 20 minutes in the simulator. Some also felt that because the simulator lacked many critical, realistic features (see comments below), its value as a training tool was less than it might have been. Some drivers were clearly negative. Those drivers in the "control group" focus group sessions highlighted variation in the traditional ride-along training. Some had the benefit of a number of on-the-job training (OJT) ride-alongs in dry and snowfall situations. They were allowed to "solo" when they felt comfortable with the tasks involved. However, since a number of districts are shorthanded, the ride-along training is sometimes abbreviated. A task identified for the simulator is to reinforce this ride-along training, and give drivers more time behind the wheel. The control groups, made up largely of experienced drivers, emphasized the multi-tasking involved with snowplow driving and the serious issues involved with whiteout conditions. 34



Drivers who had participated in simulator training generally agreed that even the small amount of simulator experience they received in December 2004 did raise their level of awareness. According to the drivers, the simulator training "opens your eyes" and "makes you think." On this point, the supervisors and engineers interviewed agreed. One noted that although some more-experienced drivers thought the simulator was "cheesy," some of them did crash during the driving scenarios due to poor decisions. While these skeptical drivers may not have realized the benefits of the simulator training, they (apparently) were learning, nonetheless. Of course, the timing of such "eye openers" is critical. In the 2004-05 snow season, because of early snow fall and some delays in the simulator training, drivers had already been plowing snow before they received the simulator training. There was also general optimism for the potential of a driving simulator (although this optimism was not unanimous). Focus group participants repeatedly expressed their desire for a more "realistic" simulator, one that would better meet their needs as a training tool. It should also be noted that some drivers who expressed rather negative opinions of the simulator itself, had nothing but praise for the classroom portion of the training (i.e., the SIPDE method used by the L-3 trainer). Drivers specifically mentioned the emphasis on safe following distance and speed-awareness considerations as particularly useful. Drivers (and supervisors) had plenty of suggestions for how the simulator might be modified and incorporated into the ADOT driver-training program, as described in the following sections. Year One Consensus In Year One, drivers repeatedly cited the simulator's lack of realism as its single greatest shortcoming. According to these drivers, the designers of the simulator needed to "ridealong" in plow trucks under severe conditions in Arizona. The general consensus was that the people designing the L-3 simulator were not aware of the severe real-world conditions with which Arizona plow drivers regularly contend. When drivers were asked which features they would include if they could design the simulator themselves, their responses most often had to do with realistic controls and displays. As one driver put it, "make it as real as you can make it." Another summed up the feelings of many when he said, "if we're gonna be in the simulator, it better match our trucks." Among the many features requested, it was expressed that the simulator should incorporate the following operational features: ? ? ? Plow controls (which vary according to the various truck interiors). Vehicle instruments (radio, defroster, etc.). Gear shifting (according to one driver, this was quite difficult to learn, and distracted him from the many other more critical -- and potentially dangerous -- tasks). This was addressed in the Fuel Management training described in Chapter VII.



35



?



Spreader controls (there are at least three different types in use, and as one driver noted, the learning curve associated with the spreader controls is actually steeper than that of the hydraulic plow controls). Washer fluid control/meter (according to the drivers, this is something you forget only once -- it's that important). Road temperature gauge (which is used to make critical decisions about when to apply de-icer and in what quantity).



? ?



Essentially, what these drivers wanted was an actual truck cab, with changeable controls (e.g., different spreader controls, hydraulic levers, etc.). Some simulators on the market do use actual vehicle cabs, but are generally the much more expensive motion-base units. Visibility -- or the lack thereof -- was perhaps the area of greatest criticism for the Year One training simulator. According to the drivers, this is the single most stressful part of the job -- the fact that visibility is often so poor that it is impossible to determine which side of the road one is actually driving on. The overwhelming consensus was that in all simulator scenarios the visibility was far too good, compared to real-world snowplow driving conditions. According to the drivers, environmental conditions should be more severe (e.g., darker nighttime driving, more blinding whiteouts, etc.). In addition, vehiclerelated visibility conditions should also be more severe (e.g., frosted/fogged up windshields, frosted mirrors, frozen wipers, etc.). Some of these changes were incorporated into the Year Two simulator experience. Driving Scenarios A variety of more appropriate driving scenarios was also an important consideration for the drivers who participated in the focus groups. Drivers on rural routes, for example, saw limited benefit in the "in town" scenarios that were part of the December 2004 SIPDE training program. Likewise, drivers who never plow highways said that they learned little from the highway portions of the simulator training. Drivers consistently reported that they would like to see simulator scenarios that reflect the actual ADOT routes with which they are familiar -- especially those considered particularly hazardous. They suggested several scenarios that would be more beneficial, including: ? ? ? ? ? Cars passing on both sides of the plow truck; according to drivers, large trucks will often pass at high speed. Predictable road hazards (railroad tracks, expansion joints, cattle guards, etc.) that are generally known on familiar routes are potentially dangerous on unfamiliar routes. Unexpected road hazards (e.g., snow-covered rock in a curve). "Getting sucked into the cut" (driving slightly off the shoulder of the road -- and struggling to get the vehicle safely back onto the road surface). Various weather and road conditions, other than snow (e.g., rain, sleet, hail, black ice, etc.); also weather changes based on temperature (air and road), altitude, etc.2



2



It was interesting to note that drivers often use the skies to determine their approach to de-icing. If the skies begin to clear up, for example, they know that the temperature will soon drop and it is important to put down anti-icing chemicals before the road surface freezes.



36



For many drivers, the feeling of the simulator didn't reflect the real world. For example, although the simulator included downhill scenarios, there was no feeling of going downhill; only visual input was available to indicate the pitch angle of the truck (as the simulator used was a fixed-base model). As it turns out, drivers often use senses other than visual for driving and plowing in the winter season. Drivers generally "shift by ear," for example, using the sound of the engine rather than the tachometer to judge engine speed. More experienced drivers can often smell the carbide wear bits when friction levels are high, and thus make adjustments (e.g., raise the plow slightly) that will prolong the life of the bits. And many drivers reported a variety of "drive by feel" tactics (using rumble strips as a way to find the shoulder of a snow-covered road, gently riding the guardrails with the edge of the plow blade to maintain consistent lane position, etc.). All of these tactics are important aspects of the snowplowing activity, and yet none was incorporated into the simulator scenarios used in the 2004-05 training period, due to current software and hardware limitations. Distractions In addition to the various individual features and driving scenarios requested, the drivers were very clear that snowplow operation is a continuous series of "distractions." A driver may be struggling to clear the windshield while downshifting, answering a radio call, and monitoring the temperature gauge, for example. In the real world, no single driving activity happens independently. By contrast, as one driver pointed out, "there [are] just absolutely no distractions in the simulator." The "distractions" are what make the snowplowing job so demanding -- and so rewarding for these drivers. One driver talked about the overall soreness he feels after long shifts plowing snow, while many drivers talked about the fatigue that comes with such severe driving demands. Drivers were quite skeptical that driving fatigue could be simulated, but suggested that it would be an important aspect of realistic driving simulation. Some drivers suggested that it would be useful to begin simulator scenarios in a very simple form, and add more distractions as the trainee becomes more comfortable and more expert. Of course, this is similar to what was done in December 2004. What was lacking was the wide range of distractions that could be included. Interestingly, this corresponds closely to what is called "part training,"3 in which a whole task is mastered by learning its constituent (and presumably, more easily learned) "parts." Time In The Simulator Drivers were asked how much time they thought should be spent in the simulator -- assuming the simulator could be "improved" to reflect their suggestions. Responses varied widely, from a low of 15 minutes to a high of five hours. The majority of the drivers suggested a four-hour training program would be appropriate. Inexperienced Drivers The general consensus in Year One was that new drivers would benefit from more time in the simulator, while experienced drivers would require only a brief pre-winter

3



See, for example, Goldstein, I. L. (1986). Training In Organizations: Needs Assessment, Development, And Evaluation (2nd ed.). Monterey, CA: Brooks/Cole Pub. Co.



37



"refresher." However, it is interesting to note that these same drivers also suggested that even the best drivers might have serious problems in severe conditions (e.g., nighttime whiteout), so experience is not necessarily an adequate predictor of success (in the simulator or the real world). Also of interest, the driver responses were quite different from those of the supervisors on this point. Some supervisors suggested that up to 40 hours of training may be useful, while emphasizing its importance for new drivers. Supervisor Perspective In Year One In general, the maintenance engineers, maintenance superintendents, and ORG supervisors were optimistic about the potential benefits of the driving simulator. Although they saw much room for improvement (their comments regarding the need for greater realism echoed those from the drivers), they recognized that the simulator -- despite its shortcomings -- is effective at "getting [drivers] to think about things."4 Part of the simulator's potential benefit lies in its ability to generate scenarios that become increasingly difficult, eventually approaching the real-world hazards of snowplowing. There are benefits to making the simulator "difficult." According to one supervisor, if a driver goes into a curve too fast, he ought to lose control and crash. This would illustrate the importance of thinking and planning ahead for drivers. One supervisor suggested that he would be very pleased if drivers crashed quite frequently in the simulator, and were (as a result) accident-free on the road. Simulator training may be even more important in states like Arizona where snowfalls are less frequent. Here, even a five-year veteran ADOT driver may have seen only a dozen significant snowstorms. In one ORG, a driver noted that he only plowed twice last snow season and felt really out of practice. Effective simulator training would keep drivers fresh. Also, it was reported that some newer drivers prefer not to plow snow, since they are intimidated by the difficulties associated with the job. However, it was suggested that if these drivers were prepared somewhat by the simulator, this issue might be diminished. TASK ANALYSIS DRIVER BEHAVIORAL MODEL Based on the Year One snowplow operator focus groups conducted in Globe, Kingman, Flagstaff, Holbrook, and Safford, the various "operator activities" were sorted into five major categories: inspecting, communicating, driving, plowing, and spreading. Michon's (1985) driving model was used as a framework into which each activity could be placed (see Table 6). The description of Michon's driving model provided by Wickens, Gordon, & Liu (1998) is especially useful, and worth quoting fully: Three levels of activity describe the complex set of tasks that comprise driving-- strategic, tactical, and control [STC model]... Strategic tasks focus on the purpose of the trip and the driver's overall goals; many of these tasks occur before we even get into the car. Strategic tasks include the general process of deciding where to go, when to go, and how to get there... Tactical tasks focus on the choice of

4



This may be related to Michon's hierarchy of driving skills: Strategic, Tactical, and Operational levels. See Michon, J. A. (1985). A Critical View of Driver Behavior Models: What Do We Know, What Should We Do? In Human Behavior and Traffic Safety (pp. 485-520), Plenum Press. See Table 6 for details.



38



maneuvers and immediate goals in getting to a destination. They include speed selection, the decision to pass another vehicle, and the choice of lanes... Control tasks focus on the moment-to-moment operation of the vehicle. These tasks include maintaining a desired speed, keeping the desired distance from the car ahead, and keeping the car in the lane. (p. 438) This type of STC analysis provides a necessary framework for evaluating which skills are best trained in the simulator, and which skills are better trained using other means (and how different skills may be transferred to the real world), as described in Chapters VIII and IX. It is also useful for evaluating training programs and policies related to snowplow operations. For example, the Strategic level corresponds to ADOT's snow policies, as they relate to the larger ADOT mission. The simulator can be used to ensure that Tactical- and Control-level skills and behaviors correspond with ADOT snow policies. Table 6 illustrates this framework.



39



Table 6: Snowplow Operator Activities and Michon's5 Driver Behavior Model Levels of Driving Skills Activities of Snowplow Operators Inspecting (pre & post trip; & while plowing) Strategic (Planning) N/A Tactical (Maneuvering) N/A Control (Operational) ? Vehicle (hydraulic lines, tires, lights, etc.) ? Snow removal equipment (wear bits, frame bolts, de-icing material, etc.) ? Adjust radio volume ? Locate and key radio microphone



Communicating



Driving



? Broad ADOT policies (e.g., public safety) ? District Snow Plan policies ? Receive orders from Snow Desk N/A



? Contact other ADOT drivers ? Assist other drivers (ADOT, Department of Public Safety (DPS), the public) ? NavigationAvoidance (other drivers, known objects, unknown objects) ? Monitor speed (by ear)



Plowing



N/A



Spreading



N/A



? Aiming (height, angle -- function of vehicle speed) ? Avoidance (expansion joints, railroad tracks, etc.) ? Monitor road temperature (gages, weather stations, and skies) ? Monitor material (salt or MgCl)



? Navigation-Aim (apply brake and gas pedals, steering inputs, etc.) ? Shifting gears and using clutch. ? Visibility (heater & defroster controls, wipers, mirrors, etc.) ? Adjust height and angle of plows (main and wing)



? Adjust spreader controls



5



From Michon, J. A. (1985). "A Critical View of Driver Behavior Models: What Do We Know, What Should We Do?" In Human Behavior and Traffic Safety (pp. 485-520), Plenum Press.



40



IV. SIMULATOR TRAINING - GLOBE DISTRICT: YEAR TWO

Year Two Simulator Training As noted previously, the focus of Year Two was on the first ADOT-owned simulator unit, newly installed in the Globe District (Figure 7). In the 2005-06 winter snow season, all drivers in the Globe District had a full four hours of training, with a combination of classroom and simulator seat time. The classes involved small groups of two or four drivers, allowing all trainees to spend about 45 minutes on the simulator. Classroom training time was reinforced, as each driver spent time in the simulator working on three increasingly difficult scenarios. The ASU team members observed several full training sessions in Globe, including both simulator and classroom activities.



Figure 7. Globe Simulator-- Fall 2005 Training In spring 2006, the Globe drivers participated in a second simulator training session, which was focused on fuel management and gear shifting. An assessment of that training program is included in Chapter VII of this report. The current chapter focuses primarily on the snowplow driver-training program offered to Globe District crews in fall 2005.



41



Figure 8. Simulator in Use -- Hands-on Training Key Changes for Year Two Training The primary change in the Year Two simulator course, as noted previously, was that the training was offered by experienced snowplow drivers from the Globe District. These trainers were able to bring their real-world experience from around the district to the other drivers. Their enthusiasm and willingness to share techniques really made the program come alive for the Globe drivers. The fall 2005 training program in Globe was based on that offered by L-3, but the District trainers made changes in the L-3 PowerPoint course material to better reflect local issues. For example, since there are no freeways in the Globe District, discussions of plowing in the gore points were replaced by references to traffic signals and signs, which are important issues to emphasize in the district's small communities. Trainees were also urged to be alert to truck load weights and grades, given the narrow, winding mountain roads in the district. In addition to the district-specific issues added to the curriculum, the Year Two training reflected some general issues of concern to ADOT. For example, Globe drivers were urged to follow the CDL checklist in reviewing equipment before operating it. There were also additional slides emphasizing the need for adherence to proper braking techniques and for communication among drivers plowing in tandem. 42



Figure 9. Globe `Train-the-Trainer' Session -- August 2005 ASU's Evaluation Approach As a parallel to the assessment conducted in Year One, the ASU study team: ? ? Reviewed the end-of-session survey conducted by Globe. Conducted a more in-depth survey of driver participants in the snowplow simulator study in April 2006 (after a late snow fall). An additional part of that survey related to the fuel management/gear shifting training program, as is discussed in chapter VII. Conducted a series of four focus groups at the end of 2005-06 snow season. All participants in the Globe District focus groups had participated in snowplow simulator training. Conducted a focus group with representatives from each of the maintenance ORGs in the Globe District.



?



?



Driver Awareness Post-Session Exit Survey In 2005, the same exit survey was distributed to driver participants in Globe as had been distributed the year before (Appendix A). The responses were predictably enthusiastic. 43



However, the drivers also seemed to be a bit more discerning, since most of them had also been through the training in 2004. One driver even commented, "This was better than the first time." Several reported enthusiastically that they appreciated working with the experienced snowplow driver instructors this year. Among the respondents, 58 percent checked "agreed" for every question, a somewhat lower rate than in Year One. However, 88 percent of respondents agreed that the program did convey its primary objective, focusing on awareness and hazard avoidance. Twelve percent felt that the class lecture time was too long, while six percent felt the simulator seat time was too short Several presented ideas for course improvements, such as accounting for the width of the plow in the scenarios, making more scenarios reflect "our routes," and making more scenarios with "night whiteouts." Overall, however, the comments were positive. As one driver put it, "This was a wellrounded course. I believe it will be helpful for the upcoming season." Another added, "This is a very good simulator." There were those, however, who felt that they did not get enough trainer feedback. ASU Spring 2006 Globe Survey As was indicated previously, in April 2006, a separate survey was sent to snowplow operators in Globe, all of whom had been through the fall 2005 Driver Awareness training program. Among those 61 drivers, 49 returned usable survey questionnaires. This survey instrument (Appendix A) was very similar to the one that was distributed to all districts in Year One. The intention was to note any differences between what was learned in each of the two years, and its potential for application to the real world. As noted, there were several distinct changes in Year Two in Globe in terms of training approaches: using local trainers and providing more classroom and simulator time, for example. Also, the class PowerPoint slides included more references to local conditions in Globe. Globe District participants completing the Year Two questionnaire included drivers with varying levels of expertise in snow removal.



44



As the graph in Figure 10 shows, 45 percent of the respondents had two years' experience or less, while 10 percent had over 16 years' experience. The median was about five years' experience on snowplows. Nevertheless, more than 75 percent of these drivers had more than five years' of experience in driving various other types of heavy equipment.



50%



40%



30%



26.6%



28.6%



20%



18.4% 12.2%



10% 6.1% 4.1% 2.0% 0% <1 Year 1-2 Years 3-5 Years 6-10 Years 11-15 Years 16-20 Years > 20 Experience Level of Drivers (Years)



Figure 10. Years of Experience with Driving Snowplows -- Year Two, Globe District



45



Figure 11 shows the age range of drivers in the Globe District in Year Two. There is a slightly lower proportion of younger and of older drivers in the Globe District, as compared with the other districts in Year One. Nevertheless, the largest proportion of drivers is in the 36 to 55 range.



50% 44.8% 40% Year 1 Year 2 37.6%



30%



27.6% 22.9% 20.2% 20.7% 13.8%



20%



10% 5.5% 3.4% 0% < 26 26-35 36-45 Age of Drivers (years) 46-55 56-65 3.4%



Figure 11. Age Categories of Drivers -- Globe District The Year Two survey asked drivers in the Globe ORGs to note the aspects of snowplow driving that they found most challenging. They were also asked to note which of these topics had been covered in the simulator training. The chart in Figure 12 presents their collective observations. The white bars indicate the proportion of drivers who identified specific driving challenges, while the dark bars indicate the proportion of respondents who felt those specific challenges were fully addressed in the simulator training program.



46



Drivers felt as though the challenges associated with the weather were addressed, and a number felt issues associated with pedestrians were adequately addressed. Fifteen percent of respondents felt that none of their real-word challenges were addressed, as Figure 12 shows. The concerns that they listed were primarily related to operating the plow itself, and real-world roadway conditions in the Globe District. Neither of those issues is part of the' awareness' simulator training program.



25% Challenges Identified

21.0%



Challenges Met



20%



19.0%



15%



15.0%



15.0%



10%

8.0% 8.0% 5.0% 4.0% 3.0% 1.5% 2.0% 8.0%



9.0%



5%



4.0%



4.0% 3.0%



5.0% 5.0%



1.5% 0.0%



0%

ffi c ue r dw ay ur n ra t io Fa tig Tr a si b st ri Sp W ea U -T R oa Vi es o U nr lv ed ee d ilit y ns s s th e an de



Pe



Driving Challenges



Figure 12. Challenges Facing Year Two Snowplow Drivers In the Year Two survey, Globe drivers listed very similar real-world challenges to those they had listed on the Year One survey -- visibility, dealing with traffic, and roadway conditions (including guard rails, cattle guards, winding mountain roads, and ice on the roadways). Fewer than half of the respondents felt that issues associated with traffic and visibility were adequately addressed. In the future, perhaps more class and simulator time could be devoted to these issues that drivers report as most serious in the real world.



47



O



pe



The drivers were asked about aspects of the driver simulation training that they had been able to use on the job. Several indicated that they used more than one aspect on the job. The chart in Figure 13 summarizes these observations.



25% 25.9% 22.2% 20%



15%



10% 7.4% 5.6% 5%



9.3% 5.6% 3.7% 3.7% 3.7% 1.9% 3.7% 1.9% 3.7%



0%

in g m un ic at in g W ea th er Ev er yt hi ng N ot hi ng Sa fe ty en es s s Is su es ee d in e es t az ar d en te rL ev er s C et R Sp riv D in g



ar



H



G



in g



ig ht



Aw



C



riv



R



Concepts Applied on the Job



Figure 13. Training Concepts Applied on the Job by Year Two Globe Drivers Since drivers were encouraged to list multiple aspects used on the job, the graph will reflect numbers higher than the 49 respondents to the survey. The fact that 26 percent of the responses referred to awareness is understandable, since that was the main focus of the Driver Awareness course. Drivers also mentioned the related issues of safety and hazard avoidance. While 6 percent of the responses indicated that the drivers used all aspects of the course on the job, another 23 percent of the responses indicated that the drivers had not used any of the Driver Awareness (SIPDE) training on the job. They felt that it was not specific enough to address their issues. Of course, it is difficult to gauge how much of the Driver Awareness the drivers actually did use on the job. The researchers only heard anecdotal reports from young drivers who said they had used the training, and that it had made them more confident on the job. The expectation of the trainers is that awareness is so much a part of the job that employing the SIPDE concepts will become almost automatic for the drivers.



D



48



om



N



Most drivers were relatively satisfied with their simulator experience. The majority felt that the four-hour classroom/simulator training in Driver Awareness was adequate. Among the Globe drivers in Year Two, 20 percent found that the simulator training was somewhat demanding, while 36 percent found it not very demanding or not demanding at all. These percentages mirror those for the multi-district survey the year before. Among the Year Two Globe respondents, 98 percent felt that they were relatively successful in the simulator training. The trainers did encourage all participants, by indicating that they were making progress through the scenarios. There was, however, limited individual feedback and no written report to actually confirm the drivers' perspective about their success in the simulator. Fuel Management Training The Year Two survey also included questions for drivers who had participated in the Fuel Management training course in Globe. More than 50 percent indicated their interest in the gear-shifting portion of the course. However, those driving automatic transmissions could not directly apply the training, unless they were reassigned to a truck with standard transmission or "loaned out" to another district where only a manual truck was available. When asked which aspects of the training they would be able to apply while driving a plow, 48 percent indicated they would use proper shifting techniques. Only 13 percent noted their interest in overall fuel management, and only six percent indicated that they would try to apply fuel management techniques while also dealing with the challenges of driving a plow. Fuel management was actually a secondary emphasis for the course that was primarily focused on driving techniques. A title such as Driving Skills would more accurately describe the course. This training program is discussed in more depth in Chapter VII of this report. Year Two Focus Groups in the Globe District On June 5 and 6, 2006, focus group sessions were conducted in the Globe District. The goal was to gain a perspective on the Year Two simulator training program, which included longer exposure to the simulator, and training programs conducted by experienced snowplow drivers from the Globe District. As noted previously, the training sessions in Globe were four hours in length, and included both classroom time and simulator time (which included three increasingly difficult scenarios). Each driver had a total of about 45 minutes on the simulator in the snowplow Driver Awareness program. Focus group questions about the length of the program seemed to confirm the point made in the mid-year survey that the four-hour program was about the right length. Four focus groups were held with the drivers who had participated in the Globe District simulator training sessions. Two groups included drivers that represented the ORGs closest to Globe, while two other groups represented the ORGs closer to the neighboring town of Show Low. While more of the drivers based around Globe used trucks with automatic transmissions, the majority of drivers in the higher-elevation Show Low region drove trucks with standard transmissions. Each group included six drivers with a mix of new hires and experienced drivers. An additional focus group involved supervisors from all the ORGs in the Globe District. 49



All Globe trainees (in either of the Year Two simulator training programs) were enthusiastic about the quality of the instructors from their own district. The drivers swapped stories with, and eagerly learned from, the experience of those who knew their specific "real world." A total of 24 drivers participated in the four focus groups. All of these drivers had gone through both the Driver Awareness (SIPDE) training in the fall and the Fuel Management training in the spring. The fall course was similar to the one offered in Year One, but the program in Globe was longer, consistent for all drivers, and taught by very experienced snowplow operators. The spring simulator-training program on Fuel Management and shifting, as indicated above, was also four hours in length. It, too, included a combination of classroom and simulator training, and was taught by the same experienced heavy truck operators from the Globe District. There was an overwhelming optimism among drivers for the potential of the simulator. However, this optimism seemed to be reduced from Year One. The drivers seemed to be somewhat frustrated that the simulator was not being used to its fullest potential and that the current curriculum did not address the real world operational issues that they were dealing with. Experienced drivers repeatedly voiced the opinion that the fall simulator awareness training was of value really only to new hires. They felt that, as veteran drivers, they did not experience anything they did not already know from real-world experience. New hires, however, were enthusiastic, indicating that the training gave them a jumpstart on the season. They were interested in having more practice time on the simulator before they faced the challenges of the real world. Experienced drivers continued to raise issues, first brought up in Year One, associated with operational controls. The simulator does not help with the challenges of backing up, turning the plow around, driving with a wing plow, handling the sander, or applying deicing chemicals, as they pointed out. One experienced driver noted that the simulator also lacked realistic controls that would raise and lower the plow blade. This made it difficult to train for the real world, where it is ADOT policy to raise the blade while crossing railroad tracks and cattle guards. Although the trainers do remind drivers of t