Sat, 13 Mar 2010 10:16:57 +0000

As the reviews and table show, there have been quite a number of program evaluations over recent years, but the findings are quite variable and the question of whether driver education programs enhance or detract from safety remains highly controversial. Findings to support either side can be found, although the larger number of studies fall on the negative side. No one study design is perfect, and progress will likely develop on a “weight-of-evidence” basis over numerous studies of different types.

There are also many other questions that need to be considered. Do some types of driver education programs lead to better educational and safety outcomes than others? Can we identify which components of driver education programs do and do not work? Do programs meet their learning objectives? How can driver education programs be improved in order to yield safer young drivers? These questions have been left partially or completely unanswered. Methodological weaknesses have plagued past evaluations, and a lack of systematic research progress has held back the development of more effective programs.

Reviews of Driver Education Evaluations

In the last decade or so, quite a number of broad scholarly reviews of driver education evaluation have appeared; seemingly outnumbering the evaluation studies themselves (Christie 2001; Engström et al. 2003; Lonero et al. 1994, 1995; Mayhew and Simpson 1997; Mayhew, Simpson and Robinson 2002; McNeil in press; Nichols 2003; Siegrist 2003; Smiley, Lonero, and Chipman 2004; Woolley 2000). These reviews have usually addressed driver education in conjunction with other forms of driver instruction or graduated driver licensing (GDL). In addition to the broader scholarly/analytic reviews; two “systematic” reviews ( Roberts and Kwan 2004; Vernick et al. 1999 ) , and a single quantitative meta-analysis (Elvik & Vaa, 2004) have appeared.

In the most inclusive and detailed of the earlier reviews, Mayhew and Simpson (1997) set up a score card of findings: 7 studies showed a positive effect, 16 showed no effect, and about 7 others showed a negative safety effect. The authors, consistent with others reviewers, conclude that, on balance, the literature is negative toward driver education effectiveness.

The Elvik and Vaa meta-analysis is potentially the most interesting of all, but unfortunately, the only report on the meta-analysis is a very brief one in the 2004 edition of the Norwegian Handbook of Road Safety Measures, although it is available in English. The combined data of 16 studies indicated that driver education graduates have 1.9% fewer crashes per driver (confidence interval, -3.8%; 0%). The overall difference is borderline statistically significant, as the combined results had the power to detect a significant difference (α=.05) if the result would have been 2% (rather than 1.9%). Per kilometer driven graduates had 4% lower crash rate (6%; -2%). When the combined results were limited to the experimental studies, a different picture emerged. No difference per driver surfaced (+/- 4%). Per kilometer driven, graduates had 11% more crashes (+8%; +15%). The authors conclude that the combined evaluation results do not indicate that driver education reduces crashes over the first couple of years of driving.

All the earlier reviews were addressed to finding whether the evaluation literature suggests that driver education “works.” As part of their methodologically oriented review, Clinton and Lonero (2006a) categorized the earlier reviews and discussed critically the reviews’ conclusions and limitations. There is a strong trend toward systematic review, particularly in the health fields, but the often narrow orientation of this approach to research review seems to limit its applicability to driver education evaluation. The more conventional scholarly reviews vary greatly in scope and quality. Some point to ways to improve driver education, but they all failed to provide much critique of the evaluations, and they do not provide much guidance as to how evaluation research might be improved.

Improving Driver Education Evaluation

A serious lack of statistical and research design sophistication was evident in the early evaluation studies (Nichols 2003). Many of the later studies have used better designs and statistical methods. No perfect study exists. Even the “gold standard” random controlled trials have had weakness in terms of maintaining clean assignment of students to training conditions and clear comparisons between assigned groups.

In the mid-1980s, the OECD assessed the efficacy of road safety education programs and provided strategies for program development, implementation, and evaluation (OECD 1986). This report outlined issues of effectiveness for program planning and implementation, but its primary focus was evaluation. It was suggested that a program should be seen as effective if it does what it was intended to do, and that it is, therefore, very important to be explicit about educational objectives, which should include measures of intermediate effects and not be solely focused on collisions. Not much happened in response to the OECD’s astute directions.

With notable exceptions, such as the DeKalb experiment, Dreyer and Janke’s 1979 experimental study, and Gregersen’s 1994 survey study, most evaluations have failed to look at intermediate outcomes. Lacking information on what the students have or have not learned, directions for program improvement are left unclear. Most existing evaluations leave many unanswered questions regarding:

• Logical links between curricula and young drivers’ needs;
• Theories explaining how a program is expected to achieve safer driving;
• The quality, comprehensibility, and usability of curriculum products;
• How well and how consistently the instructional processes actually deliver the intended learning; and
• Which learning and behavioral outcomes result, or fail to result, from the training.

Tracking of learning outcomes is an area where programs could easily build in ongoing evaluation. Knowledge and attitude measures at the beginning and end of the course and at later intervals would help keep contact with graduates and provide feedback for continuous improvement of curriculum and delivery.

Exposure differences are too often ignored in driver education evaluation. In general, simple differences in the amount of driving (exposure to risk) account for much of the differences in crash rates between groups of drivers (Chipman 1982). Qualitative exposure differences, such presence of passengers and trip purposes, also represent different collision risks for young drivers (Preusser 1996; Preusser, Ferguson, and Williams 1997; Williams, 2003).

A key implication of the evaluation literature is that there is no perfect method for evaluation of beginner driver education. A broader and more systematic evaluation approach is needed, which includes various study designs, a wide range of output and outcome measures, and a variety of carefully planned comparisons. Perhaps it is time the “gold standard” title is passed along to comprehensive, systematic evaluation that includes various approaches to developing the whole picture of program effects and improving programs.

The final implication of past evaluations of driver education is how little they have contributed to developing and improving the programs. What has occurred has been very unsystematic; no study seemed to relate to or build upon earlier studies. Even credible positive findings, such as those of Dreyer and Janke (1979), were not been followed up. Scientific knowledge usually develops through systematic replication of research, but that has not yet happened in driver education research. Earlier thinking, including that behind the DeKalb project, seemed to favor a single massive development effort to achieve driver education’s safety goals in a one big step. More recently, however, researchers and theorists have emphasized the importance of incrementally building knowledge gains and other intermediate effects, (Lonero et al. 1994; Woolley 2000), as well as continuously developing and improving programs. Keskinen et al. wrote: “We have decided … that the development of driver education will take place in short steps, with constant evaluations of the results and trying to avoid solutions which are thought to be final” (1998, p. 382).

Support for new and more systematic evaluation research has been made available. Projects are being undertaken at the National Highway Traffic Safety Administration (NHTSA). Under funding from the AAA Foundation for Traffic Safety and the National Center for Injury Prevention and Control of the Centers for Disease Control, Northport Associates and the Traffic Injury Research Foundation are implementing a program of ongoing, related and systematic evaluations of a number of driver education programs during 2007-2009. The program initially focuses on three jurisdictions – Michigan, Oregon and Manitoba, with other sites intended to be added as broader financial support is found for the Large Scale Evaluation of Driver Education (LSEDE).

Summary and Conclusions

Reviewers of the evaluation literature have typically concluded that beginner driver education has yet to demonstrate clear success in improving the safety of new drivers who receive it, or of youth on a population basis. A few studies have shown positive effects, and not only poorly designed studies. Keys to long-term effectiveness of driver education, if it is to make a net contribution to safety, are likely to be astute evaluation and continuous improvement of program content, delivery, organization, regulation and coordination with other healthy influences.

If driver education is to achieve success in its demanding mission, it needs to be more firmly based, on the one hand, in sound research and theory concerning young driver skills, behavior, motivation, and risk, and on the other in the principles of effective behavior change. Program evaluation is critical for more effective program development in the future, but its limitations in the past must be recognized and corrected for a beneficial effect. A more comprehensive approach to evaluation needs to address program theory, products, processes, and management.

Evaluating intermediate outcomes – changes in behavior, knowledge, attitudes, and exposure to risk – is also required. Intermediate measures should continue during the follow-up period if we are to have a clear picture of the effects of the program and the reasons for them. Meeting the ultimate goal of reducing novice drivers’ serious crashes will also require better management the linkage of driver education with parental and community influences, graduated licensing, and other behavioral influences such as incentives and disincentives. The increasing need, vigorous development in public and private programs, clearer guidelines for evaluation, and support for more systematic, ongoing evaluation research all point toward data-driven development of better driver education programs and ultimately improved safety outcomes.

REFERENCES

Carstensen, G. (2002). The effect on accident risk of a change in driver education in Denmark. Accident Analysis and Prevention 34(1):111-121.

Chipman, M.L. (1982). The role of exposure, experience and demerit point levels in the risk of collision. Accident Analysis and Prevention 14(6):475-483.

Christie, R. (2001). The effectiveness of driver training as a road safety measure: a review of the literature. Melbourne, Australia: Royal Automobile Club of Victoria (RACV) Ltd., Public Policy Group.

Christie, R. (2002). Road safety education and training from a public health perspective. Road Safety Research, Policing and Education Conference Proceedings.

Clinton, K. & Lonero, L. (2006a) Evaluation of Driver Education: Comprehensive Guidelines, Washington, D.C.: AAA Foundation for Traffic Safety.

Clinton, K. & Lonero, L. (2006b) Evaluation of Driver Education: How-to Guide, Washington, D.C.: AAA Foundation for Traffic Safety.

Davis, C.S. (1990). The DeKalb County, Georgia, Driver Education Demonstration Project: Analysis of its long-term effect. Washington, DC: U.S Department of Transportation, National Highway Traffic Safety Administration.

Dreyer, D., and M. Janke. (1979). The effects of range versus nonrange driver training on the accident and conviction frequencies of young drivers. Accident Analysis and Prevention 11:179-198.

Road Safety Educators’ Association – Driving School Association of the Americas (2005). Process for Curriculum Review. Ontario, Canada: DSAA and RSEA.

Elvik, R., and T. Vaa, eds. (2004). The Handbook of Road Safety Measures. Amsterdam: Elsevier.

Engström, I., N.P. Gregersen, K. Hernetkoski, E. Keskinen, and A. Nyberg. (2003). Young novice drivers, driver education and training: Literature review. Linköping, Sweden: Swedish National Road and Transport Research Institute.

Fisher, D.L., N.E. Laurie, R. Glaser, K. Connerney, A. Pollatsek, S.A. Duffy, and J. Brock. (2002). Use of a fixed-base driving simulator to evaluate the effects of experience and PC-based risk awareness training on drivers' decisions. Hum Factors 44(2):287-302.

Forsyth, E.G. Maycock, and B. Sexton. (1995). Cohort study of learner and novice drivers: Part 3, Accidents, offences and driving experience in the first three years of driving. Crowthorne, Berkshire: Transport Research Laboratory.

Fuller, R., and D. Bonney. (2003). Driver education and training in post-primary schools in Ireland. In Behavioural research in road safety: Thirteenth seminar. Crowthorne, Berkshire: Transport Research Laboratory.

Fuller, R., and D. Bonney. (2004). Driver education and training in post-primary schools. Monograph. Dublin: National Council for Curriculum and Assessment.

Glad, A. (1988). Phase 2 Driver Education: Effect on the risk of accident. Oslo, Norway: Norwegian Center for Research, Department of Transportation.

Green, L.W., and M.W. Kreuter. (1999). Health promotion planning: An educational and environmental approach. 3 rd ed. Mountain View, CA: Mayfield Publishing (McGraw-Hill).

Gregersen, N.P. (1994). Systematic cooperation between driving schools and parents in driver education, An experiment. Accident Analysis and Prevention 26(4):453-461.

Howorth, N., N. Kowadlo, and C. Tingvall. (2000). Evaluation of pre-driver education program. Victoria, Australia: Monash University Accident Research Centre.

Highway Safety Center. (2002). Traffic Safety Education Life Long Learning Process. Indiana, PA: Indiana University of Pennsylvania.

Katila, A., E. Keskinen, S. Laapotti, and M. Hatakka. (1995). Changes in slippery road accidents as an effect of renewed driver training in Finland. Turku, Finland: University of Turku.

Keskinen, E., M. Hatakka, and A. Katila. (1998). The Finnish way of driver improvement: Second phase of training. Paper presented at International Driver Improvement Workshop, at Bergisch Gladbach, Germany: Bundesanstalt fur Strassenwesen.

Levy, D.T. (1990). Youth and traffic safety: The effects of driving age, experience, and education. Accident Analysis and Prevention 22(4):327-334.

Lonero, L., and K. Clinton. (1998). Changing road user behavior: What works and what doesn’t. PDE Publications. Available from www.drivers.com.

Lonero, L., K. Clinton, J. Brock, G. Wilde, I. Laurie, and D. Black. (1995). Novice driver education model curriculum outline. Washington, DC: AAA Foundation for Traffic Safety.

Lonero, L.P., K.M. Clinton, G.J.S. Wilde, K. Roach, A.J. McKnight, H. Maclean, S.J. Guastello, and R. Lamble. (1994). The roles of legislation, education, and reinforcement in changing road user behaviour. Toronto, Ontario: Ontario Ministry of Transportation.

Lonero, L.P., K.M. Clinton, B.N. Persaud, M.L. Chipman, and A.M. Smiley. (2005). A longitudinal analysis of Manitoba Public Insurance Driver Education Program. Winnipeg, Manitoba: Manitoba Public Insurance.

Lonero, L. & Clinton, K. (2006) Evaluation of Driver Education: Management Overview, Washington, D.C.: AAA Foundation for Traffic Safety.

Lonero, L., & Clinton, K. (2006) Situation Analysis of Driver Education in Canadian Provinces and Territories. Ottawa, Ontario: Natural Resources Canada.

Lonero, L., Clinton, K., Sleet, D. (2006). Behavior Change Interventions in Road Safety.(in) Geilen, A.C., Sleet, D.A., & DiClemente, R.J. Injury and Violence Prevention: Behavioral Science Theories, Methods, and Applications, San Francisco, California: Jossey-Bass.

Lund, A.K., A.F. Williams, and P. Zador. (1986). High school driver education: Further evaluation of the DeKalb County study. Accident Analysis and Prevention 18 (4):349-357.

Masten, S.V., and E. A. Chapman. (2003). The effectiveness of home-study driver education compared to classroom instruction: The impact on student knowledge, skills, and attitudes. Report to the Legislature of the State of California. Report no. CAL-DMV-RSS-03-203. Sacramento, CA: California Department of Motor Vehicles.

Masten, S.V., and E.A. Chapman. (2004). The effectiveness of home-study driver education compared to classroom instruction: The impact on student knowledge and attitudes. Traffic Injury Prevention 5(2):117-121.

Maycock, G. (1995). Accidents in the first three years of driving. TRL Annual Review: 71-77.

Maycock, G., and E. Forsyth. (1997). Cohort study of learner and novice drivers. Part 4: Novice driver accidents in relation to methods of learning to drive, performance in the driving test and self assessed driving ability and behaviour. Crowthorne, Berkshire: Transport Research Laboratory.

Mayhew, D.R., and H.M. Simpson. (1995). The role of driving experience: Implications for the training and licensing of new drivers. Toronto, ON: Insurance Bureau of Canada.

Mayhew, D.R., and H.M. Simpson. (1997). Effectiveness and role of driver education and training in a graduated licensing system. Ottawa, Ontario: Traffic Injury Research Foundation.

Mayhew, D.R., H.M. Simpson, and A. Robinson. (2002). The safety value of driver education and training. Injury Prevention 8 (Supplement II):3ii-8.

Mayhew, D.R., H.M. Simpson, and A. Pak. (2003). Changes in collision rates among novice drivers during the first months of driving. Accident Analysis and Prevention 35(5):683-691.

McKenna, C.K., B. Yost, R.F. Munzenrider, and M.L. Young. (2000). An evaluation of driver education in Pennsylvania. Harrisburg, PA: Pennsylvania Department of Transportation.

McKnight, A.J. (1984). Driver education — when? Journal of Traffic Safety Education (Jan):13-15.

McKnight, A.J., and A.S. McKnight. (2003). Young novice drivers: Careless or clueless? Accident Analysis and Prevention 35(6):921-925.

MacNeil, S. (in press). Driver education, training and licensing. In R.E Dewar and P. L. Olson. Human Factors in Traffic Safety. Second edition. Tucson, AZ: Lawyers and Judges Publishing Co., 2007.

Nichols, J. L. (2003). Driver education review. Paper presented at the NTSB Forum on Driver Education and Training, October 2003 . Washington, DC: National Transportation Safety Board.

National Institute of Mental Health. (2004). Teenage brain: A work in progress. NIMH, 05/13/04 2001 (accessed November 16, 2004). Available from http://www.nimh.nih.gov/publicat/teenbrain.cfm.

National Research Council, Institute of Medicine, and Transportation Research Board. (2007) Preventing Teen Motor Crashes: Contributions from the Behavioral and Social Sciences, Workshop Report. Program Committee for a Workshop on Contributions from the Behavioral and Social Sciences in Reducing and Preventing Teen Motor Crashes. Washington, DC: The Natinal Academies Press.

National Transportation Safety Board. (2005). National Transportation Safety Board Public Forum on Driver Education and Training, October 29-30, 2003. Report of Proceedings. Washington, DC: NTSB.

OECD. (1986). Effectiveness of road safety education programmes. Paris: Road Transport Research, Organisation for Economic Co-operation and Development.

OECD Scientific Expert Group. (1990). Behavioural adaptations to changes in the road transport system. Paris: Road Transport Research, Organisation for Economic Co-operation and Development.

Patton, M.Q. (1997). Utilization-focused evaluation: The new century text. 3rd ed. Thousand Oaks, CA: Sage Publications.

Plato, K.C., and A. Rasp, Jr. (1983). Parental views on issues related to traffic safety education and the licensing of teenage drivers. Olympia, Washington: Washington Office of the State Superintendent of Public Instruction.

Potvin, L., F. Champagne, and C. Laberge Nadeau. (1988). Mandatory driver training and road safety: The Quebec experience. American Journal of Public Health 78:1206-1209.

Preusser, D.F. (1996). Licensing practices and crash risk in the United States. In New to the road: Reducing risks for young motorists. Proceedings of the First Annual Symposium of the Youth Enhancement Service, June 8-11, 1995, edited by H. Simpson. Los Angeles, CA: Youth Enhancement Services, University of California.

Preusser, D.F., S.A. Ferguson, and A.F. Williams. (1997). Effect of teenage passengers on the fatal crash risk of teenage drivers. Arlington, VA: Insurance Institute for Highway Safety.

Quadrel, M.J., B. Fischhoff, and W. Davis. (1993). Adolescent (in)vulnerability. American Psychologist 48(2):102-116.

Ray, H.W., M. Sadof, J. Weaver, J.R. Brink, and J.R. Stock. (1980). Safe Performance Secondary School Driver Education Curriculum Demonstration Project. Washington, DC: U.S. Department of Transportation, National Highway Traffic Safety Administration.

Roberts, I., and I. Kwan. (2004). School based driver education for the prevention of traffic crashes. The Cochrane Library (1).

Robertson, L.S. (1980). Crash involvement of teenaged drivers when driver education is eliminated from high school. American Journal of Public Health 70 (6):599-603.

Robertson, L.S., and P.L. Zador. (1978). Driver education and fatal crash involvement of teenage drivers. American Journal of Public Health 68(10):959-965.

Road Safety Educators’ Association. (2005). Process for curriculum review. Ontario, Canada: DSAA.

Shope, J.T., and L.J. Molnar. (2003). Graduated driver licensing in the United States: Evaluation results from the early programs. Journal of Safety Research, 34(1S):63-69.

Siegrist, S. (2003). Driver training and licensing — The European perspective . Paper presented at the NTSB Forum on Driver Education and Training, October, 2003. Washington, DC: National Transportation Safety Board.

Simpson, H., ed. (1996). New to the road: Reducing the risks for young motorists. First Annual International Symposium of the Youth Enhancement Service, June 8-11, 1995: Youth Enhancement Service.

Sleet, D.A. & Lonero, L. (2002). Behavioral approaches to preventing motor vehicle crashes. In L. Breslow, J. Last, L. Green, & M. McGinnis (Eds.), Encyclopedia of Public Health. Mosby.

Smiley, A., L. Lonero, and M. Chipman. (2004). Final Report. A review of the effectiveness of driver training and licensing programs in reducing road crashes. Paris, France: MAIF Foundation.

Smith, D.L. (1983). The DeKalb Driver Education Project — The same mistakes: Improper criteria. Journal of Traffic Safety Education 30(January):14 & 26.

Smith, M.F., and J. Blatt. (1987). Summary of preliminary results: Follow-up evaluation. Safe Performance Curriculum Driver Education Project. Spokane, WA: American Driver & Traffic Safety Education Association.

Smith, M. F. Research agenda for an improved novice driver education program: A report to congress. U.S. Department of Transportation, National Highway Traffic Safety Administration, Washington, D.C. 1994.

Stock, J.R., J.K. Weaver, H.W. Ray, J.R. Brink, and M.G. Sadof. (1983). Evaluation of Safe Performance Secondary School Driver Education Curriculum Demonstration Project. Washington, DC: U.S. Department of Transportation, National Highway Traffic Safety Administration.

Strang, P.M., K.B. Deutsch, R.S. James, and S.M. Manders. (1982). A comparison of on-road and off-road driver training. Canberra City, Australia: Department of Transport and Construction Office of Road Safety.

Stufflebeam, D.L. (2001). Evaluation Models. New Directions for Evaluation (89):1-106.

Stufflebeam, D.L. (2003). The CIPP model for evaluation. Paper presented at Annual Conference of the Oregon Program Evaluators' Network, October 3, at Portland, Oregon.

Taylor-Powell, E., L. Jones, and E. Henert. (2002). Enhancing Program Performance with Logic Models. [Accessed November, 2004]. Available from the University of Wisconsin-Extension web site: http://www1.uwex.edu/ces/lmcourse/

The Health Communication Unit. (2006). Evaluating health promotion programs. Toronto, ON: Centre for Health Promotion, University of Toronto.

Trochim, W. (2001). The research methods knowledge base. Cincinnati, OH: Atomic Dog Publishing.

Vernick, J.S., G. Li, S. Ogaitis, E.J. MacKenzie, S.P. Baker, and A.C. Gielen. (1999). Effects of high school driver education on motor vehicle crashes, violations, and licensure. American Journal of Preventive Medicine 16(1S):40-46.

Weaver, J.K. (1987). Summary of preliminary results. Follow-up evaluation. Safe Performance Curriculum Driver Education Project. Washington, DC: U.S. Department of Transportation, National Highway Traffic Safety Administration.

Wiggins, S. (2005). Graduated Licensing Program interim evaluation report - Year 3. Vancouver, BC: Insurance Corporation of British Columbia.

Wilde, G.J.S. (1993). Improve risk perception and reduce risk acceptance: Two proposals for driver education. In Novice Driver Education Symposium, University of Alberta, Edmonton, Alberta, April 22-33. Edmonton, AB: Alberta Motor Association Driver Education.

Wilde, G.J.S. (1994). Risk homeostasis theory and its promise for improved safety. In Challenges to accident prevention: The issue of risk compensation behaviour, edited by R. M. Trimpop and G. J. S. Wilde. Groningen, The Netherlands: Styx Publications.

Williams, A.F. (2003). Teenage drivers: Patterns of risk. Journal of Safety Research, 34(1S), 5-15.

Woolley, J. (2000). In-car driver training at high schools: A literature review. Adelaide, Australia: Transport SA.

Wynne-Jones, J.D., and P.M. Hurst. (1984). The AA driver training evaluation. In Wellington, New Zealand: Ministry of Transport. Traffic Research Report No. 33. Wellington, New Zealand: Ministry of Transport.

Zhao, J., R.E. Mann, M. Chipman, E. Adlaf, G. Stoduto, and R.G. Smart. (2006). The impact of driver education on self-reported collisions among young drivers with a graduated license. Accident Analysis and Prevention 38:35-42.