Motor Cycle Safety Foundation Is Not So Safe.
The following article being re edited at this time dates back to motorcycleclub.org 1998. revised with all permissions.

                                           MSF IS NOT SO S
The Motorcycle Safety Foundation (MSF) provides information on rider training, licensing, and government relations.

     We all remember Pythagoras as the guy who invented the theorem. But Old Py didn't see himself that way. As far as he was concerned, he was the visionary founder of a new religion, the main tenets of which were metempsychosis, abstention from beans, and the noevl idea that the universe talks to us through the language of numbers. Now, the business about beans got nowhere, and the notion of reborn souls is nowadays an odd quirk from the land of telephone tech support; but the idea that truth is numerical has sure gained wide credence over the years. Belief, logic, even common sense, all fall on their face when confronted by numbers these days. Numerical science is to all intents and purposes the reigning modern religion, just as Pythagoras envisioned. This credence has led to universal practical application of numbers to everything from bullets to cannonballs. It is generally accepted today, by thinking people everywhere, that you cannot bullshit the numbers. Denial does not alter the outcome. No amount of belief or reasoning alters the outcome. What we measure is what it is.

I begin with these preliminaries because what I am about to tell you will immediately make you expostulate your beliefs, logic, and common sense, in angry denial, denial in the face of numbers. Here it is: Motorcycle safety courses do no good at all. If anything, they increase your danger. How do I know? That's what the numbers tell us.

Go ahead. Expostulate. I know. Flabbergasting.

Okay. Now listen to this: Some months back, trailering an R1100R down to the buyer in Charlottesville VA via route 29, I passed the Insurance Institute of Highway Safety (IIHS). I had recently read an IIHS article about ABS brake effectiveness in the BMWRA magazine which happened to mention in passing the ineffectiveness of rider training courses. That got me curious. What do they do there? Why would they mention that? I hit their web site when I got back. I found some stuff. I was astonished. So I followed links and googled bibliographies from the UK to California to you name it. I read studies commissioned by DMVs from a dozen states and several different countries. They all agree. All of them. That's the funny thing. By the numbers, motorcycle rider education courses don't do a damn thing good for you.

Now, granted, there are problems with the studies. Many are based on pitifully small samples. Many suffer from methodologies so poor that they make you want to smack someone. Many are interpreted with strange illogic. All seem to be written in tortured opaque bureaucratese. As examples, let me summarize three such studies. The first is flat poorly done; the results of the second are tortured to derive a desired conclusion, but the third is lucid, simple, straightforward in method, and also based on a generous sample:

1) Here is a dumb methodology which actually recurs in many studies. The research team call several hundred people. Those who answer the phone, they ask whether they will answer a survey. Those who say yes to that, they ask whether they ride a motorcycle. Those who say yes to that, they ask whether they have taken a motorcycle safety course. Finally, have they had any accidents. Then they compare those accident stats against the general rider population. That's it. Now, you would think that riders who voluntarily sign up for non-required rider courses would of course be more concerned about safety than the general population and would therefore ride more carefully. Not so; same stats. Those who say they took classes report as many accidents as the generality. But these studies are open to dispute because the methodology is so poor: Who answered versus who wasn't home? Who was willing to be surveyed versus who wasn't? Who answered truthfully? What qualifies in a dirt bike rider's mind as an accident? How do you telephone survey any riders who had an accident and died from it?

2) Here's a dumb conclusion: California commissioned a study which compared accident statistics after passage of their new rider education requirements with stats before rider education became a requirement. This study wasn't written by a scientist or a statistician, by the way; nor even by a safety maven; instead it was contracted to a writer of homosexual mystery novels (God bless Kali). The result was simply that stats for new riders did not differ from stats for riders licensed before the new requirements. However, the mystery author, not satisfied with plain results, proceeded to deduce that since mortality for riders as a whole, both with and without education, declined during the period of the study, therefore the program must therefore be a success. The period actually coincided with a nation wide decline in mortality. And of course new riders were only a small portion of all riders as a whole. The numbers themselves only said it was a wash. The conclusion tortured from a study commissioned by mandatory education naturally supported more mandatory education.

3) Finally, here is a smart study: New York randomly assigned 26,000 new motorcycle license applicants to one of four groups. One group just took the little DMV quiz and got their motorcycle endorsement. The second group was required to take the more extensive Motorcycle Operator Safety Training (MOST) written test developed by the NHTSA. The third was required to take three hours of hands on basic rider training plus the MOST. The fourth group got the full boat, including the extensive MOST test plus 20 hours of MSF training, including 8 hours classroom and 12 hours in the saddle. Two years later, all these licenses were checked against traffic accident reports. The second group did no better than the first. The third and fourth groups did a little worse than the first and second. What does this tell us? If it were bullets or cannonballs, because numbers are the language of the universe any dispassionate observer would naturally conclude from these numbers that classroom doesn't help and in fact hands on rider courses actually hurt. Yes, they hurt, as in increasing your chances of getting in a wreck. By the numbers. Nor is this the only study which finds more accidents, not fewer, among trained riders.

Remember: These studies were generally commissioned by bureaucracies which are heavily invested in justifying safety courses. There are no opponents to the idea of safety courses among researchers. There are just the facts. The researchers don't twist facts to make it look like they should put themselves out of business; on the contrary, in typical government fashion, these invested bureaucracies generally concluded: "We evaluated our safety courses. They don't work. Therefore, we need more safety courses." But the states which already require more such courses get equally futile outcomes. Results have been repeated in one state after another, one country after another. The better the method, the wider the sample, the worse the results.

I don't expect you to buy it on my word, because it just sounds so wrong. But consider: For ages it seemed to everyone that the heavier an object is the faster it must fall. A heavy cannonball would of course fall faster than a little bullet. That's just common sense. You can feel it. Hold one in each hand: the cannonball is heavier; it must fall faster. Then one day Galileo dropped a bullet and a cannonball simultaneously off the tower of Pisa and his spotter on the ground confirmed that, in fact, they both hit the ground at the same time. Who'd have thunk it? Not me. The different rates of fall idea made natural sense to me up until the day in school they told me about Messer G and his two Bs up on the tower of P. That was a How Bout Dat! moment.

So read about it yourself and think it over and see if you have a similar how bout dat moment. Here's how: There's a suitably scholarly and lengthy study of driver education studies in general entitled "Effectiveness and role of driver education and training in a graduated licensing system" by Mayhew and Simpson, published at www.drivers.com/article/305. I have excerpted the whole section about motorcycle education and posted it on the web for you to read, MORE LATER at UNDER CONSTRUCTION BEING REVISED AT http://www.godsservants.com/msf was www.motorcycleclub.org/safety/mayhew_simpson.htm. read the article below here (SAFETY).

This excerpt goes into brief detail about each of many different studies conducted in different states and different countries. Read it. At the end of Mayhew and Simpson, there's a heck of a bibliography. Google it. This is not a study itself; it is the summary of many such studies. The farther you follow its sources, the more persuasive are the results.

Why is it this way? I don't know. Life's a funny business., full of dangling whys that you just get used to For example, why is it I can go up and down, forward and back, right and left, but in the fourth dimension, time, I can only go one way? Why? Beats the crap outta me; ask Steven Hawkings, maybe he has a clue. That's just the way it is. And this is the way motorcycle training is. The fact that the language of the universe tells us that MSF training works backwards, if at all, is no more bizarre than time, seahorses, baseball salaries, bullets versus cannonballs, or any other dangling mysteries of life.

I feel absolutely confident many of you will by now have concluded: "I don't care. It doesn't apply to me. I know I got good things out of that course. I will continue to advocate them to others. It makes sense to me." I feel equally confident that innumerable soldiers over the years have entered the battlefield confident that they would be the one who did not stop a bullet, statistics did not apply to them, they would have a good war and return a hero, yet wound up mouldering cannon fodder at the exact same numerical rate as anyone else. It is folly to assume that numbers which apply across the board do not apply to you. Projectiles will fall the same rate in your back yard as at Pisa. An open mind accepts proof. A closed mind, when confronted by facts, shoots the messenger and changes the terminology.

Now back to your regular programming. Next BMW Owners News you read, authors will extol motorcycle rider courses. Next internet forum you visit, members will extol motorcycle rider courses. Next meeting of your riders group you attend, your friends will extol motorcycle rider courses. Next novice who tells you he would like to ride, you will advise him first take a motorcycle rider course. This is the first and last time you will be warned that they won't actually do any good. In numerical fact, they are more apt to do harm. And that is why I bring this to your attention.

Ride safe through this curious world. Written and revised 1998 2013 by

Robert Davis and Brett Paulson


(SAFETY)
This is the entire motorcycle training section, excerpted from "Effectiveness and role of driver education and training in a graduated licensing system" by Mayhew and Simpson published at http://www.drivers.com/article/305/.

2.3 THE EFFECTIVENESS OF MOTORCYCLE RIDER EDUCATION AND TRAINING PROGRAMS
As in the area of formal driver instruction, considerable importance has been placed on education and training of novice motorcycle riders as an effective means of addressing their crash risk. In North America and elsewhere, specialized formal courses have been developed and delivered by instructors/teachers to provide the novice rider with the information and skills deemed necessary to operate a motorcycle and ride safely in traffic.

Formal motorcycle rider education/training has been viewed as critical to the novice's mastery of the demanding skills necessary to operate and control a motorcycle. Many believe that the unique handling characteristics of the motorcycle and the rider's vulnerability to perceptual, aerodynamic and roadway disturbances require the acquisition of a high level of skill -most effectively obtained in a formal training situation. Such initiatives are founded on the fundamental and compelling assumption that students who are exposed to the education/training will be at lower risk of traffic mishap than those who are not.

Similar to early studies on the effectiveness of driver education, almost all of the early evaluations of the effectiveness of motorcycle education courses produced encouraging results-they found that formally trained riders had a lower risk of collision than persons not so trained. However, virtually all of these investigations suffered from some methodological flaw (e.g., no control or comparison group of untrained riders whatsoever, small sample size, no random assignment of subjects to trained and untrained groups, no control for exposure), which made the reliability of the positive findings questionable (Collins 1979; Satten 1980).

By contrast, the better designed studies generally produced disappointing results-they found that formally trained riders were not at lower risk of a collision than riders who did not receive the instruction. In addition, several evaluations actually found that formally trained riders had higher crash rates (per miles ridden) than those who were informally trained.

This section examines findings from the more rigorously designed studies. Similar to the previous section, the discussion is organized by jurisdiction. This approach is necessary given that the evaluations have examined courses that differ in many respects-i.e., in length of instruction, time spent on-bike versus in-class instruction, content, and quality of instruction. Accordingly, findings in one jurisdiction do not necessarily apply to courses offered elsewhere. Nevertheless, the overall direction of the results or the "convergence of evidence" has relevance to the current investigation.

As will become evident from examining this section, developments in motorcycle rider education/training largely parallel those in formal driver instruction. Surprisingly few evaluations have been undertaken to determine the effectiveness of motorcycle rider education/training despite the importance ascribed to these programs in most jurisdictions. As suggested above, the few evaluations that have been undertaken have generally produced discouraging findings and led many to question the value of formal motorcycle rider education and training as an effective loss reduction measure. In spite of these findings, formal training has continued to be popular.
2.3.1 Experience in the United States.
Several evaluations have been conducted of the Motorcycle Safety Foundation's Motorcycle Rider Course (MRC). This course consists of 20 hours of instruction-8 of which are spent in the classroom and 12 devoted to practical riding skills.

An evaluation of the MRC in South Dakota, using self-reported information on formally trained and non -trained riders, did not find that those who took the course had a lower crash rate (per miles ridden) (Osga 1980). And Mortimer (1984) compared the self-reported collision experience of 213 MRC graduates with that of 303 riders who had not taken the MRC course in Illinois. He found that "when controlling for age and years licensed, those who took the course did not have a lower accident rate (per million miles ridden) than the control group."

More recently, using a much larger sample, Mortimer (1988) replicated the evaluation of the MRC in Illinois. In this study, he compared 914 MRC graduates with 500 riders who had not taken the MRC on a wide range of safety measures. He reports that "compared to the control group, those who had taken the course did not have a lower violation rate, a lower accident rate, a lower total cost of damage to accident involved motorcycles, a significantly lower mean cost of injury treatment per accident, or a lower total cost of injury treatment."

On the positive side, although there was no difference between the groups in the total cost of damage to collision-involved motorcycles, the average cost per collision was somewhat lower among trained riders and the trained riders made more use of protective clothing (this did not, however, produce significant differences between the groups in the mean cost of injury treatment per collision or the total cost of injury treatment).

One of the most ambitious and sophisticated evaluations of rider education and training was initiated in New York State in the early 1980s. In an effort to overcome the methodological difficulties inherent in most previous studies, the National Highway Traffic Safety Administration (NHTSA) sponsored a large-scale project to evaluate the effectiveness of motorcycle rider education, in combination with improved tests of motorcycle knowledge and skills. The study used an experimental design in which nearly 26,000 new motorcycle license applicants were randomly assigned, over a three-year period (1981-1983), to one of four groups who were to take either (1) the existing New York State motorcycle license test consisting of a brief knowledge test and an in-traffic skill test; (2) the Motorcycle Operator Skill Test (MOST 11) developed by NHTSA; (3) three hours of training plus the MOST 11; or (4) 20 hours of training (MSF's Motorcycle Rider Course that involves 8 hours of classroom instruction and 12 hours of off-street on-cycle training) in addition to the MOST II test.

The motorcycle safety community in the United States viewed the project with considerable interest and optimism since it would "yield valuable data on the effectiveness of different kinds of novice training courses" (Hartman 1982). The study was not expected to produce results that confirmed the negative findings of previous, and what many perceived as less rigorous, research. However, the study found no statistically significant differences in crash rates as a function of the different training and licensing systems. Of particular interest, riders exposed to the 3-hour or 20-hour Motorcycle Rider Course had crash rates that did not differ from the rates of those who had not received formal training.

The study has, however, been criticized on operational and administrative grounds that may have seriously biased the findings. For example, (1989) indicated that "the applicant flow was considerably less than had been expected and administrative problems led to long delays between the dates of application, training and license testing." However, these problems would only be important to the validity of the results if they differentially affected the subjects in the various groups. It is not known to what extent this may have been the case. McPherson also noted that "the applicants utilized in this study were found not to be novices, having an average three years riding experience." This criticism is perhaps more substantive since it could be argued that the training or testing programs cannot be expected to improve the performance of, or differentiate among, more experienced riders. Accordingly, the study may not have been an adequate or appropriate test of either MOST II or the MSF Motorcycle Rider Course.

Similar results were reported by McKnight (1987) from an evaluation of the Pennsylvania Motorcycle safety program. That study examined the collision experience of over 3,000 pairs of trained and untrained motorcycle license applicants, who were matched on the basis of age, sex, and prior driving records. He found no significant differences between the mileage adjusted collision rates of trained and untrained riders.

More recently, Billheimer (1996) published the results of an evaluation of the California Motorcyclist Safety Program (CMSP). Initially implemented in July 1987, the CMSP training program became mandatory in January 1988 for all riders under 18 seeking a California motorcycle license. The mandatory age was subsequently raised in January 1991 to include all riders under 21 years of age. In 1994, a further amendment to the legislation permitted students who successfully completed the course to forego the riding skills test required for licensing by the Department of Motor Vehicles.

The evaluation included several quasi-experimental designs: (1) a pre-post design compared motorcycle collisions before implementation of the CMSP and the introduction of mandatory training to collision experience after its implementation; (2) a pre-post design, with a "control", compared trends in motorcycle collisions in California to overall trends in the United States and (3) a matched group design compared the collision involvement of samples of trained and untrained riders from the Los Angeles area over three time periods following training-six months, one year and two years.

Pre-post comparisons of collision trends revealed that the total number of motorcycle collisions and fatal motorcycle collisions in California had decreased since the implementation of CMSP, especially among younger riders, the primary target for the program. For example, crashes among riders under 25 years of age dropped by 84 percent between 1987 (when the program was introduced) and 1995, compared to only a 50 percent decline among riders over the age of 25 during this same period.

Moreover, riders under 18 years of age experienced an 88 percent drop in crashes from 1988 (when training became mandatory for them) to 1995, compared to a 61 percent decline among riders over the age of 18. Similarly, those age 18 to 20 experienced a 78 percent drop in crashes from 1991 (when training became mandatory for them) to 1995, compared to a drop of only 45 percent among riders age 21 and over.

Although these findings suggest a positive effect of the CMSP, many factors other than training could account for the greater reductions in the number of collisions among young riders-e.g., changing demographics, fewer young licensed riders, relatively less riding by young people. Foremost among these would be changes in the relative numbers of young and older licensed riders. And indeed, the number of licensed riders under the age of 25 declined 68 percent between 1987 and 1995 even though the total number of licensed riders was 5.6 percent higher than in the pre-CMSP period. As a consequence, the crash rate (number of collisions per 1,000 licensed riders) for those under 25 declined only 50 percent after the implementation of the CMSP, compared to a drop of 70 percent for older riders (the crash rate for those under 25 declined from 146 to 72; the crash rate for older riders declined from 39 to 12). Thus, the reduction in collisions among riders under 25 the primary target for the CMSP-was actually less than the reduction among all riders.

Billheimer observes, however, that the number of licensed riders is not an accurate measure of the number of active riders. This is so because older riders renew their license without riding and several years pass before the records of inactive riders are purged from DMV files. This means that the estimate of the number of older licensed riders is spuriously high, producing an artificially low crash rate for that group. Importantly, however, an artificially low crash rate for older riders should apply throughout the study period-i.e., there is no reason to believe that difficulties in accurately estimating the number of older licensed riders became worse in the period after the implementation of the CMSP. Accordingly, the above finding that the decline in collision rate was more pronounced among older riders still holds.

Given his concerns with the licensing data, Billheimer used the number of registered motorcycles, which represents a more accurate estimate of the number of active riders than the number of licensed riders, as the denominator of exposure to calculate fatality rates in the periods before and after the 1987 implementation of the CMSP. This analysis showed that the number of fatalities per registered motorcycle dropped 59 percent from 1987 to 1995-1.2 fatalities per thousand registrations in 1986, to 0.50 fatalities per thousand registrations in 1995. Again however, the extent to which the CMSP contributed to this decline is unknown because the reductions could be attributed to factors other than mandatory training-e.g., an overall decline in riding, the introduction of a mandatory helmet law in 1991. Of special note, CMSP trainees accounted for only 18 percent of active riders by 1995-a total of 95,175 trainees and 525,000 registered motorcycles. This means that the vast majority of active riders had not taken training during the period in which the declines in collisions and fatalities occurred. Unfortunately, the author was unable to disaggregate the motorcycle registration data for riders of different age to determine whether declines in fatality rates were more pronounced among young riders, the primary target for the CMSP, than among older riders.

The author also compared trends in motorcycle collisions in California to those in the rest of the United States. These analyses showed that the number and rate of overall motorcycle collisions and fatal motorcycle collisions have also been dropping throughout the United States but that the declines have been greater in California. For example, the author reports that "on the average, motorcycle fatalities per thousand registrations in California have dropped by 33.1 percent since the formation of the CMSP in 1987. Over the same period, the level in the remainder of the United States only dropped 17.3 percent." (p. 39).

Although these comparative analyses as well as the trend analyses described earlier are suggestive, they do not provide convincing evidence that CMSP contributed to the declines. As Billheimer observes:

While the overall decline in accident trends since the start of the CMSP is gratifying, several factors in addition to the CMSP are likely to have contributed to these trends. These factors include a drop in motorcycle riding, the introduction of a mandatory helmet law in January 1991, and the aging population. (p. 4-1)

Given that factors other than the CMSP are uncontrolled in the analyses, the conclusion that the CMSP did contribute to these crash trends is not supported by the analyses. In fact, the finding that crashes per licensed driver decreased more among older than younger riders-the primary target of the program -could be interpreted as demonstrating that the CMSP actually eroded the decline among young riders. That is, if these riders had not taken training, their crash rates would have dropped even further, to a level similar to that of older riders. Of course, this interpretation is also highly speculative and unsubstantiated. Accordingly, the most that can be concluded is that California has experienced a dramatic decline in motorcycle crashes that has outstripped declines in the rest of the United States and that the CMSP may have contributed to this decline.

Billheimer also used a more rigorous evaluation design to examine the impact of the CMSP. He compared the collision involvement of matched pairs of trained and untrained riders in the Los Angeles/Orange County area over periods of six months, one year and two years. The sample of untrained riders was obtained by means of interviews each year from 1989 to 1994 at places frequented by motorcyclists (dealerships, accessory shops, schools, malls etc.) and at high schools and colleges. This sample was obtained at the same time riders were being trained to minimize potential sources of bias arising from different time frames.

The trained and untrained riders were matched in terms of gender, age, years ridden ("within at least six months under two years"), miles ridden last year (defined only as "within 1,000 miles under 5,000 miles" by the author, p. 4-6) and commute history ("is the motorcycle used to commute?"). Over the five-year study period, 1,158 untrained riders were successfully matched with riders who had completed the basic training program for novice motorcyclists, which lasts a minimum of 16 hours. The matching occurred from a pool of 14,832 trained riders and 16,600 untrained riders who had been interviewed.

In the analyses, Billheimer split the matched pairs of basic course graduates and untrained riders into two groups based on their pre-training (or interview) riding experience: 1) riders with 500 miles or less of prior driving experience and 2) those with more than 500 miles of prior riding experience. This yielded a 2x2 design with training (trained/untrained) and prior riding experience (500 miles or less vs. more than 500 miles) as the variables. These groups were compared in terms of crash rates during four time periods-one year prior to training (or interviewing), six months after training, one year after, and two years after. Based on this analysis, the author concluded that:

... it appears that training has an impact on riders, particularly inexperienced riders, for at least six months following training. Accident rates for untrained riders with little or no prior experience were more than double the rates of trained riders during the first six months after training. Beyond this time, riding experience levels the playing field and accident rates of the trained and untrained groups become indistinguishable. (p. 5-8, 5-9).

Thus, the only statistically significant (i.e., p<.10) difference in crash rates between trained and untrained novice riders occurred among those with little or no prior riding experience in the six month period following training (or interviewing)-the six month crash rate for untrained riders was 0.85 collisions per 100,000 miles, compared to only 0.39 collisions per 100,000 miles for trained riders (statistically significant difference at the 0.10 level). In absolute terms, in the six months following training, 615 trained riders had ridden 1,288,000 miles and had five collisions; by comparison, over the six-month period, the 615 matched untrained riders had ridden 820,000 miles and had seven collisions.

These results suggest a short-term positive effect of training but several methodological weaknesses in the research design cast doubt on their reliability. One issue concerns the extent to which the matching of trained and untrained riders successfully controlled for potentially confounding variables. As noted earlier, Billheimer did match trained and untrained riders on key variables such as gender, age, and riding experience to ensure group comparability and to minimize biases associated with self-selection-i.e., riders who take training may be more safety conscious than those who do not and therefore might have lower crash rates than untrained riders even without the course. However, the self-selection bias has not been completely eliminated because some in the study took basic training voluntarily-e.g., this would apply to 18, 19 and 20-year-olds prior to the mandatory training requirement in 1991, as well as those older than 20 for the duration of the project.

Moreover, Billheimer notes that because of the mandatory training requirement he experienced difficulty identifying untrained riders under the age of 18 for matching with trained riders. To overcome this problem, he included in the untrained group riders who ignored the mandatory training and licensing requirement and rode untrained and, therefore, unlicensed. Thus, some of the matched untrained riders differed from the trained riders on a critical factor -i.e., willingness to obey the training and licensing requirements. This difference is critical because unlicensed riders are overrepresented in fatal crashes. In fact, in another section of the report, Billheimer shows that unlicensed riders comprise about 26 percent of the riding population but account for 65 percent of the fatalities in California. Thus, the untrained group may include riders who have a higher likelihood of collision involvement.

Another design concern arises from the choice of dependent measures -collisions per mile ridden. Billheimer selected this to control for exposure, a control that was necessary because riding increased dramatically among those who took the course. Although an increase in riding was also characteristic of untrained riders in the period after their interview, it was much less pronounced. According to Billheimer, "there is no way of knowing whether confidence instilled by training leads to added riding, or whether the decision to ride (or ride more often) leads riders to take a training course" (p. 4-13). Importantly, if this latter possibility applies and the decision to ride leads to riders taking the course, trained riders once again differ from untrained riders on an important factor-i.e., intentions regarding riding.

A somewhat puzzling finding from the matched group analysis is that the trained novice riders had a higher collision rate than the group of untrained novice riders in the year prior to training-7.1 crashes per 100,000 miles compared to 2.2 crashes per 100,000 miles. Although this difference is not statistically significant, it is substantial and suggests that the trained and untrained groups may have differed in terms of their crash involvement at the outset. The fact that the trained group went from a high crash rate to a lower crash rate in the six months following training (interviewing) could be explained in part by regression to the mean.

Finally, Billheimer suggests that beyond the six month period from training (or interviewing), "riding experience levels the playing field and accident rates of the trained and untrained groups become indistinguishable." Although this may be the case, this same interpretation can be used to explain why trained riders have a collision rate that is lower than the untrained group in the six months following training. Indeed, riding experience increased by 2,200 percent among trained riders (from 56,000 miles in the year before training to 1,288,000 miles in the six months after training), compared to an increase of only 259 percent among untrained novice riders (from 228,000 miles in the year before the interview to 820,000 miles in the six months following the interview). Thus, it could be argued equally that the lower crash rate for trained riders was attributable to greater experience and not the training per se.

Given such methodological concerns, it must be concluded that the study provides little in the way of conclusive evidence that the CMSP contributed to the overall crash reductions witnessed in California from 1987 to 1995, or that it reduced the crash involvement of novice riders during the six months following training.

In summary, several studies have failed to provide definitive conclusions about the effectiveness of rider education and training in reducing crashes.
2.3.2 Experience in Canada.
Jonah et al. (1982) conducted an evaluation of the Ontario Safety League Motorcycle Training Program (MTP). It consists of a 20-hour course over two consecutive weekends (four days). About 4 hours of the course is allocated to theory and 16 hours to riding, 6 of which is on-street riding. A group of 811 graduates of the MTP and a group of 1,080 informally trained motorcyclists were surveyed to obtain information about riding experiences (including distance traveled), collisions and violations. Only 62 percent of the MTP graduates could be successfully contacted and 61 percent of these completed the telephone interview. The authors contacted 75 percent of the riders who were informally trained; 5 percent of them had actually received some form of other training and 14 percent were ineligible because they were not licensed or were not active riders.

A simple comparison of the number of collisions showed that MTP graduates were less likely than informally trained riders to be involved in collisions. However, when the analyses controlled for differences between the groups in exposure (miles reported ridden) and rider characteristics, such as age and sex, the differences disappeared-the crash picture was the same for those who were formally trained as it was for those who were not.

More recently, Rothe and Cooper (1987), using both telephone interviews and driver records of those interviewed, gathered information on three samples of motorcyclists: a group of formally trained motorcyclists (those who completed the British Columbia Safety Council's Motorcycle Training Program) who had also passed the Motor Vehicle Department's licensing exam; a group of formally trained motorcyclists (also took the B.C. MTP) who initially failed the licensing exam; and a group of informally trained motorcyclists. The results showed that if significant characteristics of the rider were controlled for in the analysis, motorcycle training was not significantly associated with fewer motorcycle crashes. Rothe and Cooper also repeated the analysis separately for 16-19 year olds and for those aged 16-24 to determine whether motorcycle training was more effective for very young novice riders. They indicate that "in neither case was motorcycle training identified as a potentially significant variable relating to motorcycle accidents" (p. 148). In addition, an examination of the frequency of collisions in the first year or first few years of riding was not significantly related to motorcycle training. Thus according to the authors, "this study could not find any evidence to suggest that motorcycle training (as represented by the single course in question) might have had a salutary safety effect as defined by police-reported accidents even during the initial stages of riding experience" (p. 149).

In contrast to these findings, McDavid et al. (1989) reported that the British Columbia Safety Council's 37-hour motorcycle safety program was associated with fewer motorcycle crashes as well as fewer collisions of all kinds. In this study, the authors compared the crash records of two matched groups of formally trained and informally trained motorcyclists. The matching procedure was adopted to overcome methodological weaknesses of previous evaluation studies related to the lack of similarity between persons who seek motorcycle training and those who do not (self-selection). Basically, the groups were matched in terms of sex (all males), age, location of licensing, the month in 1979 in which they had obtained their motorcycle license, possession of a motor vehicle license prior to acquiring their motorcycle license and previous driving record (crash and moving violation records). This matching procedure resulted in a comparison of 139 trained and 139 untrained riders.

Based on their analyses, McDavid et al. (1989) conclude:

... the findings of this study indicate that trained riders are involved in fewer accidents over time and tend to be involved in less severe accidents. The putative effects of training are not large, but they are persistent, and they can be stated with more confidence than findings from previous research (p. 71).

No doubt the confidence attached to their findings is in part due to the rigorous methods they used for controlling potentially confounding variables. However, the one variable that was not controlled was the amount of exposure. It is possible that formally trained riders had fewer collisions than informally trained riders because they traveled less often and less far on motorcycles. Of course, it could then be argued that formal training does have a positive effect-it results in people riding less often, perhaps even encouraging them not to ride during inclement weather or other potentially hazardous conditions. This is an interesting hypothesis that might be worthy of testing. Formal training may sensitize novice riders to the complexity of the task and to their increased vulnerability under certain conditions, such as low visibility.

Unfortunately, the study by McDavid et al. provides no data that bear on this hypothesis. All that can be concluded from their study is that the group of formally trained riders did have fewer overall collisions than those who were not formally trained, but the importance of this finding is diminished in the absence of exposure data. Moreover, if the analysis is restricted to only motorcycle crashes, this difference is not statistically significant.

An evaluation of rider training has also been conducted in Quebec, the only Canadian province that requires training as a prerequisite for obtaining a motorcycle license. Simard (1987) evaluated the effect of a 1985 law that required all prospective motorcycle riders to take a training course, even if they had previously taken a mandatory automobile driver training course. The crash records of two groups of first year novice riders were compared. One group had taken only the driver education course in 1983 or 1984; the other group had taken both the automobile driver course and the motorcycle rider training courses in 1985. An examination of the total number of motorcyclists injured in collisions revealed that the group subject to the new law (both courses) had considerably fewer collisions in their first year riding (125) than did the group that took only automobile driver training (243).

However, the greater licensing demands associated with the new law also resulted in a pronounced decrease in the number of motorcyclists who took the rider course and became licensed in 1985. This decrease could account for the lower number of collisions (i.e., fewer new riders to have collisions in 1985 compared to 1984). This possibility is supported by the finding that standardized comparisons using crash rates (numbers of collisions per motorcyclist) were actually higher for those who took a rider training course in addition to the driver education course (rate of 85 per 1,000 motorcyclist) than for those who took only the automobile driver training course (rate of 73).

Since these comparisons involved collisions that occurred in different years, factors unrelated to the introduction of the new law in 1985 could have influenced crash frequencies as well as the number of new motorcyclists in 1985. To control for this possibility, Simard also examined the pre- and post-law crash record of two other comparison groups shown below:

1984


1985

"CONTROL 1" COMPARISON

Licensed to ride a motorcycle at the end of 1984. License obtained either in 1983 or 1984. (took the motorcycle course only)


Licensed to ride a motorcycle at the end of 1985. License obtained either in 1984 or 1985. (took the motorcycle course only)

"CONTROL 11" COMPARISON

First licensed to drive an automobile in 1983. Also licensed to ride a motorcycle in 1983. (took driver education course for automobile only)


First licensed to drive an automobile in 1984. Also licensed to ride a motorcycle in 1984. (took driver education course for automobile only)

The first comparison "Control 1"-involved persons who had taken only the motorcycle course either in the pre- or post-law period. Since these motorcyclists had not previously taken automobile driver education, the legal change that occurred in March 1985 did not apply to them. Accordingly, any changes in collision frequencies or in the number of motorcyclists taking the motorcycle rider course from 1984 to 1985 would be accounted for by other factors. The number of these motorcyclists involved in collisions was very similar in 1984 (147) and 1985 (148), and their total numbers declined only slightly in this period (1,486 in 1984 to 1,430 in 1985). Thus the crash rate for persons who were trained and licensed to ride only a motorcycle (unaffected by the new law) increased slightly from 1984 (99) to 1985 (103).

This pattern was considerably different from the dramatic increase in the crash rate experienced by riders most affected by the law, as discussed previously. In contrast the second comparison ("Control 11"), which focused on motorcyclists licensed to drive both types of vehicles but who had taken only a driver education course for an automobile prior to the introduction of the new law (and were presumably unaffected by it), revealed a pattern similar to that of the initial (test) comparison. Fewer motorcyclists were involved in collisions in 1985 than in 1984 (118 compared to 133) but there was an even fewer number of motorcyclists in this group after the law (2,879 in 1985 compared to 3,776 in 1984). This yielded a much higher injury crash rate after the law (rate of 41 in 1985 compared to 35 in 1984).

The author speculates that the dramatic drop in the number of riders in this group may have resulted from the introduction of the new law. Although the law did not come into effect until March 1985, it was actually approved by the National Assembly in June 1984 and publicized extensively throughout that year. Accordingly, some of those licensed to drive an automobile in 1984 who had already taken driver education may have been dissuaded from becoming licensed to ride a motorcycle because they thought the law was already in effect and would require rider training as well.

In summary the Simard study suggests that a law requiring new riders to take a motorcycle training course, even if they had previously taken automobile driver training, is associated with an increase in the crash rate. This occurs even though collision frequencies have dropped because the number of riders has declined more precipitously. Simard (1987) speculates that "a direct effect of the mandatory rider course is to keep off the road those riders who would have, otherwise, shown a lower accident rate ... these riders do not ride very often in the first place. Thus, the low riding frequency rather than the rider's abilities on the road could explain the low accident rate." Perhaps the inconvenience of taking an additional training course dissuaded those who would have ridden infrequently from becoming motorcyclists. If the individuals who took both courses rode frequently, greater exposure to risk explains their higher crash rate
2.3.3 Experience in Europe.
In the United Kingdom, Raymond and Tatum (1977) compared the self-reported number of collisions of a sample of motorcyclists (258 riders) who had taken the Royal Automobile Club's motorcycle training scheme with a sample who had not (1, 104). The authors provided no details on the structure and content of the course itself but they found that trained riders had a significantly higher crash rate (per mile ridden) than did untrained riders. According to Plowden and Hillman (1984), the Raymond and Tatum study also found that training appeared to effect a positive change in performance in a few areas and that 17-year-old trainees may have benefited. The authors caution that "it is not possible to infer from this research that attendance at a training scheme may result in inferior riding ability since the predicted performance of the individual rider cannot be determined" (Raymond and Tatum 1977; Plowden and Hillman 1984).

More recently in December 1990, compulsory basic training was introduced as part of an effort to improve the road safety of motorcyclists. The safety impact of this change is currently being evaluated by the Transport Research Laboratory but the results are not yet available.
2.3.4 Summary.
A review of the evaluation research conducted in three countries provides no compelling evidence that rider training is associated with reductions in collisions. These findings are consistent with much of the evidence on the effectiveness of formal driver instruction.