Spot the Difference (in Differences)

The pro-21 crowd insists that the 21 drinking age saves lives.  In fact, they even claim to know the number of lives it saves each year:  about 900 fewer traffic deaths per year.  However, that figure is based on rather questionable assumptions.  And in this post, like in previous ones, we seek to demolish these faulty conclusions.

TRAFFIC DEATHS

Take a look at the graph below.  We have accessed the CDC's WONDER database, a publicly available health statistics database which has mortality data going back to 1979.  The graph contains all deaths among 15-19 year olds and 20-24 year olds that were labled as "motor vehicle traffic" fatalities.  All deaths are per 100,000 people.

National

As avid readers of this blog can recall, the drinking age was 18 or 19 in most states in 1979, and was 21 in all states by 1988.  Thus, in the aggregate, it seems like there was some lifesaving effect.  But as Mark Twain observed, "there are lies, damned lies, and then there are statistics".  One thing you can probably see right off the bat is that the decline in deaths had begun in 1979-1980, well before the 1984 federal law that coerced states to raise the drinking age to 21.  However, several states also raised their drinking ages voluntarily before that:  usually to 19, but a few raised it to 20 or 21 by 1983.  So we need to dis-aggregate the data to see whether or not there was any lifesaving effect.

We have separated the states into "control" and "treatment" groups to do a difference-in-differences analysis on the above data.  There were twelve states that did not change their drinking ages at all since the 1930s and 1940s (Arkansas, California, Indiana, Kentucky, Missouri, Nevada, New Mexico, North Dakota, Oregon, Pennsylvania, Utah, and Washington), remaining 21 throughout, and these form our control group.  (We omitted Utah since that is a special case due to their high Mormon population and archaic alcohol laws in general).  For the treatment group, we included every state that had a drinking age of either 18 or 19 for at least a portion of the time period 1979-1998.  All other states (Delaware, Maine, Michigan) were excluded entirely since they had a drinking age of 20 or higher since January 1, 1979.

Control Group

Treatment Group

In addition, we then modified the treatment group to exclude all states that had a drinking age higher than 18 as of January 1, 1979.  All states that are included had a drinking age of 18 at some time.

Treatment-18 Group

Funny how similar all three graphs look, despite the differences in the drinking ages.  The difference-in-differences analysis between the control and treatment-18 groups is shown below:

Control, 15-19:  -46.5%

Treatment, 15-19:  -39.2%

DD:  +7.3%

Control, 20-24:  -49.5%

Treatment, 20-24:  -40.1%

DD:  +9.4%

Wow, that sure seems like a perverse effect of the 21 drinking age!  The treatment group saw less progress overall than the control group, and they started out lower as well.  This was true even more so for the 20-24 age group.  Of course, an even greater lack of progress is evident in the 25-34 age group:

Control, 25-34: -49.5%

Treatment, 25-24: -34.2%

DD:  +15.3%

What would our difference-in-differences analysis look like if we used the ratios of 15-19 and 20-24 year old fatalities to those in the 25-34 age group, as opposed to absolute fatality rates?  The following calculations show what would happen to our results:

Control, 15-19 ratio: +5.8%

Treatment, 15-19 ratio: -8.0%

DD:  -13.8%

Control, 20-24 ratio:  0%

Treatment, 20-24 ratio:  -9.2%

DD:  -9.2%

 
So, does this mean there is a net lifesaving effect after all?  Hardly!  There could be any number of reasons why the rates of progress differed between 15-24 year olds and 25-34 year olds.  Take another look at the graphs above.  We see that most of the fatality decline has occurred by 1992, and we also know that there was no change in the drinking age from 1990 onwards (the first full year that no grandfathered 18-20 year olds could drink legally).  Thus, we now restrict our ratio DD analysis to 1979-1992:
 
Control, 15-19 ratio:  -6.5%
Treatment, 15-19 ratio:  -3.7%
DD:  +2.8%
 
Control, 20-24 ratio:  -0.1%
Treatment, 20-24 ratio:  -11.3%
DD:  -11.2%

Thus, it does not appear that the 21 drinking age had a real lifesaving effect.  The results of the 1979-1992 ratio DD analysis are totally in the wrong direction:  20-24 year olds should be only minimally affected by an increase in the drinking age to 21, yet they appeared to "benefit" (likely by chance) when we control for the relative lack of progress among 25-34 year olds in the treatment group, while the 15-19 age group saw a relative increase in deaths.  Omitting Louisiana (a state who had a massive loophole in the 21 law well into the 1990s) and states that had a drinking age for only a portion of 1979 did not substantially alter the results.  Unfortunately, we could not separate the data into 18-20 and 21-24 age groups, nor could we distinguish between alcohol-related and non-alcohol-related deaths.  However, the 15-19 year old group includes those most affected by a drinking age of 18 or 19 as well as the so-called "spillover" group of 15-17 year olds, and only one-fifth of the 20-24 year old age group would be directly affected by raising the drinking age to 21.  Also, not all "alcohol-related" crashes are necessarily caused by alcohol, and differences in reporting and testing can bias the results in either direction.  Thus, our general conclusions remain valid.

New York

Here we see New York, a state whose drinking age was 18 since 1934, raised to 19 in December 1982 voluntarily and then to 21 in December 1985 under federal duress.  In NYC, where nearly half of the state's population lives, the drinking age remained unenforced for the most part until Rudy Giuliani became mayor in 1994.  One can see that the decline in traffic fatalities began since 1980, years before the drinking age was raised, and seemed to decline at a slower rate after the drinking age was raised to 21.  The decline then resumed a few years later, but stalled again since 1994 when NYC began enforcing the 21 drinking age more strictly.  One can also clearly see this when you look specifically at fatalities in teen drunk-driving crashes since 1982.  Thus it is far from obvious that the 21 drinking age had any benefits at all as far as reducing traffic fataltites.

Now, see if you can guess what year the following, unnamed state raised its drinking age.  Go on, guess.

The answer is that they didn't!  The state shown, California, has had a drinking age of 21 since 1933, yet its graph still seems to resemble the national graph as well as New York's.  And if you look closely at the graphs, you'll see that New York was actually making more progress than California until 1985, after which the reverse was true.  Gee, what could have happened?

"ALCOHOL-RELATED" TRAFFIC DEATHS

While the CDC WONDER database does not separate out alcohol-related ones, NHTSA's FARS database does.  While not all this data is publicly available, some of it is.  Taken from a NHTSA paper, we look at the state-level experience of changes in alcohol-related fatalities involving a 16-20 year old driver from 1982-1998.  Since there are no reliable data before 1982, we define our control group as all those who had a drinking age of 21 for several years before 1982 (including Michigan and Utah this time) and our treatment group as those whose drinking ages were 18 for at least a portion of 1982.

Control, median:  -64.3%
Treatment, median:  -68.6%
DD:  -4.3%

At first glance. it looks like there might indeed be a lifesaving effect, even if it is a small one.  But what if we omit those states that had a drinking age higher than 18 for a substantial portion (i.e. more than one month) of 1982? 

Control, median:  -64.3%
Treatment, median:  -64.1%
DD:  +0.2%

Thus, we see the effect was spurious.  This further contradicts the claim that the 21 drinking age saved lives.

NON-TRAFFIC DEATHS

But what about non-traffic fatalities, such as alcohol poisoning, falls, violence, suicide, and alcohol-related diseases?  We also calculated those as well:
 
Alcohol-related deaths* (1979-1996)
 
Control, 15-19:  -82% (unreliable)
Treatment, 15-19: -60%  (unreliable)
DD:  +22%
 
Control, 20-24:  -39%
Treatment, 20-24:  -62%
DD:  -23%
 
Control, 25-34:  -33%
Treatment, 25-34:  -47%
DD:  -14%
 
*Includes all deaths labeled with "alcohol" in the cause of death.  Since the number of deaths is small, any year with fewer than 20 deaths is labled as "unreliable".
 
Suicide (all kinds, 1979-1998)
 
Control, 15-19:  -12%
Treatment, 15-19:  +10%
DD:  +22%
 
Control, 20-24:  -21%
Treatment, 20-24:  -22%
DD:  -1%
 
Control, 25-34:  -27%
Treatment, 25-34:  -20%
DD:  +7%
 
(Perhaps Alex Koronay-Palicz was right about the loss of social cohesion leading to more suicides as a result of the 21 drinking age.)

Homicide (all kinds, 1979-1998)*
 
Control, 15-19:  +17.9%
Treatment, 15-19:  +3%  (+15.2% excluding NY)
DD:  -14.9% (-2.7% excluding NY)
 
Control, 20-24:  -4.8%
Treatment, 20-24:  -12.3% (-7.8% excluding NY)
DD:  -7.5% (-3.0% excluding NY)
 
Control, 25-34: -40.1%
Treatment, 25-35: -41.0%  (-38.4% excluding NY)
DD:  -0.9% (+1.6 excluding NY)

*Interpret with caution; California (a control state) saw declines of -3.5% and -19.3% for 15-19 and 20-24 year olds, respectively.
 
Unintentional Injury (all kinds, including traffic, 1979-1998)
 
Control, 15-19:  -48.8%
Treatment, 15-19:  -41.3%
DD:  +7.5%
 
Control, 20-24:  -47.8%
Treatment, 20-24:  -41.9%
DD:  +6.9%
 
Control, 25-34:  -40.0%
Treatment, 25-34:  -33.0%
DD:  +7.0%
 
All Causes of Death (1979-1998):

Control, 15-19:  -33.7%
Treatment, 15-19:  -28.4%
DD:  +5.3%

Control, 20-24:  -31.8%
Treatment, 20-24:  -29.2%
DD:  +2.6%

Control, 25-34:  -25.5%
Treatment, 25-34:  -20.3%
DD:  +5.2%

Thus, it appears to be a wash overall at best, or even perverse at worst.  In nearly all cases, 15-19 year olds see less progress in the treatment states relative to the control states and/or less progress relative to 25-34 year olds.  If that's success, we'd hate to see what failure looks like.