Yesterday’s release of a report on the effects of legalization in Colorado inspired an entertaining game of dueling media headlines. The Guardiansaw a slight change in consumption levels: “Colorado marijuana law: uptick in adults lighting up, but not minors.” The Washington Times, meanwhile, read doom in the data: “Colorado hospitals and treatment centers say marijuana-related cases have spiked.”
What gives? Here’s the story behind the headlines.
In 2013, a few months after voters legalized marijuana, the Colorado Legislature ordered the state’s Department of Public Safety to study the impacts of legalization. Yesterday the department issued the first of what’s expected to be a series of reports issued every two years. Breaking news stories pulled the juiciest bits out of the 98-page study, but a deep dive into the report yields many more interesting, confusing, and frustrating conclusions.
Here’s the most important takeaway: In these early years of legalization, analysts are working with terrible data. That doesn’t mean the data yield negative conclusions. It means the numbers themselves — the inputs — are generally crap. It’s going to be years before Colorado, Washington, Oregon, and other states build up a record of consistent reporting and solid data. Until then, we’re left to read tea leaves.
The author of yesterday’s Colorado report, Jack Reed, is an analyst with the Department of Public Safety’s Office of Research and Statistics. He prefaced his work with a disclaimer so loud you can practically see him waving his hands.
“It is too early,” Reed wrote, to draw any conclusions about the effects of legalization on public health, public safety, or underage use. The data in his report, he wrote, are skewed by a self-reporting effect. “The decreasing social stigma regarding marijuana use could lead individuals to be more likely to report use on surveys, and to health workers in emergency departments and poison control centers,” he wrote, “making marijuana use appear to increase when perhaps it has not.”
Essentially, the Legislature demanded conclusive proof on subjects — like impaired driving, out-of-state diversion, and hospital visits — about which there are no reliable numbers.“It should be made clear,” wrote Reed, “that there are many areas of interest where the data to measure impacts do not currently exist.” Nonetheless, the report was demanded by law. So Reed did the best he could to cobble together bits and pieces into an overview.
Reed’s cautionary note didn’t stop editors from pulling out wobbly speculations and, by headlining them, transform them into repeatable fact. (Let me acknowledge that I’m one of those editors. At Leafly, our AP-reported take hewed more closely to the Guardian’s.)
Here’s a guide to which facts are wobblier than others.
Guardian: “Uptick in Adults But Not Minors”
It would be foolish to argue that the adult consumption rate hasn’t changed. With legalization and the opening of recreational stores in Colorado, Washington, and Oregon, a fair number of adults have sampled cannabis either for the first time or for the first time in a long time. But at least a portion of the higher use figure cited in the Colorado report — 12 percent of adults 26 and older reported currently using cannabis in 2014, compared to 5 percent in 2006 — can be attributed to the fact that much of the social stigma surrounding cannabis consumption dissipated between 2006 and 2014. People today are more willing to say they consumed.
Washington Times: “Marijuana Cases Have Spiked”
Similarly, the increase in emergency room visits, with cannabis exposure as a primary cause, could be apportioned between more honest self-reporting by patients and an increase in consumers ingesting far beyond their experience and tolerance. The numbers: 739 per 100,000 ER visits in the 2010-2013 period, 956 per 100,000 visits in the 2014-2015 period. Poison control center calls are up, too. There were 109 Colorado-based calls regarding cannabis in 2012, prior to legalization. In 2015 there were 227. Again: In 2006, if you called a poison control center with a case of overconsumption, you were implicating yourself in a crime. In 2015, you were asking for advice about a legal product. At the same time, the numbers regarding underage ingestion need to be faced straight up. In 2012 there were 16 calls regarding kids under 9 years old. In 2015 there were 49. That speaks to the growth of the edibles market and the need for better awareness and more appropriate handling.
Unremarked: Why Are Adults Still Being Arrested?
Perhaps the most puzzling numbers in the Colorado report are those that track cannabis-related arrests. In 2012, police arrested 12,894 people in Colorado on marijuana-related charges. Of those, 6,312 people were 21 or older. In 2014, police arrested 1,360 adults on marijuana-related charges. “There was a substantial decrease in adult arrests, down 76%,” the Department of Public Safety reports.
That represents progress. But why are police in Colorado still arresting 1,360 adults on cannabis charges? The answer: Public use.
“Discussions with law enforcement indicate that public consumption is a driving factor behind possession arrests,” Jack Reed wrote in an email to Leafly. Very interesting. In a state where adult possession arrests should have cratered to near zero, they remain in the thousands because there’s nowhere to legally partake outside a private residence.
More strange data: Of those arrested, only 757 were charged in court. Which means 603 adults were stopped, questioned, and arrested but ultimately not charged.
It’s a fair bet that too many of those adults were people of color. A recent report from Oregon illustrated a depressing fact: Legalization doesn’t wipe away the racial disparities of the prohibition era. The harm is reduced, of course, as fewer people of color are stopped and arrested for cannabis in legal states. But non-whites remain over-represented in those lower arrest numbers. In Colorado, white people saw a 51 percent drop in marijuana arrests between 2012 and 2014. Arrests dropped by 33 percent for Hispanics, and only 25 percent for African-Americans. Even after legalization, the cannabis arrest rate for African-Americans in Colorado remained almost three times that of white people.
Drugged Driving: Data Depends on "Trooper Perception"
The flimsiest data comes in the area of drugged driving. In Denver, the police noted little change. Marijuana accounted for 2.5 percent of all DUI arrests in 2014, and 2.9 percent in 2015. (Alcohol accounted for 95 percent of DUI arrests in both years.) State Patrol officers, on the other hand, noted an uptick in DUIs issued for marijuana-based impairment, from 6 percent of DUIs in 2014 to 8 percent in 2015. (Overall, DUIs dropped 18 percent in that same period.) The problem here is that the State Patrol data isn’t based on toxicology tests.
The data, as the report says in tiny print, “is based on trooper perception.”
The data taken from fatal accidents has other problems. On its surface, there’s an uptick. Of tested drivers involved in a fatal crash in 2013, 11.5 percent registered positive for THC. In 2014, that figure was 16 percent. But here’s the thing: The testing itself is haphazard. Eighty-one percent of deceased drivers are tested; only 14 percent of drivers who survived a crash (in which at least one person died) are tested. The administered test only indicates the presence of THC metabolites, which are detectible in the blood for days and sometimes weeks after ingestion. It’s like administering a test to determine if a driver has had a glass of wine sometime in the past two weeks. As the report’s author wrote: “The results reported here … are not statements about driver impairment.”
And yet they read as statements about driver impairment. And that’s how they enter the public conversation.
And that’s where reports like this really matter. At a cannabis conference a couple months ago, I listened to California Assemblymember Tom Lackey, a Republican who co-sponsored the state’s new medical marijuana regulatory act (AB 266), talk about his genuine compassion for medical marijuana patients and his concerns over adult-use legalization. As a former California Highway Patrol officer, Lackey said, he was watching the data on Colorado DUIs and fatalities carefully, and “the early reports are not good.”
He’s right in one sense. The early reports are not solid. They are unreliable and, as the author of this latest report put it, they are not “definitive statements of outcome.” But that’s how they’re going to be taken by leaders like Tom Lackey. Sketchy data begets false conclusions, which result in wrongheaded policies and laws. In cases like this, bad data may be worse than no data at all.
