Alan Schwarz (July 28) – As increasing numbers of parents keep their children from playing tackle football for safety reasons, the National Football League and other groups have sought to reassure them that the game is becoming less dangerous.
No initiative has received more backing and attention than Heads Up Football, a series of in-person and online courses for coaches to learn better safety procedures and proper tackling drills. The NFL funds and heavily promotes the program. The league and USA Football, youth football’s governing body, which oversees the program, have sold Heads Up Football to thousands of leagues and parents as having been proved effective — telling them that an independent study showed the program reducing injuries by 76 per cent and concussions by about 30 per cent.
That study, published in July 2015, showed no such thing, a review by The New York Times has found. The research and interviews with people involved with it indicate, rather, that Heads Up Football showed no demonstrable effect on concussions during the study, and significantly less effect on injuries overall, than USA Football and the league have claimed in settings ranging from online materials to congressional testimony.
“USA Football erred in not conducting a more thorough review with Datalys to ensure that our data was up to date,” Scott Hallenbeck, executive director of USA Football, said in an email to The Times. “We regret that error.” He added that the material would be removed from the organization’s print and online materials, and that “our partners and constituents” would be notified of the errors.
Brian McCarthy, an NFL spokesman, said that the league would also include updated information from now on.
Both USA Football and the league said that the questionable data and conclusions were actually preliminary results provided by Datalys five months before the study was published. The lead researchers for Datalys, Thomas Dompier and Zachary Kerr, confirmed in interviews that, despite knowing that the final paper contradicted their preliminary claims, they did not inform USA Football of this until last month, one day after speaking with The Times.
Dompier, the president of Datalys, said in an interview: “We’re the ones that put out the numbers. We’re the ones that kind of blew it.”
In an email, Kerr said that the company had released the early data because, “The results were so compelling, we felt morally obligated to make the youth football community aware of the results.”
The NFL and its players’ union formed USA Football in 2002 to oversee the sport and help it grow among children ages 6 to 14. But participation has dropped precipitously in recent years, from 3 million in 2010 to about 2.2 million last fall — a decline generally attributed to concerns about injuries, particularly to the brain.
In 2013, in consultation with the NFL, USA Football started Heads Up Football, whose primary goals were to improve safety and reassure parents. The program requires one “player safety coach” per team to attend a clinic that focuses on concussion recognition and response, blocking and tackling techniques, proper hydration and other safety topics. A team’s other coaches must take online courses in those subjects as well.
In March 2014, the NFL gave USA Football $45 million, in large part to get more youth leagues to adopt the program.
While USA Football is said to operate independently from the NFL, the league is its primary source of operating funds, and some researchers consider the two almost indistinguishable.
“In my mind, USA Football and the NFL are one,” said Dawn Comstock, a professor of epidemiology and the primary researcher into high school sports injuries at the Colorado School of Public Health. “If I’m talking with one about something involving youth football safety, my perception is I’m talking to both.”
Comstock said that in July 2014, Jeff Miller, the NFL’s senior vice president for health and safety policy, and David Krichavsky, then its director of player health and safety, asked her to propose some studies that would, she said, “highlight the potential positive aspects” of youth football’s safety initiatives and provide “a potential positive take-home message for parents.” Comstock said that she had provided some ideas but that the league did not pursue them.
McCarthy, of the NFL, said in an email Monday, “Our only interest is in research that will help us determine the efficacy of these and other programs and how we can make the game safer.”
In 2014, USA Football asked Datalys, an Indianapolis-based firm that handles all of the NCAA’s injury research, to monitor injury rates during that fall season among six youth leagues that used Heads Up Football and four leagues that did not, covering more than 2,000 players.
In February 2015, Datalys gave USA Football the results: Leagues that used Heads Up Football had 76 per cent fewer injuries, 34 per cent fewer concussions in games and 29 per cent fewer concussions in practices.
In USA Football’s blog post announcing that the safety program “reduces injuries,” Dompier said: “This is compelling data. I am actually surprised by the strength of the association but completely confident in our findings.”
These figures were prominently reported in the media and on websites of youth leagues as a means to show parents that Heads Up Football was scientifically sound. NFL promotional materials have called the program “The New Standard in Football”; a page in its 2015 Information Guide is headlined, “Study Finds USA Football Program Advances Player Safety.”
But last summer, when The Orthopaedic Journal of Sports Medicine published Datalys’ formal paper on the study, the paper did not include the same injury and concussion figures. Its data actually told a far different story about Heads Up Football’s effectiveness.
Rather than looking at Heads Up Football leagues in one category, the paper instead split them into two groups: those that did or did not also belong to Pop Warner Football, a division of youth leagues that has added its own rules to mitigate injuries. Pop Warner leagues have disallowed certain head-on blocking and tackling drills and drastically reduced full-contact practice time, measures that were not a part of USA Football’s program.
As it turned out, only leagues that adhered to Pop Warner’s rules saw a meaningful drop in concussions. Leagues that used Heads Up Football alone actually saw slightly higher concussion rates, although that uptick was not statistically significant. The previously reported drops were clearly driven by a league’s affiliation with Pop Warner, not Heads Up Football.
Similarly, Heads Up Football leagues saw no change in injuries sustained during games unless they also used Pop Warner’s practice restrictions. The drop in practice injuries among Heads Up Football-only leagues was 63 per cent, but combined with in-game injuries, the total reduction became about 45 per cent — far less than the 76 per cent presented by USA Football and the NFL for the past year and a half.
The authors did not address how the paper’s data contradicted their preliminary conclusions from five months before. Regarding the fact that Datalys did not inform USA Football or the NFL of the discrepancies, Kerr said in an email: “Datalys stands by our decision to release preliminary data in our Feb 2015 release because if we prevented even one youth football player from suffering an injury (sprain, fracture, strain, severe contusion, or concussion), then the release was a success.”
There are other instances when Datalys has presented data to the public that differed from its scientific papers.
A “Youth Football Fact Sheet” for the public currently on the Datalys website lists the most common injuries sustained by youngsters, as determined by a separate study it conducted for USA Football three years ago. But it has significant differences from the list in a paper the company published last year, also in The Orthopaedic Journal of Sports Medicine. For example, the paper’s listing for “Nervous System (stinger),” which comprised 4.2 per cent of injuries, does not appear on the fact sheet; that slot is filled instead by “Wind Knocked Out” (4.1 per cent), a category that does not appear in the paper.
Dompier said in an email that the category for stinger — where a blow to the spine causes extreme pain and numbness through the arms — was renamed Wind Knocked Out because both are neurological injuries, and the latter would be more recognized by parents.
A spokesman for USA Football, Steve Alic, said that research conducted outside Datalys has shown the effectiveness of Heads Up Football. He cited the Fairfax County public school system in Virginia, which has seen a 24 per cent decrease in total injuries and a 43 per cent drop in concussions since adopting the program in 2013.
Bill Curran, the county’s director of student activities and athletics, confirmed those numbers and praised Heads Up Football’s safety initiatives for contributing to them. He added that Fairfax County went well beyond the Heads Up program, though, in ways that included drastically reducing full-contact practice time during the season to 90 minutes per week, whereas before, he said, “we probably had some teams doing 90 minutes in a single practice.”
“I give them a huge amount of credit,” Curran said of USA Football’s efforts. “But it takes a hell of a lot more than going to their website and taking the online courses and getting accreditation.”
As the 2016 season approaches, the faulty pronouncements about the research continue to be cited by youth programs and football officials as evidence that Heads Up Football makes football safer, especially regarding concussions. During a high school sports conference in Alabama last week, a coach presented a glowing slideshow about the program to fellow coaches and athletic directors, unaware that many of the numbers and statements were not supported by the data.
In May, coaches from Columbia High School in Maplewood, New Jersey, invited some eighth-graders interested in playing football to a meeting in the cafeteria.
“They basically said they teach Heads Up Football, which reduced head injuries and concussions,” said Jacob Kasdan, one of the students who attended the meeting. “I think they’re struggling to find enough players.”
Jacob went home and asked if he could play this fall. His father declined to sign the forms.
Source: New York Times