MANAGEMENT UPDATE.

A WARNING ABOUT TAINTED SKEWED OR FLAWED DATA

One of the podcasts we listen to regularly is the “Academic Minute”, which is supported by the American Association of Colleges and Universities. It features insightful – and helpfully brief – commentary by professors who can bring their research and interests to a wider audience.

A December 4 discussion (which ran just two minutes and thirty seconds if you choose to listen) focused on data problems in public health as seen by Joseph Cimpian, professor of economics and education policy at New York University. He delved into a powerful example of how questionable data could thoroughly muck up some potentially important findings from the Centers for Disease Control and Prevention about issues pertaining to LGBQ youth.

But some of us still like to read, so here’s a lightly edited version of Cimpian’s comments:

“Data shape policy, perceptions, and resource allocation. But when data are tainted, skewed, or false, problems arise.

“We found this issue in the Centers for Disease Control and Prevention’s Youth Risk Behavior Survey. At first glance, the CDC’s findings suggested alarming disparities between male teens who identified as lesbian, gay, bisexual, or questioning – or LGBQ – and their heterosexual peers. LGBQ males reported higher rates of steroid and intravenous drug use, weapon possession, and getting into fights. But here’s the twist: after accounting for potentially invalid data, the disparities vanished, revealing greater similarity between the groups.”

Cimpian goes on to explain that using machine learning “we found discrepancies between LGBQ and heterosexual respondents in areas that should not differ, like height and carrot consumption. Some gave exaggerated, implausible responses, including unrealistic heights, excessive carrot consumption, and likely false claims of being gay. They also reported high drug use and risky behaviors. Our method also accounts for careless or random responses, reducing the influence of suspicious data.

“These methods can be applied not only to surveys of youth, but to other areas where doubts about accuracy have emerged, including studies on public health behaviors during the COVID-19 pandemic, studies of adopted youth, and surveys about sexual orientation among adults.

“Our understanding of the world is only as good as the data that informs it. We must ensure that the data we collect and interpret are as accurate as possible.”