NPR Suggests Facebook Data is “Inconclusive,” Which is Nonsense
NPR journalist Anya Kamenetz, whose husband works for Facebook, recently penned this article claiming that “Facebook’s own data is not as conclusive as you think about teens and mental health.”
Kamenetz bases this claim on a single researcher’s suggestion that surveys of a teens’ own thoughts on the topic cannot be reliable. How does the researcher, Candice Odgers, know this?
Why, based on her own study of course. And there’s absolutely nothing wrong with that. But I would assume a researcher in this field would accurately represent their own study. And would be able to pull in the data from other studies (that they didn’t co-author) in order to be more objective about the field in general. But this is what the researcher told Kamenetz:
“If you ask teens if they are addicted/harmed by social media or their phones, the vast majority say yes,” she tells NPR. “But if you actually do the research and connect their use to objective measures … there is very little to no connection.”
But in the study itself, the researchers concluded,
“Having a social media account was associated with higher psychological distress, and more conduct problems, even after controlling for potential confounding factors” (George et al., 2020).
Clearly it appears that the researcher is cherry-picking what results to share with the reporter. Or, alternatively, the researcher shared their full conclusions with the NPR reporter, but the reporter chose only to report on a part of the results.
Facebook: Sloppy Internal Research?
Kamenetz, the NPR journalist, goes on to make another claim in the article. That Facebook’s internal research is sloppy and therefore shouldn’t really be given all that much weight.
Kamenetz appears to not realize that Facebook research is a decades-old effort that consists of a team of highly-trained PhD social scientists and statisticians well-versed in good research practices and methodology. Either that or she glosses over this fact:
The Facebook research was not peer-reviewed or designed to be nationally representative, and some of the statistics that have received the most attention were based on very small numbers.
She goes on to demonstrate this by showing that only 30% of Instagram teen girls felt worse about their bodies if they already acknowledged having body image issues. Given from other social science research we know that most teenage girls have some body image issues, I’m not sure this is as helpful as the journalist would like us to believe.
And of course internal Facebook research doesn’t go through peer-review. That would be silly. But that doesn’t mean the research is automatically of lower quality than peer-reviewed research.
For instance, the NPR journalist Kamenetz claims that while the Facebook research isn’t “nationally representative,” neither is the scientific study she cites. While it indeed did have 2,100 subjects (compared to the 2,500 or “thousands” of teens the Instagram study surveyed), they were all students in a single state’s school system — North Carolina. North Carolina may be a lot of things, but to suggest a single state’s population is representative of the nation is being disingenuous.
Furthermore, Kamenetz doesn’t note anywhere when the scientific study was completed — back in 2015. That’s six years ago. In Internet time, that may as well have been last century, as teen trends and usage changes considerably between 5 or 6 year periods.
Contradicting Standards & Conflicts of Interest
Later in NPR’s article, Kamenetz highlights another researcher’s study, one conducted by a for-profit private institute, Hopelab. But this one, a “research report” focused on COVID-19 coping (which strangely is not mentioned by the reporter), wasn’t peer-reviewed either. Sometimes non-peer reviewed research is bad and of poor quality (Facebook), and other times, when its findings jibe with the point a reporter needs to drive home, it’s apparently okay (Hopelab).
At the bottom of the article, after you’re well done reading it, a little “editor’s note” appears, mentioning a huge conflict of interest. The NPR journalist who wrote the piece is married to a Facebook employee. You’d think something that is so clearly problematic might be mentioned sooner in the body of the article itself, or at the top.
Because just like most people wouldn’t trust a peer-reviewed study partially sponsored by a pharmaceutical company, I suspect many would also not particularly trust an article defending part of a family’s income stream.
There is a lot of good, peer-reviewed research demonstrating the negative impact of Facebook and Instagram on teens’ health. In the next article, I’ll review some of it.
References
George et al. (2020). Young Adolescents’ Digital Technology Use, Perceived Impairments, and Well-Being in a Representative Sample. The Journal of Pediatrics.
