Not a day goes by that I don’t search for information, and whether that information is a movie showtime or the mechanism by which a bacterial species is resistant to zinc toxicity, I need that information to be accurate. In the era of real fake-news and fake real-news, mockumentaries, and misinformation campaigns, the ability to find accurate and unbiased information is more important than ever.
Yet, assessing the validity of information and verifying sources is an under-appreciated and under-taught skill. There are some great resources available for determining the reliability (if the same results are achieved each time), and validity (is it a real effect), of a dataset, as well as of the authors. Even with fact-evaluation resources available through The National Center for Complementary and Integrated Health (NCCIS), The University of Maine, The Georgetown University Library, or Michigan State University, like any skill, finding information takes practice.
Where do I go for Science Information?
Thanks to the massive shift towards digital archiving and open-access online journals, nearly all of my information hunting is done online (and an excellent reason why Net Neutrality is vital to researchers). Most of the time, this information is in the form of scientific journal articles or books online, and finding this information can be accomplished by using regular search engines. In particular, Google has really pushed to improve its ability to index scientific publications (critical to Google Scholar and Paperpile).
However, it takes skill to compose your search request to find accurate results. I nearly always add “journal article” or “scientific study” to the end of my query because I need the original sources of information, not popular media reports on it. This cuts out A LOT of inaccuracy in search results. If I’m looking for more general information, I might add “review” to find scientific papers which broadly summarize the results of dozens to hundreds of smaller studies on a particular topic. If I have no idea where to begin and need basic information on what I’m trying to look for, I will try my luck with a general search online or even Wikipedia (scientists have made a concerted effort to improve many science-related entries). This can help me figure out the right terminology to phrase my question.
How do I know if it’s accurate?
One of the things I’m searching for when looking for accurate sources is peer-review. Typically, scientific manuscripts submitted to reputable journals are reviewed by 1 – 3 other authorities in that field, more if the paper goes through several journal submissions. The reviewers may know who the authors are, but the authors don’t know their reviewers until at least after publication, and sometimes never. This single-blind (or double-blind if the reviewers can’t see the authors’ names) process allows for manuscripts to be reviewed, edited, and challenged before they are published. Note that perspective or opinion pieces in journals are typically not peer-reviewed, as they don’t contain new data, just interpretation. The demand for rapid publishing rates and the rise of predatory journals has led some outlets to publish without peer-review, and I avoid those sources. The reason is that scientists might not see the flaws or errors in their own study, and having a third party question your results improves your ability to communicate those results accurately.
Another way to assess the validity of an article is the inclusion of correct control groups. The control group acts a baseline against which you can measure your treatment effects, those which go through the same experimental parameters except they don’t receive an active treatment. Instead, the group receives a placebo, because you want to make sure that the acts of experimentation and observation themselves do not lead to a reaction – The Placebo Effect. The Placebo Effect is a very real thing and can really throw off your results when working with humans.
Similarly, one study does not a scientific law make. Scientific results can be situational, or particular to the parameters in that study, and might not be generalizable (applicable to a broader audience or circumstances). It often takes dozens if not a hundred studies to get at the underlying mechanisms of an experimental effect, or to show that the effect is reliably recreated across experiments.
Data or it didn’t happen. I can’t stress this one enough. Making a claim, statement, or conclusion is hollow until you have supplied observations to prove it. This a really common problem in internet-based arguments, as people put forth references as fact when they are actually opinionated speeches or videos that don’t list their sources. These opinionated speeches have their place, I post a lot of them myself. They often say what I want to say in a much more eloquent manner. Unfortunately, they are not data and can’t prove your point.
The other reason you need data to match your statements is that in almost all scientific articles, the authors include speculation and theory of thought in the Discussion section. This is meant to provide context to the study, or ponder over the broader meaning, or identify things which need to be verified in future studies. But often these statements are repeated in other articles as if they were facts which were evaluated in the first article, and the ideas get perpetuated as proven facts instead of as theories to be tested. This often happens when the Discussion section of an article is hidden behind a pay wall and you end up taking that second paper’s word for it about what happened in the first paper. It’s only when the claim is traced all the way back to the original article that you find that someone mistook thought supposition for data exposition.
The “Echo Chamber Effect” is also prominent when it comes to translating scientific articles into news publications, a great example of which is discussed by 538. Researchers mapped the genome of about 30 transgender individuals – about half and half of male to female and female to male, to get an idea of whether gender identity could be described with a nuanced genetic fingerprint rather than a binary category. This is an extremely small sample group, and the paper was more about testing the idea and suggesting some genes which would be used for the fingerprint. In the mix-up, comments about the research were attributed to a journalist at 538 – comments that the journalist had not made, and this error was perpetuated when further news organizations used other news publications as the source instead of conducting their own interview or referencing the publication. In addition, the findings and impact of the study were wrongly reported – it was stated that 7 genes had been identified by researchers as your gender fingerprint, which is a gross exaggeration of what the original research article was really about. When possible, try to trace information back to its origin, and get comments straight from the source.
How do I know if it’s unbiased?
This can be tricky, as there are a number of ways someone can have a conflict of interest. One giveaway is tone, as scientific texts are supposed to remain neutral. You can also check the author affiliations (who they are and what institution they are at), the conflict of interest section, and the disclosure of funding source or acknowledgements sections, all of which are common inclusions on scientific papers. “Following the money” is a particularly good way of determining if there is biased involved, depending on the reputation of the publisher.
When in doubt, try asking a librarian
There are a lot of resources online and in-person to help you find accurate information, and public libraries and databases are free to use!
