I Saw It on the Internet, part two

This is part two of a series on posts on fact checking science related articles.

Read part one of this series. -- Part 3 - Part 4

As you sift through your daily flood of information, how do you know which to fact check? Ideally, you would fact check them all, but few of us have enough time to do that. Luckily, there are some quick, simple clues for spotting the biggest potential misinformation.

  1. Be skeptical of adjectives. When you read an “amazing”, “shocking”, and “astounding” article, your first thought should be, “Is this factual?” Dramatic headlines can be an indication of either exaggeration or lack of facts.
  2. Does the article claim that scientists have proven something? If so, raise a big, warning flag. Scientific studies do not prove things. They can produce evidence that supports or does not support an idea, but they never provide proof. In science, there is always the possibility for new studies to uncover new evidence.
  3. Are the claims in the article generic? Watch out for claims that all food additives are harmful, that all genetically modified organisms are bad, etc. Actual scientific claims are very specific, focusing on a single species, a single gene, a single disease, etc. Broad, generic claims are a warning flag that the information may not be based on sound science.
  4. Is the source of the article biased? Did you find that article on the dangers of food additives on a site devoted to promoting the dangers of food additives? Was the article about the Deadly Hobo Spider produced by a pest control company? If your source already has a strong opinion on a topic, the information could be biased. That does not mean the information is wrong, but it does mean you have to be especially careful about fact checking the information.
  5. Is this a subject that you feel strongly about? One of the hardest parts of fact checking is overcoming confirmation bias. Confirmation bias is the tendency to accept information that agrees with our beliefs. If we are reading something we strongly agree with, we are much less likely to spot errors or question the validity of information.

    An easy example of confirmation bias can be found in politics. Start paying close attention to your reactions as you read political news, and be brutally honest with yourself. Are you as critical of negative articles about the opposing party as you are of negative articles about your favored party? Do you fact check both equally?

    If you are reading an article that you strongly agree with, that is the time to be especially skeptical. It is hard to do, but very important if you really want to find the facts.

  6. Is the claim based on multiple studies? This is a big problem with many news stories. A single study may show that eating grapes while watching television could make you more likely to catch the flu. In science, a study has to be replicated over and over, each time with the same results. If the results are not the same, then there may be problems with either the methods or the conclusions.

    This is why we see so many contradictory news stories about coffee causing cancer, coffee curing cancer, coffee helping your heart, coffee hurting your heart, etc. A single study is just a single study. It may get the authors excited, but the rest of the scientific community immediately starts looking for other studies that show the same results.

    So why does the media report on single studies with such big headlines? Because saying, “Scientists show that coffee cures cancer!” is much more likely to attract viewers and readers than saying, “There is a possibility that coffee might have a possible role in reducing some form of cancer, but we won’t know until after a few more years of testing.”

Be especially careful when single studies are used as evidence by special interest groups. If there are forty studies that found that coffee does not increase your risk of cancer, and one study that shows that it does increase the risk, would that be valid proof that coffee is bad? No, of course not. Still, it is not unusual to find this kind of single study used as proof. Look for multiple studies with the same results, and compare that with the studies that found other results.

  7. Is the claim supported by the majority of the scientific community? Just because one scientist makes a claim does not make it right. It does not make it wrong either. There is a long history of revolutionary ideas in science. Revolutionary ideas that have strong evidence are accepted. Those that lack evidence are not.
  8. Does the article suggest a scientific conspiracy is hiding information? Again, this is a red flag indicating the need for extra fact checking. There is no secret society of evil scientists that controls the scientific community. Scientists love to spot flaws in the work of other scientists, and do not hesitate to point them out. This is not because scientists are mean. It is part of the process of science, trying to be sure that information is accurate.

    Any idea that is supported by the majority of the scientific community has to be backed up with some pretty strong evidence. Likewise, any idea rejected by the scientific community has to either be lacking evidence, or else there is strong evidence against it. When a revolutionary idea comes along, it requires careful, unbiased, scientific studies that can be replicated by other scientists to produce the same results.

None of these warning signs automatically mean that the information is wrong, but they are good indications that it is time for some serious fact checking before you accept the information, and especially before you repost it or pass it along.

Read part one of this series. -- Part 3 - Part 4