The Freelance Journalism Alliance supports quality science communication.
Good science journalism starts with a solid foundation in both journalism and the philosophy of science. While many authors have shown us that it’s possible to become a very popular science writer, without such a foundation, this practice has led us down a very unhealthy path of poorly communicating research findings to the general population. FJA suggests a strong understanding of quality of evidence, proper word choice, and use of citation.
Quality of Scientific Evidence
A good place to start working on our science communication is to have a clear understanding of quality of scientific justification. Not all scientific evidence is of the same quality.
Expert knowledge is the weakest form of scientific justification. While appeal to authority is valid, it is only useful if there is no reasonable counter argument presented. Unfortunately that’s rarely the case.
Case reports and case series are a little better than expert knowledge. They are descriptions of single cases, or a group of cases, which indicate the same phenomenon. So for instance, if a patient has a fever, and they take aspirin and the fever seems to go away, a detailed recording of the symptoms and recovery would be a case report. If a doctor had a number of similar experiences written up, that’s a case series.
Animal studies probably come next on this list. The issue with animal studies is that they’re imperfect models. While animal studies are generally considered far more ethical, the biology of mice, rats, and other animals differ enough from human biology that findings in animal subjects might simply not apply to humans.
Cross sectional studies are useful for gathering information from the population and can help identify correlation or lack of correlation, but they don’t do well in justifying causative relationships.
Case-control studies are up next. These studies do a little better at checking correlation and causation, by identifying individuals with and without the condition of interest, and seek to see if there was an exposure to the potential cause. These studies are retrospective. The advantage is that they’re cheap to produce, but their results aren’t as robust.
Cohort studies, on the other hand, are prospective. They start by identifying individuals with exposure to the potential cause, and those without, and track them over time. Because we’re tracking exposure and outcome, over time, rather than investigating the situation after the fact, there’s less risk of the results being due to random chance.
Randomized controlled trials are generally considered the gold standard of clinical trials and medical research. They’re the best kind of primary studies that we can use, because they limit confounding factors to the best of our ability. Unfortunately they’re often expensive, and can be quite unethical. For instance, we cannot ethically withhold a recognized treatment from a group of people, and see what happens.
Systematic reviews and meta-analyses are often placed at the pinnacle of research, and in may ways they are. But systematic review really should be placed in an entirely separate category of research. While all of the other studies mentioned above are studies of the phenomenon and theory being tested, systematic reviews and meta-analyses are studies of studies.
There are so many other ways in which we can categorize studies, and one way is to ignore a traditional hierarchy and just look at which kinds of studies are better than others.
- Systematic reviews are more robust than individual studies.
- Human studies more robust than animal studies.
- Experimental studies are more robust than observational studies.
- Blinded studies are more robust than non-blinded studies.
- Large studies are more robust than small studies.
- High statistical significance is more robust than low statistical significance.
Scientific justification can be robust or anecdotal, and some justification is more robust than others. Theories too can be more or less robust, depending on the amount of scientific justification available. Robustness however extends far beyond evidence. But as far as evidence goes, the best case scenario is having a systematic review which indicates that there are a lot of high quality studies that are consistent with a given theory. This scenario constitutes the most robust justification for a theory that we can find.
While this analysis is focused on medical research, a lot of the ideas work for other areas of research as well. Systematic review of available literature can be used to aggregate and analyze studies in physics, chemistry, and other areas as well. Experimental designs, even in general, allow us to control more of the variables and help rule out confounding variables.
Proper Word Choice
While this article started by addressing the quality of scientific evidence, it’s also important to keep in mind what science is, what it can do, and what it cannot do. In science, theory is never confirmed or proven true. Unfortunately science lacks the ability to actually confirm its theories. This issue is known as “the problem of induction” and has been a topic of discussion in philosophy of science since Hume and Popper. So words like confirm, proved, proven, fact, and so on do not belong in science communication. But even if the problem of induction weren’t a problem, there’d still be a degree of uncertainty, and there still is, even with falsification. So it’s always important to look at the quality of evidence.
Sensationalism can be reduced by refraining from such words, and by recognizing the importance of the above hierarchy of evidence. How often do we find “the cure for cancer” or “the secret to immortality” or some mind blowing revaluation that overthrows what we think we know about science? So many media articles are written in sensationalist ways, and part of the problem comes down to not using the correct word choice or recognizing the quality of the evidence.
Citations
An article I read a while back made some interesting points about mercury levels found in freshwater fish. Wouldn’t you like to read the article for yourself? It’s a shame that I didn’t include a citation of the source, huh? Well that’s how I feel every time an article references “a study” and either doesn’t provide any citation at all, or cites another article which probably doesn’t cite the original study either.
This issue is almost as frustrating as sensationalism in scicomm. And in some ways they’re related. If an author provides citation of the original study, at least the reader can, if they so choose, go to the original source and see if the article properly sums everything up.
With hyperlinks, it’s so easy to just include a link to the original study. But it’s also pretty easy, most of the time, to include a full citation. I don’t think it’s necessary; we’re not writing scholarly papers, but it can be nice, and I think it it’s better writing to include the link early on, and a full citation at the end of the article. I use a number of tools to make citations easy.
Citation Machine
I think citation machine is probably one of the most useful tools for pulling data and generating citations. It has a number of options and can pull from websites, scholarly publications, book ISBNs, etc. It has a paid option and a free ad supported option.
doi2bib
This tool is probably the one I use the most when writing scholarly papers. I don’t care too much for specific citation formats, and just write everything in LaTex, which has a wonderful citation system: BibTeX. Aside from the ease of use, something I love about BibTeX is that I can just create a “maybe.bib” file in my project and store all potential sources that I might use! A lot of sites have an export to BibTeX feature, but if not, so long as you know the doi, you can just use doi2bib.
BibTeX entry from URL
BibTeX entry from URL
This extension provides a BibTeX reference for the active URLchrome.google.com
Here’s another great option if you need a BibTeX format. I’ve used it a number of times, and it’s so easy. One issue however is that I’ve found that the access date format doesn’t quite match up on overleaf, which is the primary LaTeX editor that I use, but that’s a minor issue, and easy to fix.
BibTeX to APA
And with all the BibTeX citations that you’re generating, wouldn’t it be great if you could quickly convert them to APA? Well, you can. I haven’t used this tool all that much, and the site itself is in French, but it’s a nifty little gadget.
I do admit that sometimes I play things fast and loose when writing an article for a blog. I don’t always properly cite every single piece of data. But there is nothing more frustrating when an article claims to cite a study, but does not actually provide any easy way to figure out what the study is, or worse, links to another news source that does the same thing. It’s poor science journalism, and I’ve seen it done by a wide range of media outlets, including some major ones, like The New York Times. It’s not that hard to link to the study, or provide a citation at the end of the article. So do it. And please, make sure you’re not sensationalizing and that you actually understand the results of the study.
Summary
So make sure that article information is properly cited. The moment the scientific paper is referenced, link to it. Don’t hide the citation halfway through the discussion of the article. Make sure to avoid sensationalizing words like “proven” and “fact” and always make sure to keep in mind the quality of the evidence being discussed.
This article is really a brief summary of how we need to approach science communication and science journalism. The Freelance Journalism Alliance plans on taking this information, as well as additional information from a variety of sources, in order to develop a series of MOOCs.