Richard Feynman, in his 1974 cargo-cult science commencement address:
If you make a theory, for example, and advertise it, or put it out, then you must also put down all the facts that disagree with it, as well as those that agree with it…
In summary, the idea is to try to give all of the information to help others to judge the value of your contribution; not just the information that leads to judgment in one particular direction or another.
Unfortunately, the average scientific journal article doesn’t follow this principle. I wouldn’t go so far as to say that the average article is just a sales job, but the emphasis is really on giving the information that favors the author’s theory. I say this based on my experience as a journal editor (for PLoS ONE), a reviewer (for a few dozen journals), and as a reader and author absorbing the norms of my field.
It’s a kind of scientific integrity, a principle of scientific thought that corresponds to a kind of utter honesty—a kind of leaning over backwards. For example, if you’re doing an experiment, you should report everything that you think might make it invalid—not only what you think is right about it: other causes that could possibly explain your results; and things you thought of that you’ve eliminated by some other experiment, and how they worked—to make sure the other fellow can tell they have been eliminated.
Again, I don’t think most scientists follow this principle. But evidence charts can yield more balanced scientific communication. Currently, formal scientific communication occurs almost entirely through articles— a long series of paragraphs. For someone to easily digest an article, there has to be a strong storyline running throughout, and the paper cannot be too long. Those requirements can tempt even one of the highest integrity to omit some inconvenient truths, to use rhetorical devices to sweep objections under the rug, to unashamedly advance the advantages of one’s own theory and that theory alone. If you don’t make a good sales job of it, the reader will just move on to a scientist who does, and you won’t have much impact. I don’t think the situation is universally that bad, but there is definitely a lot of this going on.
An evidence chart is more like a list of the pluses and minuses of various theories, and how the apparent minuses might be reconciled with the theory. There’s less room for rhetorical devices to obscure or manipulate things, and the form may be more suited for driving the reader to make up their mind for themselves. Of course, an individual scientist making a chart may still omit contrary evidence or make straw men of the opposing theories, but the evidence chart format may make this easier to recognize. And, we’re working on collaborative and adversarial evidence charts to bring the opposing views to the same table.
Email me if you’re interested in participating. I like to think that Feynman would be in favor of it.