Tuesday, May 31, 2016

CUDOS in science

Here's something I hadn't seen before.

I've thought that it's interesting that there isn't a Hippocratic oath for scientists (scientists didn't exist in the days of Hippocrates). It turns out there are norms of scientific society.

I read about this in Wootton's book, but hadn't ever heard about them. Apparently, these norms were described in 1942 by Merton in his description of the sociology of science.

The norms of science go by the acronym of CUDOS:

Organized Skepticism

The wikipedia page describes them fairly well. As does this blog post.

In short, these norms describe the ideals of science. The results should be open to everyone. Ideas (and opportunities) are evaluated blind to the characteristics of the individual. Scientists report results independent of the consequences of the outcome. All ideas are subject to scrutiny.

Wootton's analogy between science and the law in these norms is pretty interesting. The legal profession holds similar ideas**. For example, evidence should never be withheld to opposing parties, which is similar to communalism.

**Though Wootton does not delve into it, the adversarial nature of legal actions is not replicated generally in science even though there is still a tradition of a "defense" of theses or dissertations.

Most scientists believe they deserve more kudos for the work they do. Apparently CUDOS are built into science.

Monday, May 30, 2016

Book review: The Invention of Science

Currently, science is undergoing a convulsion. The very way that science operates is changing. It's a change that appears to be unprecedented in modern times.

For the first time, science is being forced to deal with bias. Questions of reproducibility have become a crisis. The review process is under renewed scrutiny. The nature and openness of publishing is being assaulted legally and illegally. Everyone from editors to scientists to funding agencies are being forced to reckon with consequences of retractions at an unprecedented rate.

The Invention of Science, a new book by David Wootton addresses none of these modern ills. But, sometimes, modern crises are an important time to revisit our history. The Invention of Science is an unparalleled examination of the long, slow (and sometimes convulsive birth of science).

Note, this thing is a wrist-breaker. 600 pages before you get to the endnotes. That's a good thing. Understanding the history of a topic is not something to do in Cliff Notes form. You need a comfortable chair and a pen for the margins to absorb the lessons.

The thesis of this book is that science (as we currently define it) once did not exist. Knowledge was generated through means other than science. In order for science to be invented, a number of conventions had to be created, too. We needed a new vocabulary. People needed to act and interact differently. The conceptual framework that we recognize as scientific had to not only be assembled, it had to displace previous frameworks.

A book of this scope is hard to summarize with any justice.

Here is the first sentence of the book. "Modern science was invented between 1572, when Tyco Brahe saw a nova...and 1704, when Newton published his Opticks...." Science took a bit over 100 years to invent. It's only a bit over 300 years old.

Over a hundred years to invent something that seems so simple that we do it every day? Why so long?

The book answers why it wasn't as easy as people might think.

The middle chapters are the ones I've spent the most time on. These are their titles: Facts, Experiments, Laws, Hypotheses/Theories, Evidence and Judgment.

These chapters lay out the history of the main elements of the modern scientific approach.

I'm going to have to read these chapters one or two more times before I can crystallize them, but their scopes are the raw material for anyone trying to understand if not shape modern science.

For example, the word "fact" (with its modern meaning) did not exist in any language. The Greeks and Romans had no word for "fact". The concept of a "fact" did not exist. And facts are not the same as the truth.

Let me quote here.

"What is a fact? It is a sort of trump card in an intellectual game...Facts are a linguistic device which ensures that experience always trumps authority and reason."

Facts are a linguistic device? Since when is the truth a device? Facts must be something other than what we recognize them to be.

The experiments chapter describes a number of the early experiments. Here's a quote: "This is the first 'proper' experiment, in that it involves a carefully designed procedure, verification (the onlookers are thereto ensure this really is a reliable account), repetition and independent replication, followed rapidly by dissemination."

When did this happen? 1648 when a brother-in-law of Pascal climbed a mountain with a barometer.

But note his definition of an experiment. It involves verification. Repetition. Replication. Followed by  dissemination. Our modern crisis comes about because of a lack of verification, repetition, and replication (or reproducibility as we refer to it). Only touched on, the author highlights the motto of an Italian society. The motto was: provando i reprovando. Test and retest. Hard to imagine that as any modern society's motto.

The evidence and judgement chapter has interesting nuggets, too. In part, it examines the legal frameworks of different European countries, which affects how scientists came to prove things. Drawing techniques for a judicial system that relies on judges vs. juries leads to different ways of conducting science. That thumbprint is still with us today. Like any organism that has evolved, modern science still bears the marks of its history and past forms.

Here's a quick example he provides:

"A friend of mine was once in hospital in Paris. The doctors told him that they had an hypothesis regarding the nature of his illness which they intended to prove, where in England they would have told him that he had certain symptoms which suggested a diagnosis which they would run tests to confirm."

This is a subtle difference, but one whose distinctions should be obvious to anyone practicing science. Different paths do not always lead to the same destination, so choose the path wisely.

Right now, our science is in the middle of a transformation. The question is whether a new layer will simply be added or whether parts will be torn down and rebuilt. Anyone who offers an opinion on how science should be reformulated is wise to know it's history. This is a good book to start on that.

But buy it in hardcover so you can write in the margins.

The only drawback is that the margins are not wide enough.

Tuesday, May 24, 2016

Why we cite papers

Scholarly works are set apart from other types of writings by the use of citations. An essay on natural history might cover a scientific topic, but it is just an essay until it contains citations. Scientific papers are not scientific without citations.***

***This blog post is certainly not scientific...no citations here. OK, maybe one.

Most scientists do not question the need for citations nor the role they play in the paper itself. When we do not have a common understanding of the role of citation, we have trouble determining when citations are improper and what to do when what we think to be true shifts.

Most of us think that the big debates about citations is formatting. Do we number our citations or list the authors and dates each time? There are deeper issues that that. They have nothing to do with formatting.

The first time I really thought about citations is I remember that Stephen Jay Gould once got into trouble for citing a paper in his thesis that was not contained in his school's library.* His advisors questioned the link between the statement he was making and the original citation. They were not refuting that his statement wasn't true. Only that he didn't know it was true, because he could not have examined the original source. Another author's judgment on the assessment of truth was insufficient. That's how rigorous citation can be.

*This is a place where a citation is really needed. But, I can't remember which of his books I read this in. Structure of Evolutionary Theory? Panda's Thumb? I'm fuzzy on the details here, but whether it happened or not, it could have happened, which is all that is necessary here.

When I think about how I use citations, I feel there are two types of citations that I use.

The first I call vertical citations.

Vertical citations are the links between what has been found to be true in the past and a statement we currently would like to make to establish the truth.

For example, here is the first sentence of a paper that I just submitted to a journal:

There are approximately 1 billion cattle in the world with cattle populations steadily increasing over the past few decades (Estell et al. 2014).

This is a vertical citation. I am going back into the literature to provide evidence of the truth of a statement. I personally have not counted how many cattle there are in the world. Nor have I determined whether cattle populations are increasing or decreasing over the past few decades. So, instead of going out and counting cattle, I cite a paper that has established this to be true or has cited the papers that have established this to be true.  The paper I chose to cite is Estell et al. 2014***

***et al. stands for et alia (in the neuter form), which means and others in Latin. Et alia is almost always abbreviated et al., which is funny because we really aren't saving that many characters. Really just one. I think, in part, it gets abbreviated because the actual Latin phrase depends on whether the "others" are male, female, or both. Easier to write "et al." than determine whether et alii, et aliae, et alia is more appropriate.

So, when are vertical citations necessary?

Any time we make a statement in a scientific paper about what we consider to be true outside of the personal experience we are describing, a citation is necessary.

Any time.

If we want to say that there are a billion cattle in the world, we need a citation. If we want to say that atmospheric CO2 concentrations are increasing, we need a citation. The sky is blue? Citation. Gravity exists? Citation.

Now, if we want to say that we performed a certain procedure in an experiment, we do not need a citation. We hold it true that we might have measured something at a certain temperature, but there is no citation for this since it comes from our experience, not the literature.

Vertical citations go back into the literature to provide justification for the truth of statements we are making. Think of the Newton's phrase, if I have seen further, it is by standing on the shoulders of giants...When we cite a previous work, we are placing our foot on the shoulder of a giant that has come before us. We are reaching down vertically to build something taller.

As opposed to vertical citations, there are also horizontal citations. Like vertical citations, they reach down into the literature to establish the truth, but the purpose is different.

Horizontal citations are primarily for context. In the introduction, horizontal citations are typically used to identify intellectual tension. Study A found this. Study B found that. This and that cannot be both true under our current intellectual framework. We cite these papers to show what other researchers have found to justify our work.

In the discussion, horizontal citations are used in a similar manner, but it is not to establish that there is intellectual tension, but to see if there is intellectual tension. Study A found this. Study B found that. We found this, too. Therefore, it seems like this is more likely to be true than that.

With horizontal citations, we are not citing other giants, but instead other dwarfs (or other Isaac Newtons).**

**the original metaphor was "dwarfs standing on the shoulders of giants". Citation here. We think of Newton as a giant now, but originally he would have been a dwarf in the metaphor.

So, when I think about how I reference the literature, it is generally vertically or horizontally. I am either reaching down to stand taller, or reaching across to build linkages.

That's probably a long enough post for now. Down the line, I should cover the consequences of failing to cite the literature correctly and the consequences of determining that the findings of a published paper was not true: what happens when a giant tumbles?

Mostly as a note to myself, comparing legal citations and scientific citations is also instructive. The law only cares about what was legally true at the time the law was being examined. Science cares about what is known to be true at the time the scientific fact was established and after. Hence, changes in the law and changes in scientific understanding have much different consequences.