One of the hidden struggles in the culture of science is how to approach hypotheses.
Many ecological papers don't even use them anymore. They might write something like, "To better understand the relationships between x and y, we..."
Sometimes, they are there, but perfunctory. Like, "We hypothesize the technique will increase our explanatory power".
This is not good.
When you look at those papers that follow the time-tested approach of including hypotheses, actually very very few state null hypotheses.
This, is actually not bad.
Because in most sciences, we rarely ever prove things.
And null hypotheses give the illusion of proof.
At least with ecological data, there are always multiple competing hypotheses to explain data.
We do science ultimately to test theories. Hypotheses translate theories into mechanisms that underlie patterns that are observed.
And with just one hypothesis, we are more likely to accept a hypothesis that is wrong.
Thinking in terms of parsimony, where we evaluate multiple hypotheses can be hard to do. Partly, because thresholds for favoring one hypothesis over another do not exist. You have to ascribe certainty to different hypotheses.
How in practice do we do this? How do we evaluate which is the current best hypothesis when you cannot rule out any hypothesis completely?
I should develop this more later in scientific terms, but here's a quick story to illustrate...
Earlier this fall, arrived at Merton College in Oxford, where we'd be staying. We had hauled our suitcases up to the 2nd floor of the building and entered our apartment. A quick look around confirmed that it was as small as we thought it would be. Kendra and I then went to see what would be our bedroom while the kids checked out theirs.
When we got to our bedroom, I saw our bed. I was a tired and curious to know how hard the mattress was that I would have to try to sleep on for the next few months. So, I flopped down on the bed. As soon as I hit the mattress...
...an alarm went off.
A loud one.
And it wasn't stopping.
The first thing I thought was did I really hear that? I did. Kendra did, too.
Was I dreaming this? It didn't seem so. I've never had a dream with such a loud alarm in it. The odds of that seemed really low.
So, if I really was hearing an alarm and I wasn't dreaming...
What then?
Now it's time to assess competing hypotheses.
My first thought was there was a motion-sensitive alarm somewhere in the room. I looked around. I didn't see anything that looked like a sensor. What are the odds that Oxford puts alarms in their bedrooms? Did the kids tripped something? Maybe 1 in a 100?
I'd weight the burglar alarm hypothesis by 0.01.
It is not easy to calculate odds with loud alarms.
Maybe it is a fire alarm. What are the odds that there is a fire? Fires happen rarely. But there are false alarms. Maybe once a year? What are the odds of that happening in the very second I hit the bed? Seems incredibly low.
Quick math: 365 d * 24 h * 60 min * 60 sec = ...can't do math with an alarm going off. Let's say 1 in a million. Weight that hypothesis by a lower number: 0.000001.
But what if they are just testing the alarms? How often would that happen? Once a week? That would still be a lower number. 0.00005.
None of these hypotheses seem very certain. All fell below a 95% threshold. Still, we had to act.
The most parsimonious hypothesis was a "burglar" alarm had gone off somehow. And was still going off.
The conversation wasn't that calm, but we decided to go check with the porter at the college across the street...
Hypotheses need testing.
As we went to the door to head downstairs, the alarm stopped. After probably three minutes of loud, it was finally quiet.
When we walked downstairs, there was a man. He was standing next to the alarm box for the building.
"Sorry, if that was loud. Just testing the alarms."
That was it.
The odds seemed extremely low initially and it didn't seem the most parsimonious hypothesis, but it just so happened that the exact second I flopped down on the bed in our new apartment, someone had decided to test the fire alarm. Which apparently they do every week on Monday at about 10:30 a.m.
***
Parsimony alone didn't lead us to the right answer initially, but it was the approach of trying to formulate multiple hypotheses that gave us a path forward in investigation.
Every scientific project needs to follow the same approach. We didn't set out to disprove a null hypothesis, which would have been something like "there isn't a burglar alarm in our apartment that was just set off". We had multiple competing hypotheses that we were trying to simultaneously evaluate.
There are always more complicated explanations for phenomena than what seems most likely at the time.
If you can approach a scientific problem with multiple working hypotheses, it helps you maintain the mental flexibility to move forward when things seem uncertain.
At each stage, you also need to move forward not with the hypothesis that has a 95% probability of being true, but the one that has the most likelihood of being true.
That's the approach of competing hypotheses.
The hard part in all of this is assigning certainty to each hypothesis (or conclusion).**
**The IPCC struggled with how to do this for years.
I don't necessarily have the answers for how to translate this approach to each scientific question, but delineating potential hypotheses is one of the most crucial steps in the scientific methods.
Writing a paper that acknowledges potential competing hypotheses is a good sign that the scientific method is being carried out well.
No comments:
Post a Comment