So much of medicine is like this. My in-laws are doctors and so many of their diagnoses eventually end up as "we have no idea what's happening, but these steroids should help. Why, we have no clue."
And yet doctors reject all manner of so-called "alternative" therapies on the grounds that they have no plausible mechanism of action.
A family member of mine had a somewhat-debilitating chronic condition in high school, which only got better with a combination of what today are garden-variety alternative therapies (acupuncture, elimination diet, et al). Today, as a doctor, pooh-poohs a lot of that stuff. It's confusing and disappointing to see.
There’s a difference between “we’ve observed in clinical settings that [random herb X] treats the problem, although we have no understanding of how it does so—there’s a bunch of chemicals in [random herb X] that could be the cause, and nobody’s sat down and pinpointed the pharmacodynamics yet” and “this has never been observed in clinical settings to differentiate from placebo.”
- "this has never been studied in a clinical setting"
- "this has been tested in a clinical setting and no differentiation was seen from placebo, but there was an issue with the study that caused that result, but lacking omniscience and epistimic humility we and those who take our findings as gospel are not able to know this is the actual state of affairs, and therefore might mistakenly conclude there is no effect, even though that does not logically follow from these findings"
Those two things fall under my second clause. "This has never been proven to differentiate from placebo" isn't "this has been proven to be indistinguishable from placebo." It literally just means "there's no solid evidence that it works." There might or might not be non-solid evidence; it might or might not have ever even been tried. But "we don't have any reason yet to suspect that it might work" is usually the best knock-down argument a doctor can give against something being worth trying (anywhere other than in a research setting.)
Also, re: the second point, you're talking about studies, but I'm talking about a much weaker criterion: whether any doctors have ever personally seen the treatment make a difference. There are a lot of things that have bad or no studies, but doctors swear by due to a mountain of shared-by-grapevine clinical anecdata. (Most treatments invented before the advent of scientific positivism, for example!) When something doesn't even have that, it's immediately suspicious—doctors really like talking about things when they work!
The statement “this has never been observed in clinical settings to differentiate from placebo” is ambiguous whether it has or has not even been studied - my statement eliminates that ambiguity.
> It literally just means "there's no solid evidence that it works."
There is nothing to indicate whether there is or is not any evidence at all, let alone providing any indication as to the quality of the evidence.
On one hand one might say this is excessive pedantry, or just "semantics", but this ignores the massive amount of epistimically inaccurate memes that are taken as "generally true" "facts" across a wide variety of subjects. I would argue the prevalence of this problem is starting to cause major rifts within society, sometimes manifesting in violence, including death.
I have never met a single doctor that has given me any reason to suspect they happen to possess the extreme levels of pedantic logic required to support the authoritarian judgements they've laid upon me when asking specific questions I've asked after performing significant research on my own. I don't really see why they should be any different than a typical overconfident human being.
And it's not just doctors that are the problem. One can observe that a lack of (or, a lack of use of) logical skills and understanding of epistemology are significant factors in many of the disagreements that exist in the world. Very few people seem to appreciate how limited our understanding of things outside of fields like math and physics that follow very strict rules.
The ambiguity is intentional; the statement is intended to capture all of those cases, because the differences between them don’t matter in practice. No studies, only one good studies, one hundred bad studies: they’re all conditions where the right response is “don’t rely on this as evidence until you’ve done more studies.” And, critically, the case where there’s a bunch of evidence against efficacy cannot be practically distinguished from those cases; it could always turn out that the study methodology was wrong and your N null results will be later countermanded by 2N+1 non-null results. You can’t prove a negative; you have to treat a negative the same as an absence of evidence. (Which, in practice, means treating an absence-of-evidence as a negative.)
> doctors deploying authoritarian judgements despite not being masters of logic
That’s because doctors don’t use their own judgement. Doctors are embodiments of a system mapping a symptom-recognition model (mostly random, gets better with experience; this is the part humans are better at currently) to a fixed, explicit expert system of guidelines built by panels who study meta-analyses to reach recommendations.
In psychiatry, for example, this expert system is reified in the form of a book called the DSM. Doctors don’t need to know why the DSM recommends what it recommends to be right by making recommendations using it; just like a soldier doesn’t need to know what’s over the hill they’re lobbing grenades at in order to be aiding in winning a war. In both cases, the commander giving the orders has better “line of sight” on the problem than the person on the ground does. (And, in both cases, usually information-retaining expert consensus techniques are used to allow the commander+advisors system to actually have a greater-than-1.0xhuman level of rationality.)
> The ambiguity is intentional; the statement is intended to capture all of those cases, because the differences between them don’t matter in practice.
If an alternative remedy is effective, in fact, it matters to people who could benefit from the usage, but have been told, or led to believe (through dishonest, vague, weasel language that "covers" or "captures" more informative descriptions) by doctors or acquaintances who lack skills in logic or epistemology.
> No studies, only one good studies, one hundred bad studies: they’re all conditions where the right response is “don’t rely on this as evidence until you’ve done more studies.”
Tell that to doctors and half-educated iamverysmart people who confidently assert that an "alternative" treatment does not work, full stop, as opposed to "has not been found to work in clinical studies".
> You can’t prove a negative
Correct. And therefore, you shouldn't speak as if you have.
> you have to treat a negative the same as an absence of evidence.
Incorrect. You could simply state what is factually correct, for example: "Herb X has not been found to successfully treat ailment Y in clinical studies. This should not necessarily be treated as conclusive proof that it does not, but at this time scientific studies have not found any evidence of effectiveness. Be wary of anyone who states otherwise, whether for or against the treatment - anyone who claims to have conclusive evidence either way is mistaken."
> That’s because doctors don’t use their own judgement. Doctors are embodiments of a system mapping a symptom-recognition model (mostly random, gets better with experience; this is the part humans are better at currently) to a fixed, explicit expert system of guidelines built by panels who study meta-analyses to reach recommendations.
Doctors (and the general public) should be informed of the difference between guidelines resulting from our current, incomplete understanding, and absolute facts. Very few people, including doctors, seem to realize there even is such a difference.
> Doctors don’t need to know why the DSM recommends what it recommends to be right by making recommendations using it
A recommendation and a statement of "fact" (without actual evidence to support the fact) are two very different things.
> In both cases, the commander giving the orders has better “line of sight” on the problem than the person on the ground does.
You have no way of knowing whether an individual patient may happen to have a factually superior understanding of a particular ailment than their doctor. This is a fine example of a lack of understanding of epistemology.
> And, in both cases, usually information-retaining expert consensus techniques are used to allow the commander+advisors system to actually have a greater-than-1.0xhuman level of rationality.
This statement is better, in that it expresses uncertainty via usage of the word "usually".
Yeah, sometimes people don't understand the difference between "we don't fully understand how/why it works" and "works just the same as doing nothing".
The difference is the ability to privilege a medical establishment that is in control of the clinical setting. Doctors don't like competition. There's really nothing inherently better about randomized control trials as a way of knowing; that's why it is wrong so often.
In my experience body workers outside the medical establishment (like massage therapists for example) have a lot of ad-hoc knowledge about the body and its ailments from years of observation of their patients.
This kind of knowledge is actually a lot like most of the real knowledge employed by doctors - the knowledge of the hands and lived experience rather than some study in a journal.
Do a well designed study and it won’t be “so called ‘alternative’” anymore. We may not know why various real medical interventions work but we have better than “I’ve got a cousin” evidence that they do.
I can't speak to this from personal experience, but I constantly hear the accusation that any sort of solid study on say herbals is going to cost millions of dollars, and that no big-pharma company would even bother to sponsor such a study because they can't patent the result (because it's an herb).
They could patent a delivery mechanism, or isolate some compounds from the herb that work almost as well (that's not exactly an unusual result either). These arguments fall down because they don't make any sense.
There’s at least one pot based drug on the market. So I think there must be some way to profit from medicines that were known anecdotally to be medicinal in herb form.
And, to their credit, they don't really have time to become good statisticians. Once upon a time I had a classmate who was an experienced anesthesiologist, but when it came to probability and stats he was as much of a scrub is anyone else in the room. Why would it be any different? He's spent his life learning and applying his field of practice, not someone else's.
That said, baaic statistical analysis and statistical intuition is a severely underappreciated skill. It's hard to see it almost totally neglected in American public schools, even ones that purport to have a strong focus on STEM.
Yes, that is very true. I'm sorry I didn't include that sentiment in my own post. But it does mean that you should press your doctors for information, do your own research, argue with your doctor if you do have a good handle on stats.
And also, it is very common to read on HN that you should not google and research your own medical issues but instead listen to your doctor, because they are a trained professional.
So many neurological conditions are treated the same way. Like Bell's Palsy. We don't know what causes it, we don't know how to fix it, but steroids help a great deal
