A comment on Twitter this morning reminded me of a few points from Philip Tetlock’s book Expert Political Judgment.
- Experts are under pressure to form opinions quickly so they can respond to interviewers.
- You don’t get invited to appear on talk shows for having conventional opinions. An expert who, after long deliberation, decides things are going to continue the way they’ve been going is not likely to appear in the press. This causes a selection bias in the predictions that receive publicity. It also creates incentives for experts to make sensational predictions.
- Experts have many facts to draw on, and so can be uncommonly good at confirming incorrect beliefs. Someone less knowledgeable might be forced to do some research.
Tetlock’s book focuses on political experts, though the same principles apply to other areas.
Reconfirming my belief that the output of talking heads (and even news in general) is almost entirely noise.
Also, you should add:
4. You don’t get invited to appear on talk shows, or given any media exposure, if you can’t convey your analysis in a few-second sound bite.
This has the tendency to force the expert to oversimplify, either in analysis or presentation.
Imagine a medical “expert” being asked if poisons are good for you. What do you mean, you can’t give a simple “yes” or “no” answer?
On the other hand, the media seem to target a child’s mentality. In that case the correct answer (in my sincere and honest opinion) to the above question is, “No! Never! Never ever! Yuck!!” even if it is not strictly true. Plus it is much more entertaining, especially if the “expert” mugs for the camera!
To John Venier:
I assume you are thinking about chemotherapy when you mention “poison”? In that instance, the poison might actually do a cancer patient some good!
Clift, I thought about chemo too when I read JV’s comment. Here’s another examples: some nutrients are known as vital poisons. A simple example is iron: we need it to survive, but it’s harmful in large amounts. A more surprising example is arsenic. We actually need a little of it.
John and Clift, for sure chemotherapy is a great example of something which is principally a poison whose action as a poison is the therapeutic effect which can be lifesaving. Same thing for Botox — which was used to treat nerve disorders before the cosmetic effect was accidentally noticed. Before the medical use it was well known as one of the most deadly poisons there are. In that sense it is an even clearer example than a lot of chemotherapies, which are usually designed with a medical purpose in mind. Another example is foxglove, a deadly poison plant which can treat congestive heart failure. Although cultivated primarily for decoration and widely known as a deadly poison, its medical action to treat dropsy was also known at least to some for a long time. But the toxic action is again the same as the therapeutic action. Then come the drugs which have toxic side effects which are not the therapeutic effect. Finally, there are vital poisons as John mentions. Besides the metal ions he links to, vitamin A is a good exmple of something clearly poisonous in large amounts but necessary in humans.
So it was just an off-the-cuff example which popped in my head when thinking of a seemingly straightforward question with an obvious pithy answer, but which really deserves much more discussion, particularly coming from an “expert”.
It was also on my mind since I go camping and rambling with my kids.
A toxicologist (and many others such as myself) would say “Dose makes the poison.” It’s a succinct and accurate statement, but as viewers may be unfamiliar with the phrase or even how to interpret it, the TV news needs someone to unpack it and stretch it out to 30 seconds, as in “a little bit can be good for you or even needed, but too much can kill you.”
Even here, this whole concept may be too complicated for the mass media, whose idea of nuance is reporting “eggs are good for you”one day and “eggs are bad for you” the next.