I must have blogged about this before. Oh wait, I have. Twice. Three times. Oh fine, just count it as infinity and let's move on.
Anyhow, brand new book I came upon in an old edition of the New Yorker- Philip Tetlock's Expert Political Judgement: How Good Is It? How Can We Know? First, I love the fact that he's a psynchologist and not a commentator. It's the right perspective to take on this stuff. Second, I love the fact that he's done this on a long term basis, looking at two hundred and forty eight people commenting over 20 years. Big n... Dave like. But the piece de resistance comes right in the second paragraph review:
The accuracy of an expert's predictions actually has an inverse relationship to his or her self-confidence, renown, and, beyond a certain point, depth of knowledge. People who follow current events by regularly reading the papers and newsmagazines regularly can guess what is likely to happen about as accurately as the specialists whom the papers quote.
PRICELESS. So basically, the bum on the street is probably just as likely to give you decent advice as the guy who has been studying the situation for 20 years. Mind you, I actually think this is more an effect of the total seeming randomness to events (Tetlock tested huge society changes like political freedom, economic growth, repressions, recessions) than experts actually being misinformed, but still, it makes you put into context some of the insanity with which we trust someone who gets quoted.
Just one more thing I'd do if I had a talk show. The first thing an expert would have sitting right next to his/her head would be a scorecard of previous predictions and/or quotes from the past. Just so the audience could get a sense how good this guy/gal was.
Here's the book I was talking about:
