This book is similar to Steven Levitt's
Freakonomics: A Rogue Economist Explores the Hidden Side of Everything (P.S.)
, Nassim Taleb's
The Black Swan: Second Edition: The Impact of the Highly Improbable: With a new section: "On Robustness and Fragility"
, and James Surowiecki's
The Wisdom of Crowds
. All four books explore the intersection of data, human behavior, and outcomes. They explain how to quantify outcomes within the financial markets, professional sports or elections.
This book is especially interesting because Nate Silver has honed firsthand his statistical skills onto numerous domains including professional poker, baseball performance forecasting (he developed one of the best software program to do that), political elections (his "fivethirtyeight" blog). And, when he is not a firsthand practitioner he is a first class investigator.
The first seven chapters cover the errors and successes people have had in forecasting in various disciplines. Chapter eight is the most pedagogical, as the author explains the basics of Bayes Theorem that he considers as an overall solution to many of the errors we make in forecasting. The last five chapters focus on Bayesian thinking within various disciplines.
Nate Silver's coverage of the credit rating agencies "Catastrophic failure of prediction" (first chapter title) is excellent. In a single sentence on page 13, he captures the cause of the financial crisis: "In advance of the financial crisis, the system was so highly leveraged that a single lax assumption in the credit rating agencies played a huge role in bringing down the whole global financial system." Silver states that the AAA rated CDOs were deemed to have a default rate of only 0.12%. The actual default rate was 28% or over 200 times greater! This was because the rating agencies missed out the correlation between mortgage default rates at different locations when a nationwide home price downturn hit (see figure 1.2 on page 28. Watch out that he mislabeled column 3 and 4 from the right). Silver assesses that overall leverage was too high during the housing bubble. Fannie Mae and Freddie Mac had a debt-to-equity leverage of 70-to-1. Lehman Brothers and other investment banks were leveraged over 30-to-1. Borrowers had often loan-to-value ratios of 100% on their homes. The volume of credit default swaps, MBS, CDOs represented 30 to 60 times the volume of home sales during the bubble years (fig. 1.5 page 35). Nate Silver summarizes the errors made. Investors trusted the rating agencies. The rating agencies assumed home prices would never decline on a nationwide basis because they never had since the Great Depression. Lenders and borrowers believed rising home prices would bail them out through refinancing. Policymakers believed the financial system had enough capital and was self-disciplined. And, economists completely missed the ensuing severe recession.
Nate Silver focuses next on political predictions. This field of experts was so bad at predicting it motivated him to enter it by starting his fivethirtyeight blog. He documents their failings extensively. Within this chapter he refers to the theory of Philip Tetlock, professor of psychology and political science at Berkeley. Tetlock had surveyed predictions of experts in various fields. And, he categorized them within two archetypes: the hedgehogs and the foxes. The hedgehogs are dogmatic, rarely change their minds, and are very confident of their forecast. The foxes are just the opposite. They update their forecasts as often as new information warrants it. As a result, they make better forecasts.
The chapter on baseball is one of the best because of Silver's extensive firsthand experience. He uncovers many concepts applicable to many sports such as the age-curve of baseball performance (pg. 81). All sports have a predetermined age-curve. Actually, every single aspects of life including life itself have predetermined age-curves. His description of what it takes to be a successful professional baseball player (pg. 97) has also surprisingly broad applications. The conclusion of the chapter is also fascinating. It describes baseball management as a competitive arms race of intelligence gathering to extract small competitive edges. And, that those competitive edges are short-lived. That's a very interesting application of the Efficient Market Hypothesis.
The chapter on economists documents how inaccurate their forecasts are. The majority can't forecast a recession that has already started as they missed out on the three most recent ones (1990, 2001, 2007). In November 2007, the average economic forecast was 2.4% real GDP growth in 2008. Instead, real GDP shrank by -3.3%. Economists assigned only a 1-in-2000 chance of the economy shrinking that much. Yet, home prices were already declining. Foreclosures had picked up. Bear Stearns had gone belly up six months ago. Those were powerful signals the housing and financial markets were on the edge of a cliff. Also, economists are way too confident. The few times you can extract confidence intervals from the economic profession they are invariably way too narrow because they underestimate the error level within their forecasts (pg. 182). Nate Silver states that: "this property of overconfident prediction has been observed also in medical research, political science, finance, and psychology" (pg. 183). Despite our having so much more data and computer power at our hands, economic forecasting has not improved since 1968. This is because our underlying understanding of cause and effects has not changed much since.
Chapter 8 introduces Bayes's Theorem. Here Nate Silver often refers to a very good book on the subject:
The Theory That Would Not Die: How Bayes' Rule Cracked the Enigma Code, Hunted Down Russian Submarines, and Emerged Triumphant from Two Centuries of Controversy
by Sharon Bertsch McGrayne.
Chapter 9 and 10 about chess and poker are excellent. Kasparov was ultimately beaten by a computer bug. IBM Big Blue made a move late in the last game that did not make any sense (the team who programmed it confirmed it was due to a small programming bug). Kasparov who was in a vulnerable position could not figure out that move and in despair resigned the game and lost the series. The Pareto principle of prediction on page 312 and 314 and the ensuing economics of poker are really interesting. Poker winning are heavily dependent on the one worst player at a table. If he leaves, the winnings are a lot harder to reap.
Chapter 11 on the Efficient Market Hypothesis (EMH) is excellent. Nate Silver states that the stock market is efficient most of the time, although it is never perfectly efficient (that would preclude a market). But, it can be wildly inefficient on few occasions associated with bubbles and crashes. Nate Silver demonstrates how both technical analysis and fundamental analysis do not beat the market over the long run. Fig 11.3 on page 340 shows no correlation between the performance of mutual funds over the 2002 to 2006 period vs over the 2007 to 2011 period. Past performance is no guarantee of future returns. Next, Silver refers to Robert Shiller in showing the market is not as efficient as the EMH entails. Shiller looked at the P/E ratio of the S&P 500 over a trailing 10 year period and looked at prospective returns. And, the longer the period contemplated the greater the negative correlation between trailing P/E levels and future average yearly returns. This suggests that the market can get overvalued. But, the return correction is not apparent until looking at average return over a 10 to 20 year period. Next, Nate Silver refers to the works of Richard Thaler and Daniel Kahneman in behavioral economics to outline how market traders are not perfectly rational. They suffer from herd mentality, overconfidence, and being overly emotional rendering their trading pro-cyclical.
So, if the market is not so efficient, can you beat it? Probably not. On page 345, Nate Silver demonstrates how a hypothetical investor with perfect timing over a decade (1976-1986) would get killed by very small transaction costs. Even though this investor would handily beat the stock market before transaction costs, he would wipe out most of his capital after transaction costs. Silver next tests a prudent investment strategy over the 1970 to 2009 period. He assumes an investor is prudent and sells his position in the S&P 500 index whenever it had declined 25% from its peak and reinvests whenever it recovered 90% of its value. Such an investor would have earned only 2.6% per year vs close to 10% for a simple buy-and-hold strategy. Nate Silver does believe several hedge funds can beat the market. But, they have intellectual and technological resources that no retail investor and few mutual funds can match.
Chapter 12 on climate change is really interesting. He differentiates between where scientists agree and disagree. They all agree that the greenhouse effect exists and keeps the Earth warmer than it would otherwise be; that temperatures have risen over the past century; that greenhouse gases have contributed to that trend; and that water vapor is by far the most potent greenhouse gas (not CO2 as the Media conveys). The majority of scientists agree that rising CO2 concentration does contribute to rising temperature. But, there is a debate regarding how much. Where the scientific community is more divergent is regarding climate models and projections. They acknowledge that Al Gore's
An Inconvenient Truth
deterministic apocalyptic message was way off base.
Nate Silver explains why there is much uncertainty regarding climate models' projections. One uncertainty is figuring out CO2 levels 100 years down the road. Another uncertainty is getting the causal relationships right (there is a lot more than CO2 at play). Another uncertainty concerns whether those models are programmed correctly. Within the vast quantities of computer codes, are there a few bugs that contribute to generating erroneous forecasts? Nate Silver reviews the prediction of the IPCC's 1990 model and observes that temperatures have not risen as fast as the model predicted. Current temperatures are below the model's 95% confidence interval. This lead the IPCC to reduce their baseline temperature increase from 3 degree Celsius per century in 1990 to 1.8 degree in 1995. On page 407, Silver comes up with an interesting application of Bayes theorem applied to rising temperature predictions.
The last chapter on terrorism is intriguing. Terrorist attacks follow a similar Power Law as earthquakes. The frequency of events declines exponentially with increase in intensity. More violent events are much rarer than lesser ones. But, the few major events dominate the data in human casualties. For instance, 9/11 represented more than half of the total fatalities from terror attacks in NATO countries since 1979. Thus, it is worth exploring means of mitigating the impact of such events.
Download the free Kindle app and start reading Kindle books instantly on your smartphone, tablet, or computer - no Kindle device required.
Read instantly on your browser with Kindle for Web.
Using your mobile phone camera - scan the code below and download the Kindle app.
Something went wrong. Please try your request again later.
Book recommendations, author interviews, editors' picks, and more. Read it now.
Frequently bought together
$53.87
Only 1 left in stock - order soon.
Total price:
To see our price, add these items to your cart.
Choose items to buy together.
Similar items that may deliver to you quickly
Page 1 of 1 Start overPage 1 of 1
About the author
Follow authors to get new release updates, plus improved recommendations.
Reviews with images
Which animal would you think defines a good forecaster, fox or hedgehog?
The hedgehog knows one big thing, but the fox knows many little things.If an original method is not surely working, the hedgehog is reluctant to change, but the fox is tolerant of complexity and is adaptable to find a new approach. That is why the author suggests being foxy is a right attitude toward a good forecaster.We live in a world in which information is pervasive so that the gap between what we know and what we think we know is widening. As the study has shown, even the experts usually make incorrect predictions. For example, the probability of the skyscraper being crashed into by the terrorists is 0.05%. The possibility would rise to 38% given that the first building is under attack. If we could use one of the principles, “Today’s Forecast is the First Forecast of the Rest of Your Life” in this book, we could make a better forecast possible today—regardless what we said yesterday, last month, or last year-- and prevent the formidable catastrophe from happening.Other suggestions the writer proposes are below:-Think probabilistically: Acknowledging the real-world uncertainty in our forecast.-Look for consensus: It’s not easy to be objective. Other options could help us see the world in different viewpoints to reduce biases.-Weighing qualitative information- accounting for the qualitative information along with quantitative factorThis book is a little long but readable, not a formula-heavy, general science book.It consists of four sections. The first section considers the failures of predictions in finance, baseball, and politics. Then, the author gives the readers some advice about how we can apply our judgment to the data without succumbing to the biases. The second section focuses on dynamical systems (weather, economy, earthquake, and economy) that make forecasting more difficult. Following the third section, it turns toward a solution by an introduction of Bayer’s theorem. Finally, the discussion of applying Bayer’s theorem to more existential types of problems.If you’re interested in general science books or statistically forecasting, please enjoy it.However, if you are the audience who need depth in measuring and making data-driven decisions, you might not appreciate this book as much. I would suggest to read “How To Measure Anything” by Douglas W. Hubbard.
Top reviews from other countries
Tiago Irineu
Many examples about probabilistic thinking, with an underlying defense of Bayesian statistics
Reviewed in Brazil on August 30, 2021
Christopher Canty
Extremely Well Written and Interesting
Reviewed in Germany on November 14, 2023
