1/ Thinking in Bets: Making Smarter Decisions When You Don't Have All the Facts (Annie Duke)

Thread

"World-class poker players taught me to understand what a bet really is: a decision about an uncertain future."

https://t.co/Nr2zCrVve3

2/ "We can get better at separating outcome quality from decision quality, say “I’m not sure,” learn strategies to map out the future, become less reactive, build and sustain pods of fellow truthseekers, and recruit our past and future selves to make fewer emotional decisions."
3/ "Mistakes, emotions, losing—those things are all inevitable because we are human. The approach of thinking in bets moved me toward objectivity, accuracy, and open-mindedness. That movement compounds over time to create significant changes in our lives."
4/ "Most people didn’t want to give Carroll credit for having any reason at all for his call. Why did so many people so strongly believe he got it wrong?

"In four words: the play didn't work.

"He had control over the quality of the decision but not over how it turned out."
5/ "Hindsight bias is the tendency, after an outcome is known, to see the outcome as having been inevitable. When we say, “I should have known,” or, “I should have seen it coming,” we are succumbing to hindsight bias: an overly tight connection between outcomes and decisions."
6/ "We link results with decisions even though it is easy to point out indisputable examples where the relationship isn’t so perfectly correlated. No sober person thinks getting home safely after driving drunk reflects a good decision or good driving ability."
7/ "Working backward from results is susceptible to cognitive traps, like confusing causation with correlation or cherry-picking data to confirm a narrative. We pound square pegs into round holes to maintain the illusion of a tight relationship between outcomes and decisions."
8/ "Making more rational decisions isn’t just a matter of willpower or consciously handling more decisions. Our deliberative capacity is already maxed out. We don’t have the option, once we recognize the problem, of merely shifting the work to a different part of the brain."
9/ "We need shortcuts, but they come at a cost. Many decision-making missteps originate from the pressure on the reflexive system to do its job fast and automatically.

"We’ve tuned out, cut short, and are predisposed to dismiss anything that varies from what we already 'know.' "
10/ "We have defaults we rarely examine, from gripping a pencil to swerving to avoid an auto accident.

"Being aware of irrational behavior and wanting to change is not enough, just as knowing that you are looking at a visual illusion is not enough to make the illusion go away."
11/ "We can look for practical work-arounds like carrying around a ruler.

"Solving the problem of how to execute (avoiding common decision traps, learning from results in a rational way, keeping emotions out of the process) is even more important than innate talent in poker."
12/ "Our lives—business, saving/spending, health and lifestyle choices, raising children, relationships—involve uncertainty, risk, and occasional deception. Trouble follows when we treat life decisions as if they were chess decisions (no hidden information and very little luck)."
13/ "When we think probabilistically, we are less likely to use adverse results alone as proof that we made a decision error, because we recognize the possibility that the decision might have been good but luck and/or incomplete information (and a sample size of one) intervened."
14/ "When we move away from a world where there are only two opposing, discrete boxes that decisions can be put in, we start living in the continuum between the extremes. Making better decisions stops being about wrong or right but about calibrating among all the shades of grey."
15/ "Job and relocation decisions, sales negotiations and contracts, buying a house are bets. Ordering the chicken instead of the steak is a bet. Everything is a bet.

"We are betting that the future version of us that results from the decisions we make will be better off."
16/ "If we can find ways to become more comfortable with uncertainty, we can see the world more accurately and be better for it.

"We form beliefs in a haphazard way, believing all sorts of things based just on what we hear out in the world but haven’t researched for ourselves."
17/ ...
1. We hear something
2. We believe it to be true
3. Only sometimes, later, if we have the time or the inclination, we think about it and vet it, determining whether it is, in fact, true or false.
...
18/ ”People find it very easy to believe and very difficult to doubt. Believing is so easy, and perhaps so inevitable, that it may be more like involuntary comprehension than rational assessment.”

"A quick Google search will show many of our commonly held beliefs to be untrue."
19/ "For example, we use all parts of our brain. The 10% figure was made up to sell self-improvement books; neural imaging and brain-injury studies disprove the fabrication."
20/ On the food pyramid:

"The same belief-formation process led hundreds of millions of people (led by advice drawn, in part, from research secretly funded by the sugar industry) to bet the quality and length of their lives on their belief about the merits of a low-fat diet."
21/ "We might think of ourselves as open-minded, but the research conclusively shows otherwise. Instead of altering our beliefs to fit new information, we do the opposite, altering our interpretation of that information to fit our beliefs."
22/ "Once a belief is lodged, it becomes difficult to dislodge. It leads us to notice and seek out evidence confirming our belief, rarely challenge the validity of confirming evidence, and ignore or work hard to actively discredit information contradicting the belief."
23/ "The Internet is a playground for motivated reasoning. It provides the promise of access to a greater diversity of information sources and opinions than we’ve ever had available, yet we gravitate toward sources that confirm our beliefs, that agree with us."
24/ "Even when directly confronted with facts that disconfirm our beliefs, being wrong doesn’t fit into [our self-perception]. It feels bad, so we ignore or discredit new information.

"The smarter you are, the better at rationalizing and framing data to fit your point of view."
25/ "Subjects identifying as liberal interpreted data to support their belief (gun control reduces crime). Conservatives interpreted the same data to support the opposing belief.

"More numerate people made more mistakes interpreting data on the emotionally charged topic."
26/ "Being asked if we are willing to bet money makes it much more likely that we will examine our information in a less biased way, be more honest with ourselves about how sure we are of our beliefs, and be more open to updating and calibrating our beliefs."
27/ "If you went around challenging everyone with “Wanna bet?” it might be difficult to make friends. But we can change how we think about decisions.

"Practically every fact we’ve ever known has been subject to revision or reversal. (Samuel Arbesman’s The Half-Life of Facts)"
28/ "More than a third of 'extinct' species have subsequently been rediscovered. Given that even scientific facts can have an expiration date, we would all be well-advised to look hard at our beliefs, which are formed and updated in a much more haphazard way than in science."
29/ "Our narrative of being a knowledgeable, educated person who holds quality opinions isn’t compromised when we use new information to calibrate beliefs vs. having to make a full-on reversal. This shifts us away from treating information that disagrees with us as a threat."
30/ "By saying, “I’m 80%” and thereby communicating we aren’t sure, we open the door for others to tell us what they know. They realize they can contribute without having to confront us by saying or implying, “You’re wrong.”
31/ "When scientists publish results of experiments, they share their methods of gathering and analyzing data, the data itself, and confidence levels.

"By institutionalizing the expression of uncertainty, scientists invite their community to test and challenge the results."
32/ "Was Jane a better salesperson than Joe? Or was it because the company updated its product line? Did a low-cost competitor go out of business? Or is the difference due to any of a variety of other unconsidered reasons? It’s hard to know run a controlled experiment."
33/ "We take credit for the good stuff and blame the bad stuff on luck so it won’t be our fault. The result is that we don’t learn from experience well.

"When we figure out why something happened, we look for a plausible reason, but one that also fits our wishes."
34/ "In multiple-vehicle accidents, 91% of drivers blamed someone else. Most remarkably, MacCoun found that in single-vehicle accidents, 37% of drivers still found a way to pin the blame on someone else."
35/ "If our only options are being 100% right or 100% wrong, then information that potentially contradicts a belief requires a total downgrade from 'right' to 'wrong.' There is no “somewhat less sure” option, so we ignore or discredit information to hold steadfast in our belief."
36/ "On a fundamental level, fielding someone’s bad outcome as their fault feels good to us. On a fundamental level, fielding someone’s good outcome as luck helps our narrative along.

"The way we field other people’s outcomes is part of our self-serving bias." (p. 102)
37/ "If I field my win (loss) as having to do with my skillful play (luck), then my opponent in the hand must have lost (won) because of their less skillful play (luck).

"Any other interpretation would create cognitive dissonance." (p. 102)
38/ "By several estimates, all of these variables (objective circumstances, demographic variables, life events) put together account for no more than 8%-15% of the variance in happiness. What accounts for most of the variance is how we’re doing compared to other people." (p. 103)
39/ "We can work to feel good by being good decision-makers. Instead of feeling bad when we have to admit a mistake, what if it came from missing a learning opportunity?

"We are doing well by comparison because we are doing something hard that most people don’t do." (p. 107)
40/ "If you walked into a poker room and threw around words like “always” and “never,” you’d find yourself challenged to a bunch of bets. It’s easy to win against someone who takes extreme positions.

"Thinking in bets triggers an exploration of alternative hypotheses." (p. 111)
41/ "Outcomes are rarely attributable to a single cause: there is almost always uncertainty.

"For negative outcomes, you can recognize, in addition to things you can improve, things you did well and things outside your control. You realize that not knowing is okay." (p. 113)
42/ "In exchange for losing the fear of taking blame for bad outcomes, you also lose the unadulterated high of claiming good outcomes were 100% skill. That’s a trade you should take: being wrong (losing) feels about twice as bad as being right (winning) feels good." (p. 113)
43/ "If our opponent is someone just like us but who isn’t working to transform his outcome-processing routines, maybe he (this prior version of ourselves) will pick up five learning opportunities instead of the ten that we pick up.

"The benefits compound over time." (p. 115)
44/ "Not all situations are appropriate for truthseeking, nor are all people interested in the pursuit. It needs agreement by both parties to be effective.

"It isn’t for everyone; it must be freely chosen to be productive and sustainable." (p. 122)
45/ "We need to make it clear the social contract is being modified, or defensiveness will rear its ugly head. So, while we find some people to think in bets with us, with the rest of the world, it is generally better to not go around saying, “Wanna bet?” willy-nilly." (p. 125)
46/ "Other people can spot our errors better than we can.

"As long as there are three people in the group (two to disagree and one to referee), a truthseeking group can be stable and productive." (p. 125)
47/ "Being in a group can improve our decision quality by recognizing where our thinking might be biased, but it can also exacerbate our tendency to confirm what we already believe (→ an 'echo chamber').

”It's important to have intellectual and ideological diversity.” (p. 129)
48/ "A bet is a form of accountability. If we’re in love with our own opinions, it can cost us in a bet.

"We can't underestimate how hard it is to maintain diversity of thinking. After all, it feels good to hear our ideas echoed back to us." (p. 140)
49/ "Researchers found that diversity improved the panel’s work. Even though two politically similar judges could dictate outcomes, there were differences between heterogeneous and homogeneous panels. A single panelist from the other party had “a large disciplining effect.”
50/ "Sunstein’s group found that federal appellate judges need the diverse viewpoint of an opposing-party appointee. Judges, they found, followed the human instinct of succumbing to groupthink. “Our data provide strong evidence that like-minded judges also go to extremes...
51/ "the probability that a judge will vote in one or another direction is greatly increased by the presence of judges appointed by the president of the same party. We claim to show both strong conformity effects and group polarization within federal courts of appeals.” " (p.142)
52/ "We don’t realize when we are in the echo chamber ourselves: we’re so in love with our own ideas that it all sounds sensible. Virtually everyone asserts, “I’m in the rational group thinking these things through. The people on the other side are in an echo chamber.” " (p. 144)
53/ "In the 1990s, liberals among social psychologists outnumbered conservatives 4-to-1. More recent surveys show that the ratio has grown to greater than 10-to-1, sometimes far greater.
54/ "A tendency to hire for a conforming worldview combined with the discouraging aspects of being so decisively outnumbered suggests that this situation won’t get better. 10% of faculty identify as conservative, compared with just 2% of grad students and postdocs." (p. 145)
55/ "The authors identified instances in which political values became “embedded into research questions in ways that make some constructs unobservable and unmeasurable, thereby invalidating attempts at hypothesis testing.” " (p. 146)
56/ "Peer review ”offers much less protection when the community of peers is politically homogeneous.”

"Experimental studies found that reviewers “work extra hard to find flaws with papers whose conclusions they dislike and are more permissive when they endorse conclusions.”
57/ "Even among judges and academics, we can see how strong confirmation bias is.

"Check your Twitter followers. It’s a safe bet that the bulk of them are ideologically aligned with you. If that’s the case, start following some people from the other side of the aisle." (p. 148)
58/ "There is potentially large number of published studies with results subsequent researchers could not replicate.

"At least one study has found that a betting market where scientists wager on the likelihood of results replicating was more accurate than expert opinion alone.
59/ "Experts engaging in traditional peer review, providing their opinion on whether an experimental result would replicate, were right 58% of the time. A betting market in which the traders were the same experts and had money on the line predicted correctly 71% of the time.
60/ "Companies implementing prediction markets to test decisions include Google, Microsoft, General Electric, Eli Lilly, Pfizer, and Siemens. People are more willing to offer their opinion when the goal is to win a bet rather than get along with people in a room." (p. 150)
61/ Richard Feynman on scientific sharing/honesty: "If you’re doing an experiment, you should report everything that you think might make it invalid—not only what you think is right about it: other causes that could possibly explain your results."
62/ "Details that make us uncomfortable or require even more clarification to explain away are exactly the details we must share. The mere fact of our hesitation and discomfort is a signal that such information may be critical to providing a balanced account." (p. 155)
63/ "We have all experienced situations where we get two accounts of the same event, but the versions are dramatically different because they are informed by different facts and perspectives. This is known as the Rashomon Effect, named for the 1950 cinematic classic." (p. 156)
64/ "We can’t assume one version of a story is accurate or complete.

"Sharing data is the best way to move toward accuracy because it extracts insight from your listeners of the highest fidelity." (p. 158)
65/ "Don’t disparage or ignore an idea just because you don’t like who or where it came from.

"Likewise, when we have a positive opinion of the messenger, we tend to accept the message without much vetting. Both are bad." (p. 160)
66/ "When I had the impulse to dismiss someone as a bad player, I made myself find something that he did well.

"Even when I determined that his strategy wasn’t profitable, I had a deeper understanding of my opponent’s game, which helped me devise counter-strategies." (p. 161)
67/ This also applies to traders.

The Hedge Fund Market Wizards used lots of strategies, but most didn't consider combining for diversification.
https://t.co/Rcofe6XAot

Similarly, AQR finds that superstar investors concentrate their strategies/exposures:
https://t.co/hfFqnUUZpi
68/ "We have built-in conflicts of interest, interpreting the world to confirm our beliefs to avoid having to admit ignorance or error, to take credit for good results, to find reasons bad results were due to factors outside our control, to compare well with our peers." (p. 164)
69/ "Feynman recognized that in physics, there is demonstrable outcome bias. If those analyzing data knew or could intuit the hypothesis being tested, the analysis would be more likely to support it. Slicing and dicing the data is vulnerable to bias, even unconsciously." (p. 165)
70/ "If the group is blind to the outcome, it produces higher fidelity evaluation of decision quality. The best way to do this is to deconstruct decisions before an outcome is known.

"Attorneys can evaluate trial strategy before the verdict comes in." (p. 166)
71/ "A debate may be of only marginal value because the people debating are biased toward confirming their position, often creating stalemate. If two people disagree, a referee can get them to each argue the other person's position with the goal of being the best debater."
72/ "Skepticism approaches the world by asking why things might not be true rather than why they are true.

"We need to be particularly skeptical of information that agrees with us because we know that we are biased to just accept and applaud confirming evidence." (p. 169)
73/ "If we embrace uncertainty and wrap that into the way we communicate, confrontational dissent evaporates because we start from a place of not being sure.

"Expression of disagreement is just another way to express our beliefs, which we acknowledge are probabilistic." (p. 170)
74/ "When seeking advice, we can ask specific questions to encourage the other person to figure out reasons why we might be wrong. That way, they won’t be as reticent to challenge the action we want to pursue; we’re asking for it." (p. 171)
75/ "Those of us more actively seeking dissent will generally be in the minority. That doesn’t mean that truthseeking is off-limits. It just means we have to take the most constructive, civil elements of truthseeking communication and introduce them carefully." (p. 171)
76/ "Lead with assent. Listen for things you agree with, state those and be specific, and then follow with “and” instead of “but.” People like having their ideas affirmed.

"By affirming their ideas, we will be more open-minded to what others have to say as well." (p. 172)
77/ "Ask for a temporary agreement to engage in truthseeking. We can ask them if they are just looking to vent or if they are looking for advice.

"Even then, tread lightly: people may say they want advice when what they really want is to be affirmed." (p. 173)
78/ "We are willing to take an irrationally large discount to get a reward now instead of waiting for a bigger reward later.

"Military employees took lump-sum payments at a 40% discount compared to the present value of the annuity payments they would have received." (p. 181)
79/ "Imagine how 'future-us' is likely to feel about the decision or how we might feel today if 'past-us' had made it.

“Every 10-10-10 process starts with a question. . . . [W]hat are the consequences of each of my options in ten minutes? In ten months? In ten years?” (p. 187)
80/ "A flat tire isn’t as awful as it seems in the moment. This kind of time-travel strategy calms down in-the-moment emotions we have about an event.

"Recruiting past-us and future-us engages inhibits the emotional mind, keeping events in more rational perspective." (p. 190)
81/ "We make a long-term stock investment because we want it to appreciate over years or decades. Yet we watch a downward tick over a few minutes, imagining the worst.

"Minute-to-minute and even day-to-day changes have little effect on an investment’s general upward trajectory."
82/ "The recent past drives our emotional response much more than how we are doing overall.

"In relationships, even small disagreements seem big in the midst of the disagreement. In-the-moment emotions affect the quality of the decisions we make in those moments." (p. 195)
83/ "All those things did go wrong. The Normandy landings still succeeded b/c they prepared for as many potential scenarios as possible.

"We shouldn’t plan without advance work on the range of futures that could result and the probabilities of those futures occurring." (p. 207)
84/ "People resist guessing at the probability of future events because they feel like they can’t be certain of what the likelihood of any scenario is. But that’s the point.

"We make a prediction every time we make a decision, so we’re better off making that explicit." (p. 210)
85/ "Anticipating the range of outcomes keeps us from regret (or undeserved euphoria). We are less likely to fall prey to resulting or hindsight bias in which we gloss over the futures that did not occur and behave as if the one that did occur must have been inevitable." (p. 212)
86/ Retirement planning can be done relatively easily using a Monte Carlo simulator.

People who consider the probability of success given what they've saved, how much they plan to spend, and expected returns (low!) may need to change their plans.

Thread:
https://t.co/HObz7IoOVs
87/ "We over-plan for problems we have right now due to the assumption that conditions will remain the same.

"Arbesman’s The Half-Life of Facts makes a book-length case for the hazards of assuming the future is going to be like the present. The world changes too fast." (p. 218)
88/ "We are biased to overestimate the probability of good things happening.

"A premortem is where we imagine *not* achieving our goals. Backcasting and premortems complement each other. Backcasting imagines a positive future; a premortem imagines a negative future." (p. 221)
89/ "Popular wisdom is that we achieve success through positive visualization, but...

"Gabriele Oettingen has conducted over twenty years of research, consistently finding that people who imagine obstacles in the way of their goals are more likely to achieve success." (p. 222)
90/ "We need to have positive goals, but we are more likely to execute if we think about the negative futures.

"A premortem (“What are ways we could fail?”) frees people to identify problems they otherwise might not bring up for fear of being viewed as naysayers." (p. 224)
91/ "Once we recognize the things that can go wrong, we can protect against the bad outcomes, enable nimble responses to a wider range of future developments, and assimilate a negative reaction in advance so we aren’t so surprised by it or reactive to it." (p. 225)
92/ "When we look into the past and see only the thing that happened, it seems to have been inevitable. That’s hindsight bias.

"If we don’t hold all the potential futures in mind before one of them happens, it's almost impossible to evaluate probabilities afterward." (p. 229)
93/ "Life, like poker, is one long game, and there are going to be a lot of losses, even after making the best possible bets.

"We are going to do better, and be happier, if we start by recognizing that we’ll never be sure of the future." (p. 231)
94/ https://t.co/e1BCmKXYe4
95/ Sharpe ratios expressed as probabilities ("bets"):

Ten-year
https://t.co/FQQGy5Lgqs

One-year
https://t.co/Oh5k4Yx64X
96/ "Faced with the choice between changing one’s mind and proving that there is no need to do so, almost everyone gets busy on the proof."

John Kenneth Galbraith
97/ Dilbert: Applying math to guesses

https://t.co/4dK9Bl456f
98/ "Out of the box, we really have no trouble believing things without logic. If someone tells us it’s important to have reasons to support our opinions, we come fully-equipped to mine for them after the fact. Or simply make them up."

https://t.co/yBWtVHvTJ9
99/ https://t.co/L9Q4HlxQAg
100/ https://t.co/LGFnuAwRjb
101/ "NEVER be afraid to ask a question, even if everyone you know thinks the answer is obvious. We’ll never progress without the simple question, 'Why?' "
https://t.co/tAhcFouypp

"Many of a society’s cherished beliefs are castles built on sand."
https://t.co/OMW337rgRR
102/ https://t.co/WNDmRouMdF
103/ "Defensive pessimists come up with strategies to avoid bad situations. They up better-prepared and less anxious.

"In my case, that might mean topping this article with a clever title or pre-writing some 140-character barbs to rout the haters."

https://t.co/W75JgPh40w
104/ "Self-recognition boils down to challenging our received truths... with people who refuse to apply political party to a bond of friendship, who refuse to deny your moral worth because you have “alternative thoughts”. And demand the same in return."

https://t.co/PPrQBA56FY
105/ https://t.co/SQq6uCX4I4
106/ https://t.co/7qXrlF3UuN
107/ https://t.co/thZu1Riaxa
108/ "People were blind to logic when it was embedded in a story. Describe a very sick old man and ask people: Which is more probable, that he will die within a week or within a year? More often than not, they say, “A week.” The story masks the logic."

https://t.co/eeEMw2j2Rg
109/ Related reading:

https://t.co/PVxej6y5AF
110/ https://t.co/NcrNBtJNJJ
111/ https://t.co/HxL7IgUbQo
112/
...
* Beliefs easily become entrenched
* ‘Happiness test’ frees time for the important decisions
* Choosing ‘quittable’ things
* Doing things in parallel (choice diversification)
* Problematic committee dynamics
* Records → accountability
...
https://t.co/F8a3tztwUA
113/ https://t.co/OD64qJSSri
114/ https://t.co/wZvve5iVps
115/ https://t.co/pvqw25b9LU
116/ Superforecasting: The Art and Science of Prediction (Tetlock, Gardner)
https://t.co/Zy6vCla1l5
117/ https://t.co/OZxsd9YEhV
118/ https://t.co/YQpMSsJFQZ
119/ https://t.co/kOt9wmVT2y
120/ https://t.co/rM2Svyu7XW
121/ * Resulting gets in the way of making good decisions
* Fight hindsight bias with “knowledge tracking”
* Hitting the bull’s eye is overrated
* Apply the “only option” test as a way to choose between choices that don’t fundamentally matter

https://t.co/f6CpjJYZHp
122/ Two related books by Phil Tetlock:

Expert Political Judgment
https://t.co/BNEjjaIuR4

Superforecasting: The Art and Science of Prediction
https://t.co/Zy6vCla1l5
123/ In Defense of Troublemakers: The Power of Dissent in Life and Business (Charlan Jeanne Nemeth)
https://t.co/5rQoiCNiqU
124/ "It is possible to commit no mistakes and still lose. That is not a weakness. That is life."

https://t.co/wbdQglvngK
125/ https://t.co/AiPnwiFLCh

More from All

You May Also Like