By Dr. Ken Broda Bahm:
Predicting is tough, and can be even tougher when we don’t believe the limits of our own predictive abilities. Daniel Kahneman, a giant in the psychology world, recently wrote in the New York Times Magazine about an odd task that neatly parallels an attorney’s challenge in case assessment. Dr. Kahneman’s job as a British army psychologist in World War II was to administer a test to young officer candidates and evaluate which should go on to officer training. As the young men worked together to maneuver a large log over a high fence, he and his colleagues would observe and then confidently predict who could best perform as a leader on the battlefield. But the hitch is this: “Despite our certainty,” Kahneman writes, “our forecasts were largely useless.” The even more interesting hitch is how this team of trained psychologists responded: They carried on, continuing to believe in the accuracy of their predictions, despite the evidence to the contrary. “The statistical evidence of our failure should have shaken our confidence in our judgements…It should also have caused us to moderate our predictions, but it did not.” When it comes to trust in our own ability to predict, it turns out that we want to believe.
This post takes a look at the problems that this kind of overconfidence can pose for case assessment, and recommends several ways take greater care when you need to predict.
The Problems With Prediction
By continuing to act as though predictive ability is valid, even when it is shown to be false, Kahneman’s psychologists were just acting out the cognitive fallacy that affects everyone, including attorneys when they handicap their own chances for success at trial. We have written in the past about attorney accuracy in predicting case outcomes, and it isn’t very high. A study comparing attorney predictions with actual case outcomes showed that attorneys on both sides tend to substantially overestimate their chances of success, and that tendency does not get better as the attorney gains more experience. In other words, even when we know that our predictions aren’t reliable — even when we have information that our predictions are flawed — we continue to rely on our predictions and to act as though they’re accurate.
Why the persistence? We might simply be trusting our ability to predict because that is all we have to go on. That is, we prop up our confidence, largely independent of the evidence, because we are highly motivated to believe in the value of our own choices and actions. As Dr. Kahneman puts it, “Confidence is a feeling, one determined mostly by the coherence of the story and by the ease with which it comes to mind, even when the evidence for the story is sparse and unreliable.” The New York Times Magazine article, which is well worth the read, considers a number of studies and stories that all point in the same direction: our confidence is a coping skill, not a reaction to hard evidence.
That is an important lesson for litigators. The long walk up to trial is nothing but a series of judgments and choices based on likely future outcomes: Will continued discovery favor us or them? Will our witnesses be believable? Will the judge let us out on summary judgment? Will we win? The decision to settle or to press on is driven by our estimates of what the future holds. So if those predictions are tainted by overconfidence, then what is a careful litigator to do?
The Trick to Predicting Better
Avoiding prediction isn’t the answer, because for better or worse, the case is moving forward and decisions need to be made. When we don’t rely on the best predictions available, cases tend to settle inappropriately, based solely on the known (the costs of litigation) and not on the unknown (the real chances of success or failure).
Responsible litigators and in-house counsel need to make decisions based on the future prospects of a case. In order to guard against the hazards of over confidence, however, let me make three recommendations:
1. Don’t Over-Trust Your Gut Instinct. With the widespread celebrity of Malcolm Gladwell’s “Blink: The Power of Thinking Without Thinking,” the idea of rapid cognition, relying on quick judgment and instinct rather than careful cognition has become very fashionable. In some ways, the work that Daniel Kahneman is drawing from, a book released last month entitled “Thinking, Fast and Slow” is an answer to that. Our “fast thinking,” or blink reactions, can be surprisingly effective, but are also notoriously prone to bias and error. It is inevitable that any experienced litigator will have a gut feeling – about a case, a witness, a juror, or a ruling. It is important to recognize, but limit, the role of that instinct, and to remember that it is a feeling, susceptible to our fears and our wishes, and not a product of evidence.
2. Don’t Over-Trust Your Counselors. It is also normal in litigation that a great many people — from co-counsel, opposing counsel, judges, mediators, secretaries, paralegals, and litigation consultants like myself — will give you opinions about your case’s future chances. All may provide good feedback but none should be trusted absolutely, and I include myself in that statement. As Kahneman notes, overconfident individuals tend to look, act, and sound like experts, but that doesn’t mean they are correct: “You should not take assertive and confident people at their own evaluation unless you have independent reason to believe that they know what they are talking about.”
3. Rely on a Multipoint Review. Ultimately, the closest thing to a magic bullet for case assessment can be found in what the social scientists call “multitrait, multimethod validation,” or considering as many aspects and as many sources of information as possible. That means that you should check with all of those sources your trial budget supports, including sounding boards from colleagues, friends, and family, but also including the more neutral feedback from consultants and mock jurors. Specifically, new pretrial research designs that aim more at case assessment than trial preparation can be very effective in identifying a potential range of verdict outcomes that can inform trial or settlement decisions.
We will be writing more in the future on methods for research-based early case assessment, but the main takeaway is this: Don’t mistake confidence for knowledge. In order to be a good advocate, litigators need to believe in their cases. But the very tendency that makes for a great attorney can make for a poor predictor. In addition to acknowledging that, your best protection is to make sure that you are getting as much feedback as you can from those outside your war room.
______
Related Posts:
- Be More Realistic Than Your Opponent
- Diagnose Your MedMal Case
- Get the Gist of How Jurors Decide Damage Numbers
______
Daniel Kahneman (2011). Thinking, Fast and Slow Farrar, Straus & Giroux
Photo Credit: Outdated productions, Flickr Creative Commons