Insurance Arbitration: Debiasing the Biased

By Richard G. Waterman, CPCU

Everyone possesses certain biases and prejudices. These biases are usually not attributable to motivational effects such as wishful thinking or the distortion of judgments by rewards and penalties. Instead, leading research in decision making suggests that in many contexts decision makers employ intuitive, common-sense judgments to guide the simplification of complex information processing that must be integrated to yield a decision. For many purposes these simplifying methods, called heuristics, result in reasonable judgments, even though these intuitive judgments are often biased in a predictable manner. It is not surprising that the reliance on useful judgmental heuristics and the prevalence of biases has been found to influence people who are encouraged to be accurate and are rewarded for correct answers.

In recent years, a large body of research has been devoted to uncovering judgmental heuristics and exploring their effects.1 We know, for instance, that people gain considerable life experience and practical knowledge. We also know that people use intuitive heuristics, or heuristic methods, in problem solving because such techniques have worked well in their life experience. Problem solving requires the use of knowledge that is already acquired. Heuristic methods allow us to search our memories for possibilities and evidence that are already there. Decision making in an arbitration setting is a form of problem solving. Arbitration, similar to court adjudication, is a process to decide between disputed alternatives under conditions of uncertainty. Arbitrators seek to interpret facts and search for the truth. Arbitration decision making involves a large component of judgment, intuition, and educated guesswork. Abundant evidence from psychological research suggests that in many contexts decision makers’ intuitive, common-sense judgments can be faulty and may be held with great confidence. Indeed, the opinions, intuitive judgments, and misunderstanding can often be traced to biased experiences. Even well-informed people have difficulty judging accurately if they are not aware of factors that might bias their assessment of evidence and final judgment.

People selected to serve on arbitration panels have gained considerable life experience and practical knowledge in their business careers. Experienced people have insight, but often at the cost of heuristic biases and prejudices. Arbitrators are not expected to suddenly forget all that lifetime of experiences; they are, however, asked to judge fairly. The challenge for arbitrators is to acknowledge their experience limitations, learn to recognize the existence of potential bias or prejudice, and learn techniques to control those influences.

Arbitrators must exercise their best judgment to decide cases on their own merits. Becoming better informed and more open to new evidence, and being aware of how heuristic biases influence decision making is not merely desirable, it is essential to provide equitable dispute decision making. Two common heuristic biases prevalent in arbitration decision making are confirmation bias and overconfidence bias.

Seek and Ye Shall Find

Studies have shown that background information provided to expert fact finders prior to an evaluation of evidence has a whopping influence on their subsequent interpretation and analysis of the evidence. The heuristic effect is known as confirmation bias whereby greater weight is put on evidence that supports the background story in the process of integrating all the available evidence. Confirmation bias is defined as an inclination to retain, or a disinclination to abandon, a currently favored hypothesis.

We like to think of ourselves as rational observers of the world, weighing the facts and reaching well-reasoned decisions. Nonetheless, we all have our beliefs and we latch on to nuggets of information that support them while shrugging off dissenting views and conflicting evidence. We like to think that we are immune to confirmation bias. Studies of decision making have shown, however, that the confirmation bias is so pervasive that it affects people without their realizing it. One reason we think we are relatively immune to the risk of confirmation bias is that we exaggerate the extent to which we can control the tendency that fosters poor decision making. We are erroneously confident in our experiential knowledge and underestimate the probably that our information retrieval from memory or our beliefs will be proved wrong. That is why we tend to seek additional information in ways that confirm what we already believe.

For one example, in a confirmation bias experiment conducted with professional polygraphers, three polygraphers identified as A, B, and C were enlisted to help in discovering which of three suspected employees, identified as X, Y, and Z, were stealing from a commercial business in a mock scenario. A was told that X was suspected, B was told that Y was suspected, and C was told the Z was suspected. The three polygraphers, who were unaware of each other’s existence, conducted a standard investigation of the three suspects and provided their conclusions. A found X to be guilty. B put the blame on Y, and C determined that Z was the culprit. The experiment confirmed that a polygraph investigation, like many expert information gathering activities conducted in good faith, may yield biased results. Even expert fact finders have a tendency to see in their analysis what they hypothesized to be there.

In a similar fashion, a threat to objective information processing in tripartite arbitration decision making is the tendency of party appointed panel members to focus on particular interpretations of the facts. Before evidence is presented at a hearing, party appointed arbitrators are already aware of the disputed situation and assessment favored by the party that appointed them. Party arbitrators must learn to withhold judgment while integratinterpretation of the evidence. To minimize the negative impact of reaching a decision without a critical evaluation of alternative viewpoints, an arbitrator should attempt to make a fresh and independent analysis of the evidence. Repeated research has shown, however, that such prior knowledge can jeopardize a decision maker’s judgmental analysis. The results show that when given a prior interpretation, both experienced and inexperienced fact finders considered the favored assessment more likely. Even a temporary commitment to the favored interpretation of the facts increased confidence in the favored theory and ultimate judgment. Studies on the confirmation bias also show that people with prior knowledge tend to seek confirmation evidence and select information that is in line with a current belief.

For example, crime analysts are often called in by law enforcement investigation teams to provide an independent interpretation of the evidence at hand. The analyst’s main job is to counteract potential tunnel vision or groupthink. A well-known tendency of investigation teams is to converge on a single shared interpretation of facts without a critical examination of the underlying assumptions. In a study experiment, independent crime analysts who knew about the favored interpretation of the facts before examining the case arrived at the same interpretation much more often than crime analysts who did not know the team’s opinion in advance. This bias was observed in the interpretation of very complex cases as well as simpler cases. The study revealed that such prior knowledge may seriously compromise the independence of the analysis.

We also need to think about how we process information. For instance, many commentators from various media outlets have repeatedly reported that Barack Obama’s press conferences, speeches, and interviews were packed with the word “I.” Some people think that people who are self-confident use I-words all the time while others believe that frequent use of the I-word is proof that a person is arrogant. Contrary to the belief of highly respected news outlets, a recent analysis revealed that Barack Obama has distinguished himself as the lowest I-word user of any of the modern presidents.2 No one in the media outlets bothered to actually count his use of I-words and compare the results with the I-word usage of other presidents. Many of the commentators who observed Barack Obama’s frequent use of I-words are people who do not share his political views. The study demonstrated that if we think that someone uses the Iword frequently, we have a tendency to hear and search for evidence that confirms our beliefs.

A technique to reduce the likelihood of confirmation bias is called hypothesis testing, which is looking for features or evidence not expected to be present if the hypothesis is true. If, for example, in the process of examining evidence a fact finder is aware of confirmation bias influences and makes sufficient allowances for the presence of evidence that does not support a hypothesis, the effect of prior expectations may be diminished. Another technique to mitigate confirmation bias is the consideration of alternative or competing hypotheses. By thinking of specific alternative hypotheses, we may get a better idea of the probability of another outcome if the evidence does not support the favored hypothesis. It has been observed, however, that the persistent illusion of the validity of a hunch or initial hypothesis can be enhanced by virtue of being the hypothesis tested. Nonetheless, research findings indicate that there is reason to believe that an awareness of confirmation bias and training to learn to facilitate consideration of alternative hypotheses may reduce confirmation- based judgment.

Overconfidence Heuristic Bias

Overconfidence bias refers to a situation in which people are surprised more often than they expect to be. For instance, when teachers ask a class who will finish in the top half, on average around 80 percent of the class think they will. Effectively people are generally much too sure about what they know. This tendency is particularly pronounced amongst experienced people in specialized fields. Experts, for instance, are far more overconfident than lay people. This is consistent with the illusion of knowledge driving overconfidence. It is part of human nature to expect that a long track record of past performance in a field will lead to a sense of overconfidence, sometimes inappropriately. Once an experienced decision maker has the minimum information necessary to make an informed judgment, attaining additional information generally does not improve the accuracy of that judgment. Additional information may actually lead to becoming more confident in the original judgment to the point of overconfidence. Even when respondents in various judgmental experiments were cautioned to be aware of the overconfidence bias, overconfidence was only partially reduced.

Without specific prompting of respondents to consider reasons why they might be wrong, people seem to be insufficiently critical of their thinking or even intent on justifying their initial assessment. Research has demonstrated that a major cause for the phenomenon of overconfidence when confidence is high is a tendency to seek evidence in favor of an initial belief, as opposed to evidence against it. The overconfidence bias can be reduced but not completely eliminated by considering reasons in favor of something we would like to be true and also generate reasons why the favored judgment might be wrong. Inappropriate high confidence is the failure to think of reasons that possibly support a contrary position. Such inappropriate confidence could cause an arbitrator to stop searching for alternative possibilities, leading to insufficient thinking and incorrect decisions.

The ability to consider and evaluate our own thinking or to demonstrate what evaluations provide accurate or useful information to support our level of confidence is crucial to fair decision making. Also, for a better basic understanding of our decision making process we need to consider possible reasons for error or uncertainty and how well we assess the possibility that a high level of confidence is not always a reliable indicator of accuracy. Careless generalizations of findings must certainly be avoided. Studies of how well confidence matches accuracy, referred to as confidence calibration, show that certainty about decisions can became entirely out of proportion to accurate decisions. Therefore, errors in confidence calibration often can have negative consequences. For instance, unwarranted high levels of confidence in eyewitness testimony have been shown to influence juries to convict innocent people, particularly if highly confident assertions are assumed to be accurate. Most of the convictions overturned by DNA evidence were based on erroneous eye witness testimony. The overconfidence heuristic can also affect the level of confidence in a group decision such as arbitration deliberations if it is assumed that a more confident member of the group is more knowledgeable.

An important step in reducing arbitrators’ overconfidence based on inaccurate inferential judgments would be taken by learning to recognize that interpretations of information and evidence, rather than being simple readouts of historical events, may include inferences that make heavy use of theory. Once the heuristic tendency is recognized, the same information may look quite different and could easily support different beliefs if the information were viewed from the vantage point of alternative theories. Such recognition would invite arbitrators to consider logical alternatives before adopting a theory about a case and adjusting their confidence accordingly. When the theory is wellfounded, its heavy use in the interpretation of ambiguous events is correspondingly justified. But when the origin of the theory is dubious, interpretation of events must be severely circumscribed.

Are experienced arbitrators who have decided many cases less sensitive to the effects of judgmental biases than less experienced arbitrators? Experienced arbitrators are thought to be able to integrate prior knowledge and their own interpretation of the evidence before making a final assessment. Only a few studies have investigated whether the decision making of experienced fact finders is less affected by heuristic influences than their less experienced colleagues. Given the mixed results, it is not clear whether experienced arbitrators are less likely to be susceptible to the confirmation bias or if they become unjustifiably overconfident. Since experienced arbitrators have encountered more situations in which their independent analysis diverged from that of a prior interpretation, they may be more aware of the practical importance of independent analysis. Expertise may lead to a somewhat more critical attitude towards other people’s beliefs and interpretations. Furthermore, experienced arbitrators have seen many more cases that may cause them to question the possibility of alternative explanations. Generally, when alternative possibilities and explanations are acknowledged, further investigation and independent assessment would likely reduce the influence of judgmental biases.

Debiasing Techniques

Most arbitration decision making, like many other areas of complex activity, is done under conditions of uncertainty. Consequently, arbitrators are encouraged to focus their attention on the relevant and objective facts and thereby reduce any possible overutilization of inappropriate heuristic tendencies. But the extent to which every arbitrator is subject to unconscious biases and utilizes heuristic systems is not known. The research of human judgment shows, however, that people are influenced by judgmental biases. Therefore, the question about fairness in making arbitration decisions is not whether arbitrators are influenced by intuitive biases, but rather how judgmental heuristics affect the assumption of fairness and whether an awareness of heuristic principles improves arbitration decision making.

There is no proven antidote to heuristic biases, and it is not clear whether biases can be unlearned. Some partial remedies include an understanding of how to avoid them when processing incomplete information and second-guessing post-events. Other partial remedies are devoted to training to understand our decision-making process. Another aspect is developing an awareness of our own feelings of confidence in present knowledge that controls our pursuit of new information and the interpretation of past events. Essentially, debiasing techniques facilitate learning. Optimistically, debiasing methods can teach arbitrators to avoid the biases that arise from being captives of their personal perspectives.

Arbitration decision making is different from other types of decision making in several ways. Arbitrators range from highly qualified industry professionals who serve frequently on arbitration panels, making repeated decisions, to less experienced arbitrators making fewer decisions. Moreover, service on arbitrations panels does not provide an opportunity to learn how arbitrators make decisions, and pertinent information may not be available to the arbitration decision makers to receive much feedback about the quality of their decision. Arbitrators rarely suffer any consequences for making poor decisions. In spite of good intentions, arbitrators may not know how judgmental heuristics, including confirmation bias and overconfidence bias, influenced the accuracy of their decisions.

Educational programs providing concrete examples of intuitive errors and teaching judgmental maxims would offer many clues that might help arbitrators to identify and understand the effects of judgmental heuristics. A better understanding of informal heuristics should prompt caution about extreme judgments and predictions that may be wrong and result in poor decision making. An appreciation of heuristic principles should also serve to make arbitrators stop and reconsider the theory underlying an assessment to determine whether the interpretation is free of bias and is sufficiently well-grounded. Even if biases cannot be unlearned and survive considerable contradictory evidence, we can learn to recognize their existence and take the necessary precautions. Actively open-minded thinking can help avoid heuristic bias. Consider carefully the goals of seeking information that will be useful in the decision making process with other goals, such as the goal of seeking information in general or seeking information that supports a preconceived favored result. In addition, try to think of alternative probable outcomes. If we fail to consider alternatives, we may end up learning afterwards that a favored interpretation of facts was wrong.

Thinking about alternative outcomes may also be useful when we do not have a favored probability of a result. In formulating alternative outcomes, we must think about what we already know. The better we know and understand the facts, the better we can think about them. We must make sure that alternative outcomes are consistent with the evidence already available before we attempt to collect new information. When serving as arbitrators, we have an obligation to judge cases fairly to best of our ability based on the facts presented. In carrying out our obligation, we need to recognize that we might have some prior knowledge about the preferred interpretation of the facts and we may have a preconceived bias or prejudice. If we recognize the possibility that bias exists and determine techniques to set aside those biases and prejudices, we will be better able to judge the case fairly and justly.

Concluding Comments

Judgment and reasoning is a tendency for our expectations, preconceptions, and prior beliefs to influence our interpretation of new information. When examining evidence relevant to a given belief, we are inclined to see what we expect to see and conclude what we expect to conclude. Information that is consistent with our preexisting beliefs is often accepted at face value, whereas evidence that contradicts them is critically scrutinized and discounted. Our beliefs may thus be less responsive than they should be to the implications of new information.

Experienced people who serve on arbitration panels rely on their life experiences to critically examine the facts and to assist them in judging the credibility of witnesses and assigning weight to testimony. The debiasing techniques presented here are based on general notions about heuristic principles. One of the best ways to internally establish beliefs and filter heuristic biases is to think through the decision you have reached and the basis for that decision. When serving on an arbitration panel, be challenged to understand the strengths and weakness of the other side, work through an analysis of reasoning and opinions that disagrees with your judgment, and be forced to constantly reevaluate your initial positions. Then decide the case justly with great confidence to the best of your ability.