Justifying the Unjustifiable

Several weeks ago I met with Columbia University Marketing Professor Leonard Lee. Over a fantastic pistachio milkshake at Tom’s Café (of Seinfeld fame), we got on the topic of moral decision-making research. Leonard suggested that I read an older paper by Chicago Professor Christopher Hsee. The paper, “Elastic justification: How unjustifiable factors influence judgments,” describes how unjustifiable information can subconsciously creep into the decision-making process. To summarize, Hsee breaks information into two primary categories:

• Justifiable: Information that is directly relevant to the judgment being made by the accepted criteria of the decision-maker (e.g. a candidate’s experience)
  
• Unjustifiable: Information the decision-maker may wish to take into consideration but that he knows, by the criteria of what is acceptable, should not come to bear on the decision. (e.g. a candidate’s race or sex)

Hsee’s studies show that decision-makers do not include unjustifiable information in their decisions except under certain conditions relating to the JUSTIFIABLE information. The degree of "elasticity" in how to weigh the justifiable factors determines how much influence the unjustifiable information has on the judgment. If the importance weighting assigned to various justifiable attributes is clearly defined and fixed (inelastic), unjustifiable information does not influence the decision. However, if the weighting is vague and open to interpretation (elastic), unjustifiable information will influence the decision by reweighting the importance of the justifiable factors. Since past literature shows that people try to make justified decisions and line up consistent facts/evidence for those decisions, Hsee concludes that people do not knowingly incorporate unjustifiable information. The process is subconscious.

The paper’s findings stand out because they are contrary to normative decision theory which would not recognize a split between justifiable and unjustifiable information. Instead, the normative approach takes in all information, assigns each factor a relevant weight according to its probable utility impact, and concludes with the utility maximizing decision (see for example Jon Baron’s work on Multi-Attribute Utility Theory). In contrast, Hsee’s work shows that decision-makers exclude unjustifiable information in certain cases. Reweighting utility impact for a given factor is also non-normative.

In the paper Hsee sites three historical studies as evidence as well as two of his own. It is the second of Hsee’s studies with which I take some issue. In this second study subjects were given a “language intuition test” by which they were asked to select from a multiple choice list the meaning of unfamiliar Chinese characters. 20 questions were asked of each subject, only 10 of which counted towards a total score. Subject later self reported their score with a financial incentive for higher scores. Hsee suggests that two factors would influence the self reported score: 1) actual performance and 2) desire to report a high score for the financial incentive. He then asserts that the former is a justifiable factor and the later is unjustifiable and also predicts that the degree of elasticity in the grading process will determine the amount the unjustifiable factor will influence scores upward. In the inelastic condition subjects were asked to score only odd numbered questions. In the elastic condition, scores were based on the subject’s evaluation of the “ying/yang nature of the symbols” – a highly vague and subjective approach. Subjects were to count the 10 questions whose symbols they found to be the most “yang” in nature. As anticipated, subjects in the elastic condition reported higher overall scores.

Unfortunately the study design does not seem well suited as a test of Hsee’s theory. I do not believe it is possible to determine whether or not subjects are indirectly incorporating unjustifiable information by reweighting the justifiable or merely directly incorporating their “unjustifiable” desire for a high score. In fact, in the elastic condition there is not a justifiable method of selection provided to subjects at all. How does one justifiably assess the “yangness” of a character? Faced with a process such as this one it is no surprise that subjects did not feel unjustified in selecting questions that they happened to get right as yang characters. The the yangness approach may break a social contract with the subjects, allowing them to have free reign in “cheating” on their scoring. One might ask what the selection process of students who did not wish to cheat would have looked like. The study would be better formed if it provided a subjective yet seemingly justifiable procedure for selection.

I do not wish to discard the entire paper based on Study #2. Hsee has a very interesting theory with significant merit conferred by the other studies. There are also some very interesting additional questions the follow from the study. Further research seem warranted to answer the following:

• Can we evaluate the specific weighting assigned by subjects to justifiable factors? For example in Hsee’s study #1 involving real estate appraisal, what relative weights do subjects place on living room size, appliances, carpets, etc?


• Can we demonstrate that the weight assigned to a specific factor changed? For instance can we demonstrate that subjects specifically reweighted the value of good appliances relative to carpets as a means to indirectly incorporate unjustifiable factors? Perhaps this can be solicited by asking subjects to assign numeric weightings or at least by rank ordering the importance of the factors.


• How might we prove/disprove that this process takes place subconsciously? Perhaps subjects instead make conscious decisions and then use a purposeful forgetting/self-deception mechanism to remain justified in their own eyes – (more forthcoming on a "breadcrumbs of self-deception" idea). There would seem to be significant moral implications if this is a conscious process.