N-Effect Review: Part Two

In this second installment of a three part series, I explore alternatives to Stephen M. Garcia and Avishalom Tor’s “N-Effect” theory and suggest several additional studies that would help solidify the research findings.

Alternative Explanations

Regression: One starting point for an alternative explanation to the authors’ findings on N and motivation can be found in Study 5. As stated in the Results and Discussion section, “Participants felt it would be easier to win the cash prize in the 10 competitors condition than in the 10,000 competitors condition.” Putting aside the implications this result has on a ratio bias explanation, let us just grant that for whatever reason subjects believe it is easier to win competitions with fewer competitors. If this is true, objective social comparison around N could theoretically be completely removed from the explanation of competitive motivation and replaced by subject confidence in performing easy or difficult tasks. Past research (e.g. Moore & Cain, 2005) suggests that for easy tasks subjects believe they will be above average in their performance of the task and on difficult tasks they will perform below average. Since subjects have better access to their own abilities “their beliefs about others’ performances tend to be regressive and less extreme than their beliefs about their own performances.” This Overconfidence and Underconfidence effect has been demonstrated at low and high values for N.

However, similar to the authors own observations in regard to ease or difficulty of achieving a payoff, high confidence could either decrease or increase motivation (“I’m going to win anyway so I do not need to work hard” or “I am confident I can win so I’m going to work very hard as I know the effort will pay off”). A clear conclusion cannot be determined using only the results of the N-Effect paper. We can assume that subjects in Study 2 would judge that task to be easy. Collectively the task in Study 5 has a mean difficulty assessment that isn’t close to either extreme. Finally, we have no way of knowing how subjects feel about the foot race and job interview imagined tasks of Studies 3 and 4. If we had a set of both inherently difficult and inherently easy tasks we could compare the low N and high N motivation rankings between tasks. If high confidence yields high motivation, we should find the motivation to complete in low N versions of easy tasks to be higher than in low N versions of difficult tasks. Factored into the analysis it may be that task difficulty plays a mediating role in competitive motivation instead of social comparison.

Numeracy Failings: The study 4 control for subject ratio biases combined with the study 5 finding that subjects judged small N tasks as easier than large N tasks constitutes persuasive evidence against a classic ratio bias explanation for the N-Effect. However, this finding does not preclude other failures in subject numeracy from playing a role. For instance, subjects might be influence by formatting effects related to frequency (e.g. Gigerenzer, 1994; Gigerenzer & Hoffrage, 1995). This theory suggests that subjects are better able to interpret information expressed as a frequency (2 out of 10 or 20 out of 100) better than when the same information is expressed as a probability (20% chance of winning). N-Effect Studies 2, 3, 4, and 5 all use percentage probabilities instead of frequencies when presenting information to subjects. This formatting effect could cause subjects to misinterpret their probability of winning in each scenario, perhaps assuming they somehow have a better chance of winning when there are fewer competitors. Again, social comparison would not be necessary to explain competitive motivation. Note that Study 5 describes some “manipulation checks about N and the percentage of competitors that would win.” From the context it can be assumed that the checks were also formatted as percentages; however, these checks are not described in detail in the paper.

Winner-Take-All Heuristic: Many competitions only reward the very top performing competitor or at least have special rewards for the top competitor. This situation could lead to a heuristic for subjects that competitions with more competitors are harder to win -- even when faced with information that the competitions have an equal probability of positive outcome. Instead of translating a 20% chance of winning with 50 or 500 competitors into 10/50 or 100/500 odds respectively, subjects may instead have an internal representation that is tugged toward an extreme of 1/50 and 1/500. This incorrect assumption on ease of winning could influence competitive motivation without social comparison playing a role as discussed previously.

Additional N-Effect Studies

Multiple additional studies could be run to more clearly demonstrate the N-Effect and rule out alternative explanations.

N-Effect Continuum: The authors of the N-Effect admit that the effect arises only within a certain territory of N values but it is unclear where this territory lies. Starting from no competitors and adding one at a time, at what point do social facilitation improvements in motivation give way to N-Effect reductions in motivation due to reduced social comparison? Is the continuum different for various competitive arenas like poker, running, debating, etc.?

Direct Measures of Motivation: As mentioned in the previous posting, Studies 3, 4, and 5 would be much stronger if the authors had a more direct way of measuring competitive motivation such as effort invested in actually performing a task or perhaps willingness to make actual payments to eliminate other competitors from consideration.

Actual N: Further studies involving actual task performance in the presence of small and large N competitors groups should be run to prove the N effect is not limited to imaged N sets.

Ratio Bias on Stimulus: Study 4 looks at subjects’ general susceptibility to the ratio bias but it neglects to question subjects directly to see if they exhibit a ratio bias on the scenario stimulus itself. Subjects could be asked to choose which competition they would rather compete in [10, 30, 50, 100, or indifferent for Study 4] as a more direct test for the bias.

SCO Expansion: Rerun Studies 2, 4, and 5 including the SCO measure to make sure the same confirming results are found as in Study 3.

Easy v. Hard Tasks: As mentioned in the “Regression” alternative explanation above, a study designed to compare N level competitive motivation assessments for easy and hard tasks could be quite informative. If subjects rate low (high) N condition competitive motivation in easy tasks very similarly to low (high) N condition competitive motivation in difficult tasks, the N-Effect theory would be strengthened.

Format Effect Control: Rerun the studies using frequency numbers instead of percentages to describe the winning outcomes and thereby control for a formatting explanation.

Winner-Take-All Control: Remind subjects of several instances where winner take all is not the rule of competition and where all subjects who win are treated equally (passing the legal board exam for example) as an attempt to reduce any winner take all bias prior to rerunning N-Effect studies.

Next Installment: Conclusions and Future Directions

Gigerenzar, G. (1994). Why the distinction between single-event probabilities and frequencies is important for psychology (and vice versa). In G. Wright & P. Ayton (Eds.), Subjective probability (pp. 129-161). New York: Wiley --- NOTE reference found in Reyna and Brainerd’s “Numeracy, ratio bias, and denominator neglect in judgments of risk and probability,” Learning and Individual Differences, March 2007

Gigerenzer, G., & Hoffrage, U. (1995). How to improve Bayesian reasoning without instruction: Frequency formats. Psychological Review, 102, 684-704. --- NOTE reference found in Reyna and Brainerd’s “Numeracy, ratio bias, and denominator neglect in judgments of risk and probability,” Learning and Individual Differences, March 2007

Moore, D. A. & Cain, D. M. (2007). Overconfidence and underconfidence: When and why people underestimate (and overestimate) the competition. Organizational Behavior and Human
Decision Processes, 103, 197-213.

Scapegoat the Visual

I have a meeting with Harvard Professor Max Bazerman today. As most readers will know, Professor Bazerman is world renowned for his research on negotiations and judgment in managerial decision making. He also studies bounded awareness and ethicality, societal decision making, and want/should conflicts. Along with collaborators Don Moore, Francesca Gino, and Lisa Shu, Professor Bazerman’s recent research dives into several aspects of “Moral Luck” discussed previously. In preparation for the meeting I created the following visual to better explain some of my thinking on the topic. I am happy to share it here on http://www.prospecttheory.net/.

Moral Luck

In philosophy, the term moral luck is used to describe the morality of an action judged in the light of its uncontrollable and unforeseeable consequences instead of in isolation. Such judgments are not normative. The morality of an action should not be different because of a lucky good outcome or an unfortunate bad end. Questions of moral luck are not new to philosophy but they seemed like fertile ground for experimental behavioral research. In fact I have been working on a rough experimental design to demonstrate the moral luck effect in collaboration with Columbia’s Leonard Lee.

Here is the draft set up:

--------------------------------------------------------

A doctor is visited by a patient complaining of a stomach ache and other vague symptoms. The doctor has a “gut feeling” that the patient may be suffering from Disease X. Disease X is a serious condition and, left untreated, it can reduce the expected lifespan of a sufferer by up to 5 years. The majority of medical experts estimate there is only a 1 in 10,000 chance that a randomly selected person in the population will have Disease X. None of the symptoms of which the patient is complaining are associated with Disease X and two other doctors have already examined the patient and ruled out the disease. These other two doctors believe the patient has a mild form of a flu virus that should resolve itself in a few days.

There is a test for Disease X that is 100% accurate in its diagnosis. Diagnosed early the disease can be cheaply treated with outstanding success; however, the test costs $5,000 to administer and in 2% of cases the test itself results in a serious infection, which also has negative effects on expected lifespan.

The doctor decided to run the test based on his own judgment. [GOOD OUTCOME: The lab results from the test show that the patient does have Disease X which can now be cheaply and effectively treated. The patient may or may not have an infection resulting from the test (2% chance of infection). BAD OUTCOME: The lab results from the test show that the patient does not have Disease X. The patient may or may not have an infection resulting from the test (2% chance of infection).] On a scale of 1 to 5, how moral was the doctor’s decision to run the test?

Very Immoral (1 to 7 scale) Highly Moral


Should an experienced doctor be allowed to make such a decision even if the statistical odds are not in favor of his or her decision?

Yes / No


If you believe the patient’s family is justified in suing the doctor for medical malpractice, what is a reasonable dollar amount that the doctor’s insurance should be expected to pay in compensation? The average malpractice payment at the doctor’s hospital is $50,000.

$__________________


--------------------------------------------------------

There are a number of improvements that should be made to this set up before running it with real subjects but that may not be necessary. In researching the project I ran across two new papers on the subject that already provide quite solid evidence of a moral luck like effect.

Francesca Gino, Don A. Moore, and Max H. Bazerman, “No harm, no foul: The outcome bias in ethical judgments,” HBS Working Paper Number: 08-080, February 2008

Gino, Francesca, Lisa Lixin Shu, and Max H. Bazerman. "Nameless + Harmless = Blameless: When Seemingly Irrelevant Factors Influence Judgment of (Un)ethical Behavior." Harvard Business School Working Paper, No. 09-020, August 2008.

So it looks like we do not get to be the experimental moral luck vanguard. On the plus side I was at least lucky enough to have a wonderful coffee conversation with one of the authors yesterday, Lisa Shu. I look forward to reading more of her work.