Stereotype threat in applied settings re-examined
This paper offers a re-analysis of data collected and presented by Stricker & Ward (2004), in which the claim was made that there were no "effects of inquiring about ethnicity and gender on performance on [standardized] tests" (p. 685). The current authors critique the use of conservative decision criteria used in the original analyses. Specifically, Stricker and Ward (2004) only considered results to be significant if p<.05 in the overall ANOVA, p<.05 for planned comparisons using a very conservative post hoc Bonferroni correction, and effect sizes were >.10 and d>.20. While these criteria do minimize the chances of making a Type I error (erroneously claiming a significant effect), they also elevate the likelihood of making a Type II error (failing to detect a real, even if small, difference). Given the importance of standardized test performance in determining educational opportunities, career paths, and life choices, Danaher and Crandall argue that use of standard statistical decision criteria is misplaced in this context. Accordingly, these authors re-examined the data presented by Stricker and Ward (2004), but used criteria of p<.05 from the overall ANOVA and .05 for determining significance. These small changes produced several conclusions differing with those offered by Stricker and Ward (2004). Timing of soliciting identity information interacted with gender in affecting AP Calculus AB Grades and Formula Scores. Womens performance on these measures improved and men's performance declined when when they were not asked about gender before taking the test. Put in different terms, soliciting identity information at the end rather than at the beginning of the test-taking session shrunk sex differences in performance by 33%. Analysis of performance on the Computerized Placement Test also yielded an identity timing x gender interaction, with women's performance improving substantially and men's scores declining slightly, when they did not report their identities before taking the test. This re-analysis suggests that soliciting social identity information prior to test taking does produce small differences in performance consistent with previous findings in the stereotype threat literature that, when generalized to the population of test-takers, can produce profound differences in outcomes for members of different groups.