How Stroop scores an IAT

Stroop scores every IAT, the test Greenwald, McGhee and Schwartz introduced in 1998 [1], with the improved algorithm that Greenwald, Nosek and Banaji published in 2003 [2]. A positive D means the participant was faster when the reference pairing shared a key; which pairing is the reference is your choice when you build the study, and it never depends on the order a participant saw the blocks in.

The procedure

1. Take the trials from blocks 3, 4, 6 and 7. Blocks 1, 2 and 5 are never scored.
2. Delete trials with a latency over 10,000 ms.
3. If more than 10% of the remaining trials are under 300 ms, the participant gets no D-score and is flagged as a fast responder.
4. For D2, D5 and D6 only, delete trials under 400 ms.
5. Treat errors: D1 and D2 keep the recorded latency to the correct key, which already carries the penalty; D3 and D5 replace an error with the block's correct mean plus two SDs; D4 and D6 with the block's correct mean plus 600 ms.
6. Work out the inclusive SD of all practice latencies from both pairings, and the inclusive SD of all test latencies from both pairings, using the sample SD.
7. Work out the mean latency of each of the four blocks.
8. D practice is the second pairing's practice mean minus the first pairing's, divided by the practice SD. D test is the same for the test blocks.
9. D is the average of D practice and D test. If either SD is zero or a block is empty, there is no D and the participant is flagged as insufficient data.

The six variants

The default is D1 with correction required: a wrong key shows a red X and the participant has to press the right one, so the latency already includes the time it took to fix the mistake. Greenwald and colleagues recommend that design [2]. You can choose any of the six, and the export carries all of them for every participant.

A worked example

Sample respondent R-0137 from the brand IAT on the implicit testing page, Larkspur Bank against Harbourline, scored with D1. The reference pairing (Larkspur with trustworthy) came first, in blocks 3 and 4.

No trial was over 10,000 ms and 0% were under 300 ms, so nobody is deleted or excluded. The practice SD across blocks 3 and 6 is 176.3 ms; the test SD across blocks 4 and 7 is 225.6 ms.

• D practice = (924.4 ms minus 843.7 ms) divided by 176.3 ms = 0.46
• D test = (1,001.7 ms minus 870.9 ms) divided by 225.6 ms = 0.58
• D = (0.46 plus 0.58) divided by 2 = 0.52

Stroop reports that as a moderate association in the reference direction: faster to pair Larkspur Bank with trustworthy. The bands (under 0.15 little or none, 0.15 slight, 0.35 moderate, 0.65 strong) are conventions used by common IAT software, not validated cut-offs, and the results page says so beside them.

Exclusions

The fast-responder rule in step 3 is part of the algorithm, so it always applies. Everything else is a switch on the results page, and each switch shows how many participants it removes before you turn it on: a Poor timing grade, more than 10% of trials interrupted, an error rate above 30%, leaving full screen more than twice, failed attention checks, or a different input (keys or touch) from the study's. Exclusions filter the analysis; they never delete data.

The brief IAT

Brief IATs follow the procedure Nosek and colleagues recommend in 2014 [3]. Delete trials over 10,000 ms, drop the first four trials of each response block, keep errors with their latency to the correct key, and recode latencies under 400 ms to 400 and over 2,000 ms to 2,000. More than 10% of trials under 300 ms excludes the participant. D is computed for each consecutive pair of response blocks (the mean with target B focal minus the mean with target A focal, over the pair's SD) and averaged across pairs, so a positive D means faster with target A.

Used on Implicit testing. Stroop is the all-in-one research and testing platform. Run usability tests, prototypes, surveys, tree tests, diaries and IATs in one study, and get results back transcribed, scored and charted, with every finding linked to the session it came from.