This is a guest blog post by Dr. Dale Watson. The opinions expressed do not necessarily reflect the official position of the ICDP blog or the blogmaster. However, it is of interest to note that the error Dr. James Flynn (2009) made in reporting the WAIS-IV norming date (here reported by Dr. Dale Watson) is true, and was also in a published review that I received a few days after I received Dr. Watson's guest post. This second verification source (Kaufman, Dillon, & Kirsch, 2013) will be the subject of my next post.
Dr. Dale Watson's guest post
In an article entitled, The WAIS-III and WAIS-IV: Daubert motions favor the certainly false
over the approximately true, Dr. James Flynn analyzed data from a number of
IQ tests, including the WAIS-R, WAIS-III, and WAIS-IV to estimate the rate of
the “Flynn Effect” on the Wechsler scales in the U.S. over time.[i] He
concluded, as have others, that in order to account for the obsolescence of
aging IQ test norms, a “Flynn Effect” adjustment of 0.30 points per year from
the date of a tests norming should be applied to the obtained IQ test scores
(Flynn, 2009; Fletcher et al., 2010). For example, if the WAIS-III (normed in
1995) was administered to an individual in 2005, the obtained IQ should be
downwardly adjusted by 0.30 x 10 or 3.0 points. Thus, an obtained IQ score of
72 would result in a Flynn-adjusted score of 69. Such adjustments have been
recommended for use in Atkins evaluations
(Flynn, 2009; Gresham & Reschly, 2011; cf
Hagen et al., 2010).[ii]
Flynn compared the IQ scores obtained on the
WAIS-III and the WAIS-IV in a sample of 240 examinees reported in the Technical and Interpretive Manual for the WAIS-IV (2008).[iii]
The Technical Manual reported that the mean IQs differed by 2.9 points with the
sample mean for the WAIS-IV being 100 and for the WAIS-III 102.9 (Wechsler,
2008, p. 75). However, because these IQ scores were calculated using different
combinations of subtests, Flynn re-calculated the IQ scores utilizing the same
combination of 11 subtest scores used on the WAIS-III to calculate the IQs.
Flynn (2009) noted, “The list of subtests used to compute Full Scale IQ had not
only changed, but had dropped from 11 to 10. But, once again, they gave the
comparison group all 11 of the old WAIS-III subtests, and once again that was
fortunate because it meant that the true obsolescence of the WAIS-III could be
measured. I calculated the total standard score the group got on the same 11
WAIS-III and WAIS-IV subtests. Using the totals and the WAIS-III conversion
table, I calculated Full Scale IQs for the two tests” (p. 102).
In examining Flynn’s Table 2, it appears that these
calculations included scores for the Picture Arrangement subtest for both the
WAIS-III and WAIS-IV. However, the Picture Arrangement subtest is not included
in the WAIS-IV so it is quite unclear how this calculation was performed. Moreover,
there is a footnote to this table indicating that the “WAIS-IV estimate is
eccentric in carrying over WISC-III subtests (and scoring vs. the WAIS-III
tables)…” but the meaning of this statement is also uncertain. In addition,
substitution of the Symbol Search subtest for Picture Arrangement appears to
yield very similar results.
In any case, the point of this note is not to
recalculate Flynn’s estimates but rather to point out what appears to be a discrepancy between WAIS-IV norming date
provided by Flynn and that found in the Technical and Interpretive Manual for
the WAIS-IV. Flynn indicated that the WAIS-IV was normed in 2006 (Table 1)
whereas the Manual reported, “The WAIS-IV normative data was established using
a sample collected from March 2007 to April 2008.” [iv]
If we use 2007 as the mid-point norming date, the time between the norming of
the WAIS-III and WAIS-IV is 12 years and not 11 as provided by Flynn. Using the
Flynn 2006 date resulted in a calculated Flynn Effect between the WAIS-III and WAIS-IV
of 0.306 points per year (+3.37 / 11 years). Using the norming date provided in
the manual resulted in a calculated score of 0.281 points per year (+3.37 / 12
years). It is understood that this discrepancy of just 0.025 points is of little
practical significance but it should be noted nonetheless. Moreover, the
metaphorical splitting of hairs is not uncommon when discussing the Flynn
Effect. Hagan et al. (2010) asserted, “Decades of FE research and testimony…
depict the amount of this shift as a moving target. For example, Flynn
(1998) once identified the annual shift as 0.25 rather than 0.30, but later
testified in Ex Parte Eric Dewayne Cathey (2010) that 0.29 would be
appropriate. Schalock et al. (2010) have called for an annual adjustment of
0.33” pp. 1-2.[v]
Flynn has acknowledged that the results reported in his report are estimates
for the Wechsler scales, writing, “It is quite possible that the rate of gain
on Wechsler tests is 0.275 or 0.325 points per year” (Flynn, 2009, p. 104). The
recalculation noted here is consistent with this judgment. Further, the weight
of the available evidence, including that of a recent meta-analysis, continues
to support the Flynn Effect adjustment of 0.3 points per year.[vi]
[i] Flynn, J. R.
(2009). The WAIS-III and WAIS-IV: Daubert
motions favor the certainly false over the approximately true. Applied
Neuropsychology, 16(2), 98-104. doi: 10.1080/09084280902864360
[ii] Gresham, F. M.,
& Reschly, D. J. (2011). Standard of practice and Flynn Effect testimony in
death penalty cases. Intellectual and Developmental Disabilities, 49(3),
131-140. doi: 10.1352/1934-9556-49.3.131
[iii] Wechsler, D.
(2008). Wechsler Adult Intelligence
Scale: Technical and interpretive manual (4th ed.). San Antonio,
TX: Pearson.
[iv] Id., p. 22.
[v] Hagan, L. D.,
Drogin, E. Y., & Guilmette, T. J. (2010). IQ scores should not be adjusted
for the Flynn Effect in capital punishment cases. Journal of
Psychoeducational Assessment, 28(5), 474-476. doi:
10.1177/0734282910373343
[vi]
Fletcher, J. M., Stuebing, K. K., & Hughes, L. C. (2010). IQ scores should
be corrected for the Flynn Effect in high-stakes decisions. Journal of Psychoeducational Assessment,
28(5), 469-473. doi:
10.1177/0734282910373341
