AP 101 Brief #18: Misunderstanding and misuse of achievement test scores in Atkins MR/ID death penalty cases: Part 2 -- Range of expected grade equivalents
Kevin S. McGrew, PhD.
Institute for Applied Psychometrics (IAP)
In Part 1 of
this AP 101 Brief Report (AP
101 Brief #17: Misunderstanding and misuse of achievement test scores
in Atkins MR/ID death penalty cases: Part 1 -- Range of
expected standard scores), the 95% confidence band around the expected/predicted
achievement standard score for an individual with an IQ of 70 was calculated to
be 36 points (+18), assuming a correlation between IQ and
achievement tests of .75.[1] The point-specific expected/predicted
achievement standard score (that accounted for regression-to-the-mean) was 78 (+18; 60 to 96 95% confidence
band for expected/predicted scores). If
the reader has not read the first installment in this series, I strongly
recommend you stop reading the current brief and read the first brief. Part 1 provides considerable background
information upon which this second part in the series is based. Below is the visual-graphic summary of Part 1
of this series. [Click on all images to enlarge them for better viewing]
In Part 2 of
this series, the expected/achievement score of 78, as well as the expected
range (@ 95% confidence) of standard scores of 60 to 90 are converted to grade
equivalents (GE) for Broad Reading, Math and Written Language at ages 25, 35,
and 45 years of age in the WJ III NU
norm data. The following general
procedure was followed by accessing the WJ
III norm tables. The WJ III NU norm tables were used as they
provide data-based values associated with expected standard scores (and GEs)
and not values based on prediction equations not based on real data or
statistical simulations.
·
For each of the three WJ III achievement clusters, the specific WJ III W-score[2]
associated with a standard score (based on age norms) of 78 was
identified. This W-score was then entered in the WJ
III NU grade norm tables to identify the specific GE associated with the W-score. This step was repeated for the lower (60) and
upper (96) standard scores of the 95% confidence band scores—resulting in GE
values for both standard scores for each of the three achievement clusters. This resulted in three GE values at each of
the three selected age groups (GE for achievement SS = 78; GE for achievement
SS = 60; GE for achievement SS = 96).
These three sets of values were then plotted on graphs and lines
connecting each corresponding GE/SS value connect.
The three resulting figures are presented
below. The Broad Reading figure is
discussed with the general interpretation being the same for Broad Math and
Written Language, although the specific GE values in each figure should be
substituted for those discussed with regard to Broad Reading. [Click on images to enlarge for better viewing].
The Broad
Reading GE values associated with the expected/predicted SS of 78 is 5.5 (25
years of age), 4.1 (35 years of age), and 4.4 (45 years of age)—ranging from the
beginning of 4th grade to the middle of 5th grade. This is the bold middle line. The Broad
Reading GE values associated with the expected/predicted SS of 60 is 3.0 (25
years of age), 2.0 (35 years of age), and 2.7 (45 years of age). This is the bottom line in each figure. The Broad Reading GE values associated with
the expected/predicted SS of 96 is 10.9 (25 years of age), 11.2 (35 years of
age), and 11.4 (45 years of age). This
is the top line in each figure.
Thus, for a person with an IQ score of 70, the
expected WJ II Broad Reading achievement GE’s range between 4.1 and 5.5,
depending on age. However, given the
large range of standard scores associated with the 95% prediction confidence
band (range of 36 points), it is not surprising that this range, when converted
to GE’s, can vary from between 2.0 /3.0 to the end of 10th grade and
the beginning 11th grade (10.9 to 11.4).
A quick
review of the figures most likely raises many questions. For example, why is the distance between the
bottom line (GE’s associated with SS=60) and the middle line (GE associated with
expected/predicted score of 78) much narrower than the distance between the
same middle line and the top line (GE associated with expected/predicted score
of 96). Also, why are the three lines not
consistently linear? The answers to
these questions would require excessive detail, statistical explanations, more
graphs, etc., that would likely confuse readers. The answer lies in the fact that (a) standard
scores are equal interval metrics and GE’s are not, (b) standard scores
are partially derived from the standard deviation (SD) of the W-scores at
each age within each achievement domain, and these values are not the
same across achievement domains nor across ages, and (c) W-score growth score curves show
differential rates of rapid growth during the early ages/grades, then a
plateau, and then a much slower rate of decline. Enough said.
Summary
Similar to
the conclusion from Part 1 of this report, which dealt with expected standard
scores, the expected range of GE’s for adults (ages 25 to 45) with an IQ of 70
can, for some individuals, vary tremendously.
The presence of some achievement scores significantly above expectations
for an IQ associated with mild MR/ID (70), possibly into the junior and senior
high grade levels, are possible when the less-than-perfect correlation between
IQ and achievement scores is acknowledged.
One must recognize that although correlations in the .70’s are high
and statistically significant, they indicate that IQ scores can only account for up to approximately ½ (50% of tested
achievement scores).[3]
Too many lay persons and, unfortunately many
educators and psychologists, have fallen prey to the IQ-Ach fallacy, which is the non-science based assumption or belief
that individuals can only achieve at or below their measured achievement. The appropriate scientific fact is that for any IQ score there is a symmetrical range
of possible expected achievement scores which, whether reported in terms of
standard scores or GE’s, can be large. Achievement
scores that are above predicted levels based on measured IQ scores will occur
with some degree of regularity for individuals with mild MR/ID and should
not be incorrectly interpreted as a knee-jerk indication that a person may
not considered for diagnoses as MR/ID, assuming they meet all relevant criteria
or prongs.
Finally, the all the calculations in Part 1 and 2 of this series are based on the WJ III NU norm data. The extent to which the results, especially the GE results, generalize to other achievement tests is unknown. However, I am reasonably confident that although the specific GE's that would be obtained by completing the same methods with the norm data from different achievement tests (e.g., WIAT series) might vary slightly, the overarching conclusion that "achievement
scores that are above predicted levels based on measured IQ scores will occur
with some degree of regularity for individuals with mild MR/ID and should
not be incorrectly interpreted as a knee-jerk indication that a person may
not considered for diagnoses as MR/ID, assuming they meet all relevant criteria
or prongs" would generalize.
[1] If a lower level of IQ/ACH
correlation is assumed then the range of expected standard scores (Part 1
report) or grade equivalents (GE) will be larger.
[2] The WJ III scales are based on Rasch
Item Response Theory (IRT) scaling methods that results in raw scores being
converted to the equal interval W-score
growth metric, which is then used to calculate all derived scores (AE, GE, SS,
etc.)
[3] This percent figure represents
the coefficient of determination which is calculated by squaring a correlation
(e.g., r = .70 squared is .49) and then multiplying the value by 100% (thus,
49%).
