IQ score differences across time may relfect real changes in the brain

Lay people and many professionals often express consternation when an individuals measured IQ scores are different at different times in their life.  This concern is particularly heightened in high stakes settings where differences in IQ scores can result in changes in eligibility for programs (e.g., social security disability income) or life-or-death decisions (e.g., Atkins MR/ID death penalty cases).

Factors contributing to significant IQ score differences are many (McGrew, in press a) and may include: (a) procedural or test administration errors (e.g., scoring errors; improper nonstandardized test administration; malingering; age vs. grade norms; practice effects), (b) test norm or standardization differences (e.g., norm obsolescence or the Flynn Effect; McGrew, in press b), (c) content differences across different test batteries or between different editions of the same battery, or (d) variations in a person’s performance on different occasions.

 An article "in press" (Neuroimage) by Burgaleta et al. (click here to view copy with annotated comments)  provides the important reminder that differences in IQ scores for an individual (across time) may be due to real changes in general intelligence related to real changes in brain development.  These researchers found that changes in cortical brain thickness were related to changes in IQ scores.  They concluded that "the dynamic nature of intelligence-brain relations...support the idea that changes in IQ across development can reflect meaningful general cognitive ability changes and have a neuroanatomical substrate" (viz., changes in cortical thickness in key brain regions).  The hypothesis was offered that changes in the the cortical areas of  frontoparietal brain network (see P-FIT model of intelligence) may be related to changes in working memory, which in turn has been strongly associated with general reasoning (fluid intelligence; Gf).

The cortical thickness-IQ change relation was deemed consistent with "cellular events that are sensitive to postnatal development and experience."  Possible causal factors suggested included insufficient education or social stimulation during sensitive developmental periods, as well as lifestyle, diet and nutrition, and genetic factors.

• McGrew, K. S. (in press a).  Intellectual functioning:  Conceptual issues.  In E. Polloway (Ed.), Determining intellectual disability in the courts:  Focus on capital cases.  AAIDD, Washington, DC.

•  McGrew, K. S. (in press b).  Norm obsolescence:  The Flynn Effect.  In E. Polloway (Ed.), Determining intellectual disability in the courts:  Focus on capital cases.  AAIDD, Washington, DC.

[Click on images to enlarge]

IAP AP101 Brief # 5:The Wechsler-like IQ subtest scaled score metric: The potential for misuse, misinterpretation and impact on critical life decisions

This is a revised post of a previous post (which has now been deleted).  The earlier post indicated that the report brief described below was in draft form---and I was seeking feedback and comments.  A number of individuals did provide some constructive feedback.  As a result, I revised the report (only slightly) and have posted the final version at the link mentioned below.  Thanks for the feedback.  This is now listed under the IAP AP101 Brief section of the blog sidebar.

Below are the introductory paragraphs to IAP AP Brief #5.  The complete report is available for online viewing or downloading by clicking here.  Enjoy.

I've recently been skimming James Flynn's new book (What is Intelligence:  Beyond the Flynn Effect) to better understand the methodology and interpretation of the Flynn effect. Of particular interest to me (as an applied measurement person) is his analysis of the individual subtest scores from the various Wechsler scales across time. As most psychologists know, Wechsler subtest scaled scores (ss) are on a scale with a mean (M) = 10 and a standard deviation (SD) = 3. The subtest ss range from 1 to 19.  In Appendix 1 of his book, Flynn states "it is customary to score subtests on a scale in which the SD is 3, as opposed to IQ scores which are scaled with SD set at 15. To convert to IQ, just multiply subtest gains by five, as was done to get the IQ gains in the last column."  At first glance, this statement makes it sound as if the transformation of subtest ss to IQ SS is an easy (“just multiply….”; emphasis added by me) and mathematically acceptable procedure without problems. However, on close inspection this transformation has the potential to introduce unknown sources of error into the precision of the transformed SS scores.  It is the goal of this brief technical post to explain the issues involved when making this ss-to- IQ SS conversion.

The ss 1-19 scale has a long history in the Wechsler batteries. For sample, in Appendix 1 of Measurement of Adult Intelligence (Wechsler, 1944), Wechsler described the steps used to translate subtest raw scores to the new ss metric. The Wechsler batteries have continued this tradition in each new revision, although the methodology and procedures to calculate the ss 1-19 values have become more sophisticated over time.   Although the methods used to develop the Wechsler ss 1-19 scale may have become more sophisticated, the resultant underlying scale for each subtest has not…scores still range from 1-19 (M=10; SD=3).  Also, the most recent Stanford-Binet—5th Edition (SB5; Roid, 2003) and Kaufman Assessment Battery for Children-2nd Edition (KABC-II) have both adopted the same ss 1-19 scale for their respective individual subtests.

Why is this relatively crude (to be defined below) scale metric still used in some intelligence batteries when other contemporary intelligence batteries provide subtest scale metrics with finer measurement resolution?  For example, the DAS-II (Elliott, 2007) places individual test scores on the T-scale (M=50; SD=10), with scores that range from 10-90.  The WJ III (McGrew & Woodcock, 2001) places all test and composite scores on the standard score (SS) metric associated with full scale and composite scores (M=100; SD=15).  The critical question to be asked is “are there advantages or disadvantages to retaining the historical ss 1-19 scale or, are their real advantages to having individual test scales with finer measurement resolution (DAS-II; WJ III)?”

......continued............
(complete report available at links in first paragraph of this post)

[Double click on image to enlarge]

Technorati Tags: psychology, Flynn Effect,school psychology, educational psychology, neuropsychology, forensice psychology, criminal psychology, criminal justice, psychometrics, Wechslers, WISC-III, WISC-IV, WAIS-III, WAIS-IV, WJ III, DAS-II, KABC-II, SB5, intelligence, IQ tests, IQ scores, scaled scores, MR, ID, mental retardation, intellectual disability, capital punishment, death penalty, IAP AP101 report

Why IQ test scores differ: Applied Psychometrics 101---IQ scoring errors (next report)

Is it possible for an individual evaluated for mental retardation (as part of an Atkins proceedings) to have an increased probability of facing execution depending on whom administers them an intelligence test?  Is it possible for an experienced psychological examiner to make a sufficient number of scoring errors that significantly change a person's IQ score from what it should be (if properly scored)?  Unfortunately, the answers are "yes." 

I learned this first hand when I reviewed the test record and scoring of a intelligence test (on which I'm a coauthor) in a Federal death penalty appeal hinging on the diagnosis of mental retardation.  I've since been locating research articles on the accuracy of IQ test scoring for novice and experienced psychological examiners.  The results are discouraging. 

The next AP101 report will address the issue of test scoring errors in intelligence testing, with a particular emphasis on implications for Atkins MR death penalty cases.  The report will include a summary of representative literature, a discussion of my findings in the recent case for which I was a consultant (presented in such a manner to not reveal the identity of the case or any individuals/agencies involved in the case), and recommendations to address the issue.

Stay tuned.

If you have not read the first report in the series, check it out.  AP101 101:  IQ Test Score Difference Series--#1 Understanding global IQ test correlations.

echnorati Tags: psychology, forensic psychology, criminal psychology, criminal justice, clinical psychology, neuropsychology, Atkins, MR, mental retardation, developmental disabilities, IQ, IQ tests, IQ scores, scoring errors, reliability, ISIR, intelligence testing