Retest effects in working memory capacity tests: A meta-analysis
Jana Scharfen, Katrin Jansen, Heinz Holling. Article link
© Psychonomic Society, Inc. 2018
Abstract
The repeated administration of working memory capacity tests is common in clinical and research settings. For cognitive ability tests and different neuropsychological tests, meta-analyses have shown that they are prone to retest effects, which have to be accounted for when interpreting retest scores. Using a multilevel approach, this meta-analysis aims at showing the reproducibility of retest effects in working memory capacity tests for up to seven test administrations, and examines the impact of the length of the test-retest interval, test modality, equivalence of test forms and participant age on the size of retest effects. Furthermore, it is assessed whether the size of retest effects depends on the test paradigm. An extensive literature search revealed 234 effect sizes from 95 samples and 68 studies, in which healthy participants between 12 and 70 years repeatedly performed a working memory capacity test. Results yield a weighted average of g = 0.28 for retest effects from the first to the second test administration, and a significant increase in effect sizes was observed up to the fourth test administration. The length of the test-retest interval and publication year were found to moderate the size of retest effects. Retest effects differed between the paradigms of working memory capacity tests. These findings call for the development and use of appropriate experimental or statistical methods to address retest effects in working memory capacity tests.
Keywords Meta-analysis · Retest effect · Practice effect · Working memory
- Posted using BlogPress from my iPad
Showing posts with label meta-analysis. Show all posts
Showing posts with label meta-analysis. Show all posts
Sunday, July 8, 2018
Saturday, June 2, 2018
Can emotional intelligence (Gei) be trained: A meta-analysis
Can emotional intelligence be trained? A meta-analysis
Please cite this article as: Mattingly, V., Human Resource Management Review (2018), https://doi.org/10.1016/j.hrmr.2018.03.002
Victoria Mattingly, Kurt Kraiger
Keywords: Emotional intelligence, Training Meta-analysis
A B S T R A C T
Human resource practitioners place value on selecting and training a more emotionally in-telligent workforce. Despite this, research has yet to systematically investigate whether emo-tional intelligence can in fact be trained. This study addresses this question by conducting a meta-analysis to assess the effect of training on emotional intelligence, and whether effects are mod-erated by substantive and methodological moderators. We identified a total of 58 published and unpublished studies that included an emotional intelligence training program using either a pre-post or treatment-control design. We calculated Cohen's d to estimate the effect of formal training on emotional intelligence scores. The results showed a moderate positive effect for training, regardless of design. Effect sizes were larger for published studies than dissertations. Effect sizes were relatively robust over gender of participants, and type of EI measure (ability v. mix-edmodel). Further, our effect sizes are in line with other meta-analytic studies of competency-based training programs. Implications for practice and future research on EI training are discussed.
See prior Gei posts here and here.
- Posted using BlogPress from my iPad
Please cite this article as: Mattingly, V., Human Resource Management Review (2018), https://doi.org/10.1016/j.hrmr.2018.03.002
Victoria Mattingly, Kurt Kraiger
Keywords: Emotional intelligence, Training Meta-analysis
A B S T R A C T
Human resource practitioners place value on selecting and training a more emotionally in-telligent workforce. Despite this, research has yet to systematically investigate whether emo-tional intelligence can in fact be trained. This study addresses this question by conducting a meta-analysis to assess the effect of training on emotional intelligence, and whether effects are mod-erated by substantive and methodological moderators. We identified a total of 58 published and unpublished studies that included an emotional intelligence training program using either a pre-post or treatment-control design. We calculated Cohen's d to estimate the effect of formal training on emotional intelligence scores. The results showed a moderate positive effect for training, regardless of design. Effect sizes were larger for published studies than dissertations. Effect sizes were relatively robust over gender of participants, and type of EI measure (ability v. mix-edmodel). Further, our effect sizes are in line with other meta-analytic studies of competency-based training programs. Implications for practice and future research on EI training are discussed.
See prior Gei posts here and here.
- Posted using BlogPress from my iPad
Saturday, May 19, 2018
The Relation between Intelligence and Adaptive Behavior: A Meta-Analysis
Very important meta-analysis of AB IQ relation. Primary finding on target with prior informal synthesis by McGrew (2015)
The Relation between Intelligence and Adaptive Behavior: A Meta-Analysis
Ryan M. Alexander
ABSTRACT
Intelligence tests and adaptive behavior scales measure vital aspects of the multidimensional nature of human functioning. Assessment of each is a required component in the diagnosis or identification of intellectual disability, and both are frequently used conjointly in the assessment and identification of other developmental disabilities. The present study investigated the population correlation between intelligence and adaptive behavior using psychometric meta-analysis. The main analysis included 148 samples with 16,468 participants overall. Following correction for sampling error, measurement error, and range departure, analysis resulted in an estimated population correlation of ρ = .51. Moderator analyses indicated that the relation between intelligence and adaptive behavior tended to decrease as IQ increased, was strongest for very young children, and varied by disability type, adaptive measure respondent, and IQ measure used. Additionally, curvilinear regression analysis of adaptive behavior composite scores onto full scale IQ scores from datasets used to report the correlation between the Wechsler Intelligence Scales for Children- Fifth edition and Vineland-II scores in the WISC-V manuals indicated a curvilinear relation—adaptive behavior scores had little relation with IQ scores below 50 (WISC-V scores do not go below 45), from which there was positive relation up until an IQ of approximately 100, at which point and beyond the relation flattened out. Practical implications of varying correlation magnitudes between intelligence and adaptive behavior are discussed (viz., how the size of the correlation affects eligibility rates for intellectual disability).
Other Key Findings Reported
McGrew (2012) augmented Harrison's data-set and conducted an informal analysis including a total of 60 correlations, describing the distributional characteristics observed in the literature regarding the relation. He concluded that a reasonable estimate of the correlation is approximately .50, but made no attempt to explore factors potentially influencing the strength of the relation.
Results from the present study corroborate the conclusions of Harrison (1987) and McGrew (2012) that the IQ/adaptive behavior relation is moderate, indicating distinct yet related constructs. The results showed indeed that the correlation is likely to be stronger at lower IQ levels—a trend that spans the entire ID range, not just the severe range. The estimated true mean population is .51, and study artifacts such as sampling error, measurement error, and range departure resulted in somewhat attenuated findings in individual studies (a difference of about .05 between observed and estimated true correlations overall).
The present study found the estimated true population mean correlation to be .51, meaning that adaptive behavior and intelligence share 26% common variance. In practical terms, this magnitude of relation suggests that an individual's IQ score and adaptive behavior composite score will not always be commensurate and will frequently diverge, and not by a trivial amount. Using the formula Ŷ = Ȳ + ρ (X - X ̅ ), where Ŷ is the predicted adaptive behavior composite score, Ȳ is the mean adaptive behavior score in the population, ρ is the correlation between adaptive behavior and intelligence, X is the observed IQ score for an individual, and X ̅ is the mean IQ score, and accounting for regression to the mean, the predicted adaptive behavior composite score corresponding to an IQ score of 70, given a correlation of .51, would be 85 —a score that is a full standard deviation above an adaptive behavior composite score of 70, the cut score recommended by some entities to meet ID eligibility requirements. With a correlation of .51, and accounting for regression to the mean, an IQ score of 41 would be needed in order to have a predicted adaptive behavior composite score of 70. Considering that approximately 85% of individuals with ID have reported IQ scores between 55 and 70±5 (Heflinger et al., 1987; Reschly, 1981), the eligibility implications, especially for those with less severe intellectual impairment, are alarming. In fact, derived from calculations by Lohman and Korb (2006), only 17% of individuals obtaining an IQ score of 70 or below would be expected to also obtain an adaptive behavior composite score of 70 or below when the correlation between the two is .50.
The purpose of this study was to investigate the relation between IQ and adaptive behavior and variables moderating the relation using psychometric meta-analysis. The findings contributed in several ways to the current literature with regard to IQ and adaptive behavior. First, the estimated true mean population correlation between intelligence and adaptive behavior following correction for sampling error, measurement error, and range departure is moderate, indicating that intelligence and adaptive behavior are distinct, yet related, constructs. Second, IQ level has a moderating effect on the relation between IQ and adaptive behavior. The correlation is likely to be stronger at lower IQ levels, and weaker as IQ increases. Third, while not linear, age has an effect on the IQ/adaptive behavior relation. The population correlation is highest for very young children, and lowest for children between the ages of five and 12. Fourth, the magnitude of IQ/adaptive behavior correlations varies by disability type. The correlation is weakest for those without disability, and strongest for very young children with developmental delays. IQ/adaptive behavior correlations for those with ID are comparable to those with autism when not matched on IQ level. Fifth, the IQ/adaptive correlation when parents/caregivers serve as adaptive behavior respondents is comparable to when teachers act as respondents, but direct assessment of adaptive behavior results in a stronger correlation. Sixth, an individual's race does not significantly alter the correlation between IQ and adaptive behavior, but future research should evaluate the influence of race of the rater on adaptive behavior ratings. Seventh, the correlation between IQ and adaptive behavior varies depending on IQ measure used—the population correlation when Stanford-Binet scales are employed is significantly higher than when Wechsler scales are employed. And eighth, the correlation between IQ and adaptive behavior is not significantly different between adaptive behavior composite scores obtained from the Vineland, SIB, and ABAS families of adaptive behavior measures, which are among those that have been deemed appropriate for disability identification. Limitations of this study notwithstanding, it is the first to employ meta-analysis procedures and techniques to examine the correlation between intelligence and adaptive behavior and how moderators alter this relation. The results of this study provide information that can help guide practitioners, researchers, and policy makers with regard to the diagnosis or identification of intellectual and developmental disabilities.
- Posted using BlogPress from my iPad
The Relation between Intelligence and Adaptive Behavior: A Meta-Analysis
Ryan M. Alexander
ABSTRACT
Intelligence tests and adaptive behavior scales measure vital aspects of the multidimensional nature of human functioning. Assessment of each is a required component in the diagnosis or identification of intellectual disability, and both are frequently used conjointly in the assessment and identification of other developmental disabilities. The present study investigated the population correlation between intelligence and adaptive behavior using psychometric meta-analysis. The main analysis included 148 samples with 16,468 participants overall. Following correction for sampling error, measurement error, and range departure, analysis resulted in an estimated population correlation of ρ = .51. Moderator analyses indicated that the relation between intelligence and adaptive behavior tended to decrease as IQ increased, was strongest for very young children, and varied by disability type, adaptive measure respondent, and IQ measure used. Additionally, curvilinear regression analysis of adaptive behavior composite scores onto full scale IQ scores from datasets used to report the correlation between the Wechsler Intelligence Scales for Children- Fifth edition and Vineland-II scores in the WISC-V manuals indicated a curvilinear relation—adaptive behavior scores had little relation with IQ scores below 50 (WISC-V scores do not go below 45), from which there was positive relation up until an IQ of approximately 100, at which point and beyond the relation flattened out. Practical implications of varying correlation magnitudes between intelligence and adaptive behavior are discussed (viz., how the size of the correlation affects eligibility rates for intellectual disability).
Other Key Findings Reported
McGrew (2012) augmented Harrison's data-set and conducted an informal analysis including a total of 60 correlations, describing the distributional characteristics observed in the literature regarding the relation. He concluded that a reasonable estimate of the correlation is approximately .50, but made no attempt to explore factors potentially influencing the strength of the relation.
Results from the present study corroborate the conclusions of Harrison (1987) and McGrew (2012) that the IQ/adaptive behavior relation is moderate, indicating distinct yet related constructs. The results showed indeed that the correlation is likely to be stronger at lower IQ levels—a trend that spans the entire ID range, not just the severe range. The estimated true mean population is .51, and study artifacts such as sampling error, measurement error, and range departure resulted in somewhat attenuated findings in individual studies (a difference of about .05 between observed and estimated true correlations overall).
The present study found the estimated true population mean correlation to be .51, meaning that adaptive behavior and intelligence share 26% common variance. In practical terms, this magnitude of relation suggests that an individual's IQ score and adaptive behavior composite score will not always be commensurate and will frequently diverge, and not by a trivial amount. Using the formula Ŷ = Ȳ + ρ (X - X ̅ ), where Ŷ is the predicted adaptive behavior composite score, Ȳ is the mean adaptive behavior score in the population, ρ is the correlation between adaptive behavior and intelligence, X is the observed IQ score for an individual, and X ̅ is the mean IQ score, and accounting for regression to the mean, the predicted adaptive behavior composite score corresponding to an IQ score of 70, given a correlation of .51, would be 85 —a score that is a full standard deviation above an adaptive behavior composite score of 70, the cut score recommended by some entities to meet ID eligibility requirements. With a correlation of .51, and accounting for regression to the mean, an IQ score of 41 would be needed in order to have a predicted adaptive behavior composite score of 70. Considering that approximately 85% of individuals with ID have reported IQ scores between 55 and 70±5 (Heflinger et al., 1987; Reschly, 1981), the eligibility implications, especially for those with less severe intellectual impairment, are alarming. In fact, derived from calculations by Lohman and Korb (2006), only 17% of individuals obtaining an IQ score of 70 or below would be expected to also obtain an adaptive behavior composite score of 70 or below when the correlation between the two is .50.
The purpose of this study was to investigate the relation between IQ and adaptive behavior and variables moderating the relation using psychometric meta-analysis. The findings contributed in several ways to the current literature with regard to IQ and adaptive behavior. First, the estimated true mean population correlation between intelligence and adaptive behavior following correction for sampling error, measurement error, and range departure is moderate, indicating that intelligence and adaptive behavior are distinct, yet related, constructs. Second, IQ level has a moderating effect on the relation between IQ and adaptive behavior. The correlation is likely to be stronger at lower IQ levels, and weaker as IQ increases. Third, while not linear, age has an effect on the IQ/adaptive behavior relation. The population correlation is highest for very young children, and lowest for children between the ages of five and 12. Fourth, the magnitude of IQ/adaptive behavior correlations varies by disability type. The correlation is weakest for those without disability, and strongest for very young children with developmental delays. IQ/adaptive behavior correlations for those with ID are comparable to those with autism when not matched on IQ level. Fifth, the IQ/adaptive correlation when parents/caregivers serve as adaptive behavior respondents is comparable to when teachers act as respondents, but direct assessment of adaptive behavior results in a stronger correlation. Sixth, an individual's race does not significantly alter the correlation between IQ and adaptive behavior, but future research should evaluate the influence of race of the rater on adaptive behavior ratings. Seventh, the correlation between IQ and adaptive behavior varies depending on IQ measure used—the population correlation when Stanford-Binet scales are employed is significantly higher than when Wechsler scales are employed. And eighth, the correlation between IQ and adaptive behavior is not significantly different between adaptive behavior composite scores obtained from the Vineland, SIB, and ABAS families of adaptive behavior measures, which are among those that have been deemed appropriate for disability identification. Limitations of this study notwithstanding, it is the first to employ meta-analysis procedures and techniques to examine the correlation between intelligence and adaptive behavior and how moderators alter this relation. The results of this study provide information that can help guide practitioners, researchers, and policy makers with regard to the diagnosis or identification of intellectual and developmental disabilities.
- Posted using BlogPress from my iPad
Sunday, October 29, 2017
Meta-analysis supports cognitive ability differentiation hypotheses (SLODOR)
A B S T R A C T
The cognitive ability differentiation hypothesis, which is also termed Spearman's Law of Diminishing Returns, proposes that cognitive ability tests are less correlated and less g loaded in higher ability populations. In ad-dition, the age differentiation hypothesis proposes that the structure of cognitive ability varies across respondent age. To clarify the literature regarding these expectations, 106 articles containing 408 studies, which were published over a 100-year time span, were analyzed to evaluate the empirical basis for ability as well as age differentiation hypotheses. Meta-analyses provide support for both hypotheses and related expectations. Results demonstrate that the mean correlation and g loadings of cognitive ability tests decrease with increasing ability, yet increase with respondent age. Moreover, these effects have been nearly constant throughout the century of analyzed data. These results are important because we cannot assume an invariant cognitive structure for dif-ferent ability and age levels. Implications for practice as well as drawbacks are further discussed.
The cognitive ability differentiation hypothesis, which is also termed Spearman's Law of Diminishing Returns, proposes that cognitive ability tests are less correlated and less g loaded in higher ability populations. In ad-dition, the age differentiation hypothesis proposes that the structure of cognitive ability varies across respondent age. To clarify the literature regarding these expectations, 106 articles containing 408 studies, which were published over a 100-year time span, were analyzed to evaluate the empirical basis for ability as well as age differentiation hypotheses. Meta-analyses provide support for both hypotheses and related expectations. Results demonstrate that the mean correlation and g loadings of cognitive ability tests decrease with increasing ability, yet increase with respondent age. Moreover, these effects have been nearly constant throughout the century of analyzed data. These results are important because we cannot assume an invariant cognitive structure for dif-ferent ability and age levels. Implications for practice as well as drawbacks are further discussed.
Monday, June 15, 2015
Another Flynn effect (norm obsolescence) meta-analysis
A second massive meta-analysis of the Flynn effect (norm obsolescence) was recently published (Pietschnig & Voraseck, 2015). The study investigated different Flynn effect ability domain effects (by Gf, Gc, Gv), and other moderating variables. However, the most important conclusion is that the authors conclude that the 3 IQ points per decade rule-of-thumb appears to be the best estimate of the FE on global IQ scores. This conclusion is consistent with the Trahan et al. (2014) meta-analysis and the recent AAIDD recommendation.
Friday, June 29, 2012
Research byte: Practice effects on neuropsych and IQ measures: Meta-analysis
Click on images to enlarge
Posted using BlogPress from Kevin McGrew's iPad
www.themindhub.com
Monday, January 30, 2012
Monday, December 27, 2010
German Gc (crystallized IQ) res. synthesis supports Flynn Effect
Copy of open access article can be found by clicking here. Double click on image to enlarge.
- iPost using BlogPress from my Kevin McGrew's iPad
intelligence intelligence testing Atkins cases ICDP blog psychology school psychology neuropsychology Forensic psychology criminal psychology criminal justice death penalty capital punishment ABA IQ tests IQ scores adaptive behavior AAIDD mental retardation intellectual disability Flynn effect
- iPost using BlogPress from my Kevin McGrew's iPad
intelligence intelligence testing Atkins cases ICDP blog psychology school psychology neuropsychology Forensic psychology criminal psychology criminal justice death penalty capital punishment ABA IQ tests IQ scores adaptive behavior AAIDD mental retardation intellectual disability Flynn effect
Subscribe to:
Posts (Atom)