Ever wanted to combine two or three tests from an intelligence or achievement battery into a cluster composite not provided by the authors/publisher of a test?
Most assessment clinicians have. Unfortunately, there has been no psychometrically sound method for doing so. As will be described in a future post, calculating the arithmetic average of individual test scaled or standard scores to generate a pseudo-composite is a psychometric no-no that results in inaccurate scores with serious psychometric flaws In order to calculate defensible composite scores from the standard scores from different subtests, users would need to engage in tedious calculations that include information regarding the intercorrelations between all tests for the desired new cluster, the standard deviations of the different tests (unless all are on a common scale), and the individual reliability of each test. Simply collecting the required information from the technical manual from a test battery took too much time.
Enter the Compositator by Dr. Joel Schneider. This has to be seen and played with to be believed!
The FREE Compositator was released yesterday by the Woodcock-Munoz-Foundation. The description at the WMF web page is:
The Compositator, by W. Joel Schneider, Ph.D., is a tool designed to provide assessment professionals who use the Woodcock-Johnson® III Normative Update Compuscore and Profiles Program, version 3.1 (WJ III NU) and the Woodcock Interpretation and Instructional Interventions Program, version 1.0 (WIIIP) with user-friendly statistical methods to create customized composite scores
So, for users of the WJ II NU, there is now a psychometrically defensible method for creating special clusters. But that is only the beginning and first level use of the program. Dr. Schneider has included a wide variety of features that allow users to create factor analysis scores, prediction equations of unique sets of norm-based or Compositator created composites (e.g., examining the predictive power of a set of variables for predicting basic reading skills, etc), etc.
The unique and innovative value of the program for all test professionals (just not those who use the WJ III NU) is that the program is loaded with all kinds of psychometric-based features that include instructional videos explaining how they work and the underlying basic statistics. I see the program as serving as a very powerful instructional tool in graduate level intelligence testing courses. Students can do "what if" scenarios with different test variables and see how different decisions result in different psychometric outcomes (e.g., different levels of prediction via multiple regression). Causal modeling scenarios can be simulated and compared. These examples only scratch the surface of the programs features.
The PDF manual associated with the program is worth a read if nothing else...as much can be learned about basic psychometric concepts.
Dr. Schneider's free software is highly innovative. IMHO it is a major innovation in intelligence test interpretation and instruction re: basic psychometric concepts.
Yes----the program only works with the WJ III NU. But it is FREE. WMF worked with Dr. Schneider as it saw the need to "push the edge of the envelope" of intelligence test instruction and interpretation. As a result, WMF provided Dr. Schneider access to all the electronic WJ III NU data files that provided the necessary test characteristic information required (as noted in the introduction to this post) to create psychometrically defensible composite scores. The intent is to pull the field of intelligent intelligence testing forward----with other test authors and publishers viewing the concepts embedded in the Compositator as ideas for incorporation in their respective test batteries and software.
The Compositator should be considered a prototype that will serve as a potential illustrative "tipping point" for new innovations in intelligence test interpretation and the teaching of current and future users of intelligence tests about basic psychometric concepts. The goal is to influence the field beyond the WJ III NU. The WJ III NU psychometric guts provided a means for Dr. Schneider to implement his creative and innovative ideas---hopefully to demonstrate what can and should be done to improve intelligence testing practice across the board.
Of course, being able to create any cluster one wants is not without caveats. Crafting new composite scores should only be based on sound clinical, theoretical or research-based evidence. In this sense the program is a dumb tool that cannot be separated from the expertise and skills of the user----after all, "if you give a monkey a Stradivarius violin and you get bad music--you don't blame the violin."
Finally, Dr. Schneider's unique approach to teaching the basics for using the program can be followed at his new blog (Psychometrics from the ground up) where he explains statistical and psychometric concepts via video tutorials...amazing stuff. Check it out and bookmark it for frequent visits (or put it in your RSS feed reader)
[Conflict of interest disclosure - I, Kevin McGrew, am a coauthor of the WJ III NU. I am also the Research Director for WMF].
intelligence IQ tests IQ scores CHC theory Cattell-Horn-Carroll human cognitive abilities psychology school psychology individual differences cognitive psychology neuropsychology special education educational psychology psychometrics psychological assessment psychological measurement IQs Corner Compositator Joel Schneider WMF test composites
- iPost using BlogPress from my Kevin McGrew's iPad
