logo logo European Journal of Psychology and Educational Research

EJPER is is a, peer reviewed, online academic research journal.

Subscribe to

Receive Email Alerts

for special events, calls for papers, and professional development opportunities.

Subscribe

Publisher (HQ)

Eurasian Society of Educational Research
Eurasian Society of Educational Research
Christiaan Huygensstraat 44, Zipcode:7533XB, Enschede, THE NETHERLANDS
Eurasian Society of Educational Research
Headquarters
Christiaan Huygensstraat 44, Zipcode:7533XB, Enschede, THE NETHERLANDS
Research Article

Construct Validity and Diagnostic Utility of the Woodcock Johnson Tests of Cognitive Abilities and Clinical Clusters for Children with Attention Deficit/ Hyperactivity Disorder: A Preliminary Investigation

Linda A. Reddy , Alexander Alperin , Adam Lekwa

The diagnostic utility of the Woodcock- Johnson III Tests of Cognitive Abilities and Clinical Clusters was assessed in a sample of 52 children (26 Att.

T

The diagnostic utility of the Woodcock- Johnson III Tests of Cognitive Abilities and Clinical Clusters was assessed in a sample of 52 children (26 Attention Deficit/Hyperactivity Disordered (ADHD) and 26 matched controls). Multivariate analysis of variance followed by post-hoc testing and d-ratios yielded some statistically significant and clinically meaningful differences between groups on the Cognitive Fluency Cluster and the Tests of Auditory Attention, and Rapid Picture Naming. Discriminant function analyses indicated that the WJ III COG Tests collectively classified 80.77% of the sample correctly (76.92% of controls and 84.62% of children with ADHD correctly identified). The Auditory Attention and Rapid Picture Naming tests were found to make the most significant contribution overall to the discriminant function. Using a cut-score of 85, the WJ-III COG Clinical clusters and subtests examined in this study offered fair to weak diagnostic utility based on indices of sensitivity, specificity, positive and negative predictive power, as well as results of Receiver Operating Characteristic curve analyses. Implications for research and practice are outlined.

Keywords: Assessment, attention deficit hyperactivity disorder, Cattell-Horn-Carroll.

cloud_download PDF
Cite
Article Metrics
Views
526
Download
736
Citations
Crossref
0

References

American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed. Text rev.). American Psychiatric Association.

Barkley, R. A. (1997). Behavioral inhibition, sustained attention, and executive functions: Constructing a unifying theory of ADHD. Psychological Bulletin, 121(1), 65-94. https://doi.org/10.1037/0033-2909.121.1.65

Barkley, R. A., & Murphy, K. R. (2006). Attention–deficit hyperactivity disorder: A clinical workbook (3rd ed.). The Guilford Press. 

Bartolucci, M., & Batini, F. (2020). Reading aloud narrative material as a means for the student's cognitive empowerment. Mind, Brain, and Education14(3), 235-242. https://doi.org/10.1111/mbe.12241

Benson, N. F., Floyd, R. G., Kranzler, J. H., Eckert, T. L., Fefer, S. A., & Morgan, G. B. (2019). Test use and assessment practices of school psychologists in the United States: Findings from the 2017 national survey. Journal of School Psychology72, 29-48. https://doi.org/10.1016/j.jsp.2018.12.004

Björkenstam, E., Björkenstam, C., Jablonska, B., & Kosidou, K. (2018). Cumulative exposure to childhood adversity, and treated attention deficit/hyperactivity disorder: a cohort study of 543 650 adolescents and young adults in Sweden. Psychological medicine, 48(3), 498-507. https://doi.org/10.1017/S0033291717001933

Brown, N. M., Brown, S. N., Briggs, R. D., German, M., Belamarich, P. F., & Oyeku, S. O. (2017). Associations between adverse childhood experiences and ADHD diagnosis and severity. Academic pediatrics, 17(4), 349-355. https://doi.org/10.1016/j.acap.2016.08.013

Canivez, G. L., & Gaboury, A. R. (2016). Construct validity and diagnostic utility of the cognitive assessment system for ADHD. Journal of attention disorders, 20(6), 519-526. https://doi.org/10.1177/1087054713489021

Canivez, G. L., & Sprouls, K. (2005). Assessing the construct validity of the Adjustment Scales for Children and Adolescents. Journal of Psychoeducational Assessment, 23(1), 3-14. https://doi.org/10.1177/073428290502300101

Carper, M. S. (2003). A concurrent validity study comparing executive functioning of the Woodcock-Johnson III Tests of Cognitive Ability and the NEPSY [Unpublished doctoral dissertation]. Texas Woman’s University.

Castellanos, F. X., & Tannock, R. (2002). Neuroscience of attention-deficit/hyperactivity disorder: The search for endophenotypes. Nature Reviews Neuroscience, 3(8), 617-628.

Cohen, J. (1988). Statistical power analyses for the behavioral sciences (2nd ed.). Academic Press.

Crawford, E. N. (2002). Profiles for exceptional samples on the Cognitive Assessment System using configural frequency analysis [Unpublished doctoral dissertation]. University of Northern Colorado.  

Das, J. P., Naglieri, J. A., & Kirby, J. R. (1994). Assessment of cognitive processes: The PASS theory of intelligence. Allyn & Bacon.

Delis, D. C., Kaplan, E., & Kramer, J. H. (2001). Delis Kaplan executive function system. The Psychological Corporation.

Dombrowski, S. C. (2013). Investigating the structure of the WJ-III Cognitive at school age. School Psychology Quarterly, 28(8), 154-169. https://doi.org/10.1037/spq0000010

Dombrowski, S. C. (2014). Exploratory bifactor analysis of the WJ-III Cognitive in adulthood via the Schmid-Leiman procedure. Journal of Psychoeducational Assessment, 32(4), 330 -341. https://doi.org/10.1177/0734282913508243

Dombrowski, S. C., & Watkins, M. W. (2013). Exploratory and higher order factor analysis of the WJ-III full test battery: A school-aged analysis. Psychological Assessment, 25(2), 442-455. https://doi.org/10.1037/a00311335

Doyle, A. E., Biederman, J., Seidman, L. J., Weber, W., & Faraone, S. V. (2000). Diagnostic efficiency of neuropsychological test scores for discriminating boys with and without attention deficit/hyperactivity disorder. Journal of Consulting and Clinical Psychology, 68(3), 477-488. https://doi.org/10.1037/0022-006X.68.3.477

Elliott, C. D. (1990). Differential Ability Scales. The Psychological Corporation.

Floyd, R. G., McCormack, A. C., Ingram, E. L., Davis, A. E., Bergeron, R., & Hamilton, G. (2006). Relations between the Woodcock-Johnson III clinical clusters and measures of executive functions from the Delis-Kaplan Executive Function System. Journal of Psychoeducational Assessment24(4), 303-317. https://doi.org/10.1177/0734282906287823

Floyd, R. G., McGrew, K. S., Barry, A., Rafael, F., & Rogers, J. (2009). General and specific effects on Cattell–Horn–Carroll broad ability composites: Analysis of the Woodcock–Johnson III normative update Cattell–Horn–Carroll factor clusters across development. School Psychology Review, 38(2), 249 –265. https://doi.org/10.1080/02796015.2009.12087836

Ford, L., Keith, T. Z., Floyd, R. G., Fields, C., & Schrank, F. A. (2003). Using the Woodcock-Johnson III Tests of Cognitive Abilities with students with attention deficit/hyperactivity disorder. In F. A. Schrank & D. P. Flanagan (Eds.), WJ III Clinical Use and Interpretation: Scientist-Practitioner Perspectives (pp. 320-342). Academic Press.

Ford-Jones, P. C. (2015). Misdiagnosis of attention deficit hyperactivity disorder: ‘Normal behaviour’and relative maturity. Pediatrics & child health, 20(4), 200-202. https://doi.org/10.1093/pch/20.4.200

Goldstein, S., & Naglieri, J. A. (2015). Using the CAS2 in the comprehensive assessment of ADHD. The ADHD Report, 23(7), 8-10. https://doi.org/10.1521/adhd.2015.23.7.8

Gray, J. A. (1994). Personality dimensions and emotion systems. In P. Ekman & R. J. Davidson (Eds.), The nature of emotion (pp.329-331). Oxford University Press.

Gyldrenkaerne, P., Dillion, H., Sharma, M., & Purdy, S. C. (2014). Attend to this: The relationship between auditory processing disorders and attention deficits. Journal of the American Academy of Audiology. 25(7), 676-687.

Hale, J. B., Reddy, L. A., Decker, S. L., Thompson, R., Henzel, J., Teodori, A., Forrest, E., Eusebio, E., & Denckla, M. B. (2009). Development and validation of an Attention-Deficit/ Hyperactivity Disorder (ADHD) executive function and behavior rating screening battery. Journal of Clinical and Experimental Neuropsychology, 31(8), 897-912. https://doi.org/10.1080/13803390802687423

Hale, J. B., Reddy, L. A., Wilcox, G., McLaughlin, A., Hain, L., Stern, A., Henzel, J., & Eusebio, E. (2010). Best practices in assessing and intervening with ADD/ADHD children and children with other frontal-striatal circuit disorders. In D. C. Miller (Ed.), Best practices in school neuropsychology: Guidelines for effective practice, assessment and evidence-based interventions. John Wiley & Sons.

Harrier, L., & DeOrnellas, K. (2005). Performance of children diagnosed with ADHD on selected planning and reconstitution tests. Applied Neuropsychology, 12(2), 106-119. https://doi.org/10.1207/s15324826an1202_6

Hart, J. S. (2020). A CHC theoretical approach to examining cognitive and Academic Deficits Among Students with ADHD Using a Three-Battery Approach (3999) [Doctoral dissertation, University of Nevada]. UNVL Theses, Dissertation, Professional Papers and Capstones. https://digitalscholarship.unlv.edu/thesesdissertations/3999

Heaton, R. K., Chelune, G. J., Talley, J. L., Kay, G. G., & Curtiss, G. (1993). Wisconsin Card Sorting Test manual: Revised and expanded. Psychological Assessment Resources.

Huberty, C. J. (1984). Issues in the use of discriminant analysis. Psychological Bulletin, 95(1), 156-171. https://doi.org/10.1037/0033-2909.95.1.156

EveryCRSReport. (2007). Individuals with Disabilities Education Act of 2004, P.L. 108-446. EveryCRSReport. https://www.everycrsreport.com/reports/RL32716.html

Keith, T. Z., & Reynolds, M. R. (2012). Using confirmatory factor analysis to aid in understanding the constructs measured by intelligence tests. In D. P. Flanagan & P. L. Harrison (Eds.), Contemporary intellectual assessment: Theories, tests, and issues (3rd ed., pp. 758 –799). Guilford Press.

Keith, T. Z., Reynolds, M. R., Patel, P. G., & Ridley, K. P. (2008). Sex differences in latent cognitive abilities ages 6 to 59: Evidence from the Woodcock–Johnson III tests of cognitive abilities. Intelligence, 36(6), 502–525. https://doi.org/10.1016/j.intell.2007.11.001

Kempton, S., Vance, A., Maruff, P., Luk, E., Costin, J., & Pantelis, C. (1999). Executive function and attention deficit hyperactivity disorder: Stimulant medication and better executive performance in children. Psychological Medicine, 29(3), 527-538. https://doi.org/10.1017/S0033291799008338

Kessel, J. B., & Zimmerman, M. (1993). Reporting errors in studies of the diagnostic performance of self-administered questionnaires: Extent of the problem, recommendations for standardized presentation of results, and implications for the peer review process. Psychological Assessment, 5(4), 395-399. https://doi.org/10.1037/1040-3590.5.4.395

Koonce, D. A. (2007). Attention Deficit Hyperactivity Disorder assessment practices by practicing school psychologists. Journal of Psychoeducational Assessment, 25(4), 319-333. https://doi.org/10.1177/0734282906298264

Landau, S., Milich, R., & Widiger, T. A. (1991). Predictive power methods may be more helpful in making a diagnosis than sensitivity and specificity. Journal of Child and Adolescent Psychopharmacology, 1(5), 343-351. https://doi.org/10.1089/cap.1991.1.343

Lerner, J., & Yasutake, D. (2000). School strategies. In P. Accardo, T.A. Blondis, B.Y. Whitman, & M. Stein (Eds.), Attention Deficit Disorders and Hyperactivity in children and adults (2nd ed.) (pp. 461-476). Marcel Dekker, Inc.

Lezak, M. D. (1995). Neuropsychological Assessment, Third Edition. Oxford University Press.

Locke, S., McGrew, K. S., & Ford, L. (2011). A multiple group confirmatory factor analysis of the structural invariance of the Cattell–Horn– Carroll theory of cognitive abilities across matched Canadian and U.S. samples. WMF Press Bulletin, (1), 1–18.

McDermott, P. A., Marston, N. C., & Stott, D. H. (1993). Adjustment Scales for Children and Adolescents. Edumetric and Clinical Science.

McGill, R. J. (2015). Incremental criterion validity of the WJ-III COG clinical clusters: Marginal predictive effects beyond the general factor. Canadian Journal of School Psychology, 30(1), 51-63. https://doi.org/10.1177/0829573514553926

McQuade, J. D., Tomb, M., Hoza, B., Waschbusch, D. A., Hurt, E. A., & Vaughn, A. J. (2011). Cognitive deficits and positively biased self-perceptions in children with ADHD. Journal of abnormal child psychology39(2), 307-319. https://doi.org/10.1007/s10802-010-9453-7

Meehl, P. E., & Rosen, A. (1955). Antecedent probability and the efficiency of psychometric signs, patterns, or cutting scores. Psychological Bulletin, 52(3), 194-215. https://doi.org/10.1037/h0048070

Meyers, J. E., & Meyers, K. R. (1995). Rey Complex Figure Test and Recognition Trial: Professional manual. Psychological Assessment Resources.

Milich, R., Widiger, T. A., & Landau, D. (1987). Differential diagnosis of attention deficit and conduct disorders using conditional probabilities. Journal of Consulting and Clinical Psychology, 55(5), 762-767. https://doi.org/10.1037/0022-006X.55.5.762

Naglieri, J. A., & Das, J. P. (1997). Cognitive Assessment System. Riverside.

Naglieri, J. A., Das, J. P., & Goldstein, S. (2014). Cognitive Assessment System (2nd ed.). Pro-Ed.

Naglieri, J. A., Goldstein, S., Iseman, J. S., & Schwebach, A. (2003). Performance of children with attention deficit hyperactivity disorder and anxiety/depression on the WISC-III and Cognitive Assessment System. Journal of Psychoeducational Assessment, 21(1), 32-42. https://doi.org/10.1177/073428290302100103

Naglieri, J. A., Salter, C. J., & Edwards, G. H. (2004). Assessment of children with attention and reading difficulties using the PASS theory and Cognitive Assessment System. Journal of Psychoeducational Assessment, 22(2), 93-105. https://doi.org/10.1177/073428290402200201

Paolitto, A. W. (1999). Clinical validation of the Cognitive Assessment System for children with ADHD. ADHD Report, 7(1), 1-5.

Penny, A. M., Waschbusch, D. A., Carrey, N., & Drabman, R. S. (2005). Applying a psychoeducational perspective to ADHD. Journal of Attention Disorders, 8(4), 208-220. https://doi.org/10.1177/1087054705278746

Pottinger, L. S. (2002). Identifying AD/HD subtypes using the Cognitive Assessment System and the NEPSY [Unpublished doctoral dissertation]. University of North Texas.

Reale, L., Bartoli, B., Cartabia, M., Zanetti, M., Costantino, M. A., Canevini, M. P., Termine C., & Bonati, M. (2017). Comorbidity prevalence and treatment outcome in children and adolescents with ADHD. European child & adolescent psychiatry26, 1443-1457. https://doi.org/10.1007/s00787-017-1005-z

Reddy, L. A., Braunstein, D., & Dumont, R. (2008). Use of the Differential Ability Scales for children with ADHD. School Psychology Quarterly, 23(1), 139-148. https://psycnet.apa.org/doi/10.1037/1045-3830.23.1.139

Reddy, L. A., & Hale, J. B. (2007). Inattentiveness. In A. R. Eisen (Ed.), Treating childhood behavioral and emotional problems: A step-by-step evidence-based approach. (pp. 156-211). Guilford Publications, Inc.

Reddy, L. A., Hale, J. B., & Brodzinsky, L. K. (2011). Discriminant validity of the Behavior Rating Inventory of Executive Function Parent Form for children with Attention-Deficit/ Hyperactivity Disorder. School Psychology Quarterly, 26(1), 45-55. https://doi.org/10.1037/a0022585

Reddy, L. A., Weissman, A., & Hale, J. B. (2013). Neuropsychological assessment and intervention for emotional and behavior disordered youth: An integrated step-by-step evidence-based approach. American Psychological Association Press.

Rowland, J. E. (2013). Discriminative ability of CHC factor scores from the WJ III tests of cognitive abilities in children with ADHD [Doctoral dissertation, Oklahoma State University]. SHAREOK.

Salthouse, T. A. (2005). Relations between cognitive abilities and measures of executive functioning. Neuropsychology, 19(4), 532-545. https://doi.org/10.1037/0894-4105.19.4.532

Schrank, F. A., Flanagan, D. P., Woodcock, R. W., & Mascolo, J. T. (2002). Essentials of WJ III cognitive abilities assessment. Wiley.

Semrud-Clikeman, M., Filipek, P. A., Biederman, J., Steingard, R., Kennedy, D., Renshaw, P., & Bekken, K. (1994). Attention-deficit hyperactivity disorder: magnetic resonance imaging morphometric analysis of the corpus callosum. Journal of the American Academy of Child & Adolescent Psychiatry33(6), 875-881. https://doi.org/10.1097/00004583-199407000-00014

Spenceley, L. M., Wood, W. L., & Lovett, B. J. (2020). Using the Woodcock-Johnson IV tests of cognitive abilities to detect feigned ADHD. Applied Neuropsychology: Adult. Advance Online Publication. 1-9. https://doi.org/10.1080/23279095.2020.1748631

Tabachnick, B. G., & Fidell, L. S. (1996). Using multivariate statistics (3rd ed.). Harper Collins.

Tannock, R. (1998). Attention deficit hyperactivity disorder: Advances in cognitive, neurobiological, and genetic research. Journal of Child Psychology and Psychiatry, 39(1), 65-99. https://doi.org/10.1111/1469-7610.00304

Taub, G. E., Keith, T. Z., Floyd, R. G., & McGrew, K. S. (2008). Effects of general and broad cognitive abilities on mathematics achievement. School Psychology Quarterly23(2), 187-198. https://doi/10.1037/1045-3830.23.2.187

Taub, G. E., & McGrew, K. S. (2004). A confirmatory factor analysis of Cattell–Horn–Carroll theory and cross-age invariance of the Woodcock– Johnson Tests of Cognitive Abilities III. School Psychology Quarterly, 19(1), 72– 87. https://doi.org/10.1521/scpq.19.1.72.29409

Vanderwood, M. L., McGrew, K. S., Flanagan, D. P., & Keith, T. Z. (2002). The contribution of general and specific cognitive abilities to reading achievement. Learning and Individual Differences13(2), 159-188.  https://doi.org/10.1016/S1041-6080(02)00077-8

Van Luit, J. E. H., Kroesbergen, E. H., & Naglieri, J. A. (2005). Utility of the PASS theory and Cognitive Assessment System for Dutch children with and without ADHD. Journal of Learning Disabilities, 38(5), 434-439. https://doi.org/10.1177/00222194050380050601

Weis, R. (2017). Introduction to abnormal child and adolescent psychology. Sage Publications.

Woodcock, R. W., McGrew, K. S., & Mather, N. (2001a). Woodcock-Johnson III: Tests of Achievement. Riverside Publishing

Woodcock, R. W., McGrew, K. S., & Mather, N. (2001b). Woodcock-Johnson III: Tests of Cognitive Abilities. Riverside Publishing

...