Construct Validity and Diagnostic Utility of the Woodcock Johnson Tests of Cognitive Abilities and Clinical Clusters for Children with Attention Deficit/ Hyperactivity Disorder: A Preliminary Investigation
The diagnostic utility of the Woodcock- Johnson III Tests of Cognitive Abilities and Clinical Clusters was assessed in a sample of 52 children (26 Att.
- Pub. date: June 15, 2021
- Pages: 37-49
- 516 Downloads
- 643 Views
- 0 Citations
The diagnostic utility of the Woodcock- Johnson III Tests of Cognitive Abilities and Clinical Clusters was assessed in a sample of 52 children (26 Attention Deficit/Hyperactivity Disordered (ADHD) and 26 matched controls). Multivariate analysis of variance followed by post-hoc testing and d-ratios yielded some statistically significant and clinically meaningful differences between groups on the Cognitive Fluency Cluster and the Tests of Auditory Attention, and Rapid Picture Naming. Discriminant function analyses indicated that the WJ III COG Tests collectively classified 80.77% of the sample correctly (76.92% of controls and 84.62% of children with ADHD correctly identified). The Auditory Attention and Rapid Picture Naming tests were found to make the most significant contribution overall to the discriminant function. Using a cut-score of 85, the WJ-III COG Clinical clusters and subtests examined in this study offered fair to weak diagnostic utility based on indices of sensitivity, specificity, positive and negative predictive power, as well as results of Receiver Operating Characteristic curve analyses. Implications for research and practice are outlined.
Keywords: Assessment, attention deficit hyperactivity disorder, Cattell-Horn-Carroll.
0
References
American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed. Text rev.). American Psychiatric Association.
Barkley, R. A. (1997). Behavioral inhibition, sustained attention, and executive functions: Constructing a unifying theory of ADHD. Psychological Bulletin, 121(1), 65-94. https://doi.org/10.1037/0033-2909.121.1.65
Barkley, R. A., & Murphy, K. R. (2006). Attention–deficit hyperactivity disorder: A clinical workbook (3rd ed.). The Guilford Press.
Bartolucci, M., & Batini, F. (2020). Reading aloud narrative material as a means for the student's cognitive empowerment. Mind, Brain, and Education, 14(3), 235-242. https://doi.org/10.1111/mbe.12241
Benson, N. F., Floyd, R. G., Kranzler, J. H., Eckert, T. L., Fefer, S. A., & Morgan, G. B. (2019). Test use and assessment practices of school psychologists in the United States: Findings from the 2017 national survey. Journal of School Psychology, 72, 29-48. https://doi.org/10.1016/j.jsp.2018.12.004
Björkenstam, E., Björkenstam, C., Jablonska, B., & Kosidou, K. (2018). Cumulative exposure to childhood adversity, and treated attention deficit/hyperactivity disorder: a cohort study of 543 650 adolescents and young adults in Sweden. Psychological medicine, 48(3), 498-507. https://doi.org/10.1017/S0033291717001933
Brown, N. M., Brown, S. N., Briggs, R. D., German, M., Belamarich, P. F., & Oyeku, S. O. (2017). Associations between adverse childhood experiences and ADHD diagnosis and severity. Academic pediatrics, 17(4), 349-355. https://doi.org/10.1016/j.acap.2016.08.013
Canivez, G. L., & Gaboury, A. R. (2016). Construct validity and diagnostic utility of the cognitive assessment system for ADHD. Journal of attention disorders, 20(6), 519-526. https://doi.org/10.1177/1087054713489021
Canivez, G. L., & Sprouls, K. (2005). Assessing the construct validity of the Adjustment Scales for Children and Adolescents. Journal of Psychoeducational Assessment, 23(1), 3-14. https://doi.org/10.1177/073428290502300101
Carper, M. S. (2003). A concurrent validity study comparing executive functioning of the Woodcock-Johnson III Tests of Cognitive Ability and the NEPSY [Unpublished doctoral dissertation]. Texas Woman’s University.
Castellanos, F. X., & Tannock, R. (2002). Neuroscience of attention-deficit/hyperactivity disorder: The search for endophenotypes. Nature Reviews Neuroscience, 3(8), 617-628.
Cohen, J. (1988). Statistical power analyses for the behavioral sciences (2nd ed.). Academic Press.
Crawford, E. N. (2002). Profiles for exceptional samples on the Cognitive Assessment System using configural frequency analysis [Unpublished doctoral dissertation]. University of Northern Colorado.
Das, J. P., Naglieri, J. A., & Kirby, J. R. (1994). Assessment of cognitive processes: The PASS theory of intelligence. Allyn & Bacon.
Delis, D. C., Kaplan, E., & Kramer, J. H. (2001). Delis Kaplan executive function system. The Psychological Corporation.
Dombrowski, S. C. (2013). Investigating the structure of the WJ-III Cognitive at school age. School Psychology Quarterly, 28(8), 154-169. https://doi.org/10.1037/spq0000010
Dombrowski, S. C. (2014). Exploratory bifactor analysis of the WJ-III Cognitive in adulthood via the Schmid-Leiman procedure. Journal of Psychoeducational Assessment, 32(4), 330 -341. https://doi.org/10.1177/0734282913508243
Dombrowski, S. C., & Watkins, M. W. (2013). Exploratory and higher order factor analysis of the WJ-III full test battery: A school-aged analysis. Psychological Assessment, 25(2), 442-455. https://doi.org/10.1037/a00311335
Doyle, A. E., Biederman, J., Seidman, L. J., Weber, W., & Faraone, S. V. (2000). Diagnostic efficiency of neuropsychological test scores for discriminating boys with and without attention deficit/hyperactivity disorder. Journal of Consulting and Clinical Psychology, 68(3), 477-488. https://doi.org/10.1037/0022-006X.68.3.477
Elliott, C. D. (1990). Differential Ability Scales. The Psychological Corporation.
Floyd, R. G., McCormack, A. C., Ingram, E. L., Davis, A. E., Bergeron, R., & Hamilton, G. (2006). Relations between the Woodcock-Johnson III clinical clusters and measures of executive functions from the Delis-Kaplan Executive Function System. Journal of Psychoeducational Assessment, 24(4), 303-317. https://doi.org/10.1177/0734282906287823
Floyd, R. G., McGrew, K. S., Barry, A., Rafael, F., & Rogers, J. (2009). General and specific effects on Cattell–Horn–Carroll broad ability composites: Analysis of the Woodcock–Johnson III normative update Cattell–Horn–Carroll factor clusters across development. School Psychology Review, 38(2), 249 –265. https://doi.org/10.1080/02796015.2009.12087836
Ford, L., Keith, T. Z., Floyd, R. G., Fields, C., & Schrank, F. A. (2003). Using the Woodcock-Johnson III Tests of Cognitive Abilities with students with attention deficit/hyperactivity disorder. In F. A. Schrank & D. P. Flanagan (Eds.), WJ III Clinical Use and Interpretation: Scientist-Practitioner Perspectives (pp. 320-342). Academic Press.
Ford-Jones, P. C. (2015). Misdiagnosis of attention deficit hyperactivity disorder: ‘Normal behaviour’and relative maturity. Pediatrics & child health, 20(4), 200-202. https://doi.org/10.1093/pch/20.4.200
Goldstein, S., & Naglieri, J. A. (2015). Using the CAS2 in the comprehensive assessment of ADHD. The ADHD Report, 23(7), 8-10. https://doi.org/10.1521/adhd.2015.23.7.8
Gray, J. A. (1994). Personality dimensions and emotion systems. In P. Ekman & R. J. Davidson (Eds.), The nature of emotion (pp.329-331). Oxford University Press.
Gyldrenkaerne, P., Dillion, H., Sharma, M., & Purdy, S. C. (2014). Attend to this: The relationship between auditory processing disorders and attention deficits. Journal of the American Academy of Audiology. 25(7), 676-687.
Hale, J. B., Reddy, L. A., Decker, S. L., Thompson, R., Henzel, J., Teodori, A., Forrest, E., Eusebio, E., & Denckla, M. B. (2009). Development and validation of an Attention-Deficit/ Hyperactivity Disorder (ADHD) executive function and behavior rating screening battery. Journal of Clinical and Experimental Neuropsychology, 31(8), 897-912. https://doi.org/10.1080/13803390802687423
Hale, J. B., Reddy, L. A., Wilcox, G., McLaughlin, A., Hain, L., Stern, A., Henzel, J., & Eusebio, E. (2010). Best practices in assessing and intervening with ADD/ADHD children and children with other frontal-striatal circuit disorders. In D. C. Miller (Ed.), Best practices in school neuropsychology: Guidelines for effective practice, assessment and evidence-based interventions. John Wiley & Sons.
Harrier, L., & DeOrnellas, K. (2005). Performance of children diagnosed with ADHD on selected planning and reconstitution tests. Applied Neuropsychology, 12(2), 106-119. https://doi.org/10.1207/s15324826an1202_6
Hart, J. S. (2020). A CHC theoretical approach to examining cognitive and Academic Deficits Among Students with ADHD Using a Three-Battery Approach (3999) [Doctoral dissertation, University of Nevada]. UNVL Theses, Dissertation, Professional Papers and Capstones. https://digitalscholarship.unlv.edu/thesesdissertations/3999
Heaton, R. K., Chelune, G. J., Talley, J. L., Kay, G. G., & Curtiss, G. (1993). Wisconsin Card Sorting Test manual: Revised and expanded. Psychological Assessment Resources.
Huberty, C. J. (1984). Issues in the use of discriminant analysis. Psychological Bulletin, 95(1), 156-171. https://doi.org/10.1037/0033-2909.95.1.156
EveryCRSReport. (2007). Individuals with Disabilities Education Act of 2004, P.L. 108-446. EveryCRSReport. https://www.everycrsreport.com/reports/RL32716.html
Keith, T. Z., & Reynolds, M. R. (2012). Using confirmatory factor analysis to aid in understanding the constructs measured by intelligence tests. In D. P. Flanagan & P. L. Harrison (Eds.), Contemporary intellectual assessment: Theories, tests, and issues (3rd ed., pp. 758 –799). Guilford Press.
Keith, T. Z., Reynolds, M. R., Patel, P. G., & Ridley, K. P. (2008). Sex differences in latent cognitive abilities ages 6 to 59: Evidence from the Woodcock–Johnson III tests of cognitive abilities. Intelligence, 36(6), 502–525. https://doi.org/10.1016/j.intell.2007.11.001
Kempton, S., Vance, A., Maruff, P., Luk, E., Costin, J., & Pantelis, C. (1999). Executive function and attention deficit hyperactivity disorder: Stimulant medication and better executive performance in children. Psychological Medicine, 29(3), 527-538. https://doi.org/10.1017/S0033291799008338
Kessel, J. B., & Zimmerman, M. (1993). Reporting errors in studies of the diagnostic performance of self-administered questionnaires: Extent of the problem, recommendations for standardized presentation of results, and implications for the peer review process. Psychological Assessment, 5(4), 395-399. https://doi.org/10.1037/1040-3590.5.4.395
Koonce, D. A. (2007). Attention Deficit Hyperactivity Disorder assessment practices by practicing school psychologists. Journal of Psychoeducational Assessment, 25(4), 319-333. https://doi.org/10.1177/0734282906298264
Landau, S., Milich, R., & Widiger, T. A. (1991). Predictive power methods may be more helpful in making a diagnosis than sensitivity and specificity. Journal of Child and Adolescent Psychopharmacology, 1(5), 343-351. https://doi.org/10.1089/cap.1991.1.343
Lerner, J., & Yasutake, D. (2000). School strategies. In P. Accardo, T.A. Blondis, B.Y. Whitman, & M. Stein (Eds.), Attention Deficit Disorders and Hyperactivity in children and adults (2nd ed.) (pp. 461-476). Marcel Dekker, Inc.
Lezak, M. D. (1995). Neuropsychological Assessment, Third Edition. Oxford University Press.
Locke, S., McGrew, K. S., & Ford, L. (2011). A multiple group confirmatory factor analysis of the structural invariance of the Cattell–Horn– Carroll theory of cognitive abilities across matched Canadian and U.S. samples. WMF Press Bulletin, (1), 1–18.
McDermott, P. A., Marston, N. C., & Stott, D. H. (1993). Adjustment Scales for Children and Adolescents. Edumetric and Clinical Science.
McGill, R. J. (2015). Incremental criterion validity of the WJ-III COG clinical clusters: Marginal predictive effects beyond the general factor. Canadian Journal of School Psychology, 30(1), 51-63. https://doi.org/10.1177/0829573514553926
McQuade, J. D., Tomb, M., Hoza, B., Waschbusch, D. A., Hurt, E. A., & Vaughn, A. J. (2011). Cognitive deficits and positively biased self-perceptions in children with ADHD. Journal of abnormal child psychology, 39(2), 307-319. https://doi.org/10.1007/s10802-010-9453-7
Meehl, P. E., & Rosen, A. (1955). Antecedent probability and the efficiency of psychometric signs, patterns, or cutting scores. Psychological Bulletin, 52(3), 194-215. https://doi.org/10.1037/h0048070
Meyers, J. E., & Meyers, K. R. (1995). Rey Complex Figure Test and Recognition Trial: Professional manual. Psychological Assessment Resources.
Milich, R., Widiger, T. A., & Landau, D. (1987). Differential diagnosis of attention deficit and conduct disorders using conditional probabilities. Journal of Consulting and Clinical Psychology, 55(5), 762-767. https://doi.org/10.1037/0022-006X.55.5.762
Naglieri, J. A., & Das, J. P. (1997). Cognitive Assessment System. Riverside.
Naglieri, J. A., Das, J. P., & Goldstein, S. (2014). Cognitive Assessment System (2nd ed.). Pro-Ed.
Naglieri, J. A., Goldstein, S., Iseman, J. S., & Schwebach, A. (2003). Performance of children with attention deficit hyperactivity disorder and anxiety/depression on the WISC-III and Cognitive Assessment System. Journal of Psychoeducational Assessment, 21(1), 32-42. https://doi.org/10.1177/073428290302100103
Naglieri, J. A., Salter, C. J., & Edwards, G. H. (2004). Assessment of children with attention and reading difficulties using the PASS theory and Cognitive Assessment System. Journal of Psychoeducational Assessment, 22(2), 93-105. https://doi.org/10.1177/073428290402200201
Paolitto, A. W. (1999). Clinical validation of the Cognitive Assessment System for children with ADHD. ADHD Report, 7(1), 1-5.
Penny, A. M., Waschbusch, D. A., Carrey, N., & Drabman, R. S. (2005). Applying a psychoeducational perspective to ADHD. Journal of Attention Disorders, 8(4), 208-220. https://doi.org/10.1177/1087054705278746
Pottinger, L. S. (2002). Identifying AD/HD subtypes using the Cognitive Assessment System and the NEPSY [Unpublished doctoral dissertation]. University of North Texas.
Reale, L., Bartoli, B., Cartabia, M., Zanetti, M., Costantino, M. A., Canevini, M. P., Termine C., & Bonati, M. (2017). Comorbidity prevalence and treatment outcome in children and adolescents with ADHD. European child & adolescent psychiatry, 26, 1443-1457. https://doi.org/10.1007/s00787-017-1005-z
Reddy, L. A., Braunstein, D., & Dumont, R. (2008). Use of the Differential Ability Scales for children with ADHD. School Psychology Quarterly, 23(1), 139-148. https://psycnet.apa.org/doi/10.1037/1045-3830.23.1.139
Reddy, L. A., & Hale, J. B. (2007). Inattentiveness. In A. R. Eisen (Ed.), Treating childhood behavioral and emotional problems: A step-by-step evidence-based approach. (pp. 156-211). Guilford Publications, Inc.
Reddy, L. A., Hale, J. B., & Brodzinsky, L. K. (2011). Discriminant validity of the Behavior Rating Inventory of Executive Function Parent Form for children with Attention-Deficit/ Hyperactivity Disorder. School Psychology Quarterly, 26(1), 45-55. https://doi.org/10.1037/a0022585
Reddy, L. A., Weissman, A., & Hale, J. B. (2013). Neuropsychological assessment and intervention for emotional and behavior disordered youth: An integrated step-by-step evidence-based approach. American Psychological Association Press.
Rowland, J. E. (2013). Discriminative ability of CHC factor scores from the WJ III tests of cognitive abilities in children with ADHD [Doctoral dissertation, Oklahoma State University]. SHAREOK.
Salthouse, T. A. (2005). Relations between cognitive abilities and measures of executive functioning. Neuropsychology, 19(4), 532-545. https://doi.org/10.1037/0894-4105.19.4.532
Schrank, F. A., Flanagan, D. P., Woodcock, R. W., & Mascolo, J. T. (2002). Essentials of WJ III cognitive abilities assessment. Wiley.
Semrud-Clikeman, M., Filipek, P. A., Biederman, J., Steingard, R., Kennedy, D., Renshaw, P., & Bekken, K. (1994). Attention-deficit hyperactivity disorder: magnetic resonance imaging morphometric analysis of the corpus callosum. Journal of the American Academy of Child & Adolescent Psychiatry, 33(6), 875-881. https://doi.org/10.1097/00004583-199407000-00014
Spenceley, L. M., Wood, W. L., & Lovett, B. J. (2020). Using the Woodcock-Johnson IV tests of cognitive abilities to detect feigned ADHD. Applied Neuropsychology: Adult. Advance Online Publication. 1-9. https://doi.org/10.1080/23279095.2020.1748631
Tabachnick, B. G., & Fidell, L. S. (1996). Using multivariate statistics (3rd ed.). Harper Collins.
Tannock, R. (1998). Attention deficit hyperactivity disorder: Advances in cognitive, neurobiological, and genetic research. Journal of Child Psychology and Psychiatry, 39(1), 65-99. https://doi.org/10.1111/1469-7610.00304
Taub, G. E., Keith, T. Z., Floyd, R. G., & McGrew, K. S. (2008). Effects of general and broad cognitive abilities on mathematics achievement. School Psychology Quarterly, 23(2), 187-198. https://doi/10.1037/1045-3830.23.2.187
Taub, G. E., & McGrew, K. S. (2004). A confirmatory factor analysis of Cattell–Horn–Carroll theory and cross-age invariance of the Woodcock– Johnson Tests of Cognitive Abilities III. School Psychology Quarterly, 19(1), 72– 87. https://doi.org/10.1521/scpq.19.1.72.29409
Vanderwood, M. L., McGrew, K. S., Flanagan, D. P., & Keith, T. Z. (2002). The contribution of general and specific cognitive abilities to reading achievement. Learning and Individual Differences, 13(2), 159-188. https://doi.org/10.1016/S1041-6080(02)00077-8
Van Luit, J. E. H., Kroesbergen, E. H., & Naglieri, J. A. (2005). Utility of the PASS theory and Cognitive Assessment System for Dutch children with and without ADHD. Journal of Learning Disabilities, 38(5), 434-439. https://doi.org/10.1177/00222194050380050601
Weis, R. (2017). Introduction to abnormal child and adolescent psychology. Sage Publications.
Woodcock, R. W., McGrew, K. S., & Mather, N. (2001a). Woodcock-Johnson III: Tests of Achievement. Riverside Publishing
Woodcock, R. W., McGrew, K. S., & Mather, N. (2001b). Woodcock-Johnson III: Tests of Cognitive Abilities. Riverside Publishing