Ratings of Relations Between DSM-IV Diagnostic Categories and Items of the CBCL/1½-5/LDS and C-TRF

Thomas M. Achenbach & Levent Dumenci 
University of Vermont 

Leslie A. Rescorla 
Bryn Mawr College

Abstract

The purpose of this project was to determine the degree to which experienced mental health professionals would judge particular CBCL/1½-5/LDS and C-TRF items as being consistent with particular DSM-IV categories. Child psychiatrists and psychologists who had published research on children’s behavioral/emotional problems rated the consistency of each CBCL and C-TRF problem item with DSM categories that are potentially relevant to preschoolers. The 16 raters came from 10 cultures. Items that were rated by at least 10 raters as being very consistent with a diagnostic category were assigned to that category. After combining some categories that had overlaps in DSM criteria and item ratings, we constructed scales for the following categories: Affective Problems (including Dysthymia and Major Depression); Anxiety Problems (including GAD, SAD, and Specific Phobia); Pervasive Developmental Problems (including Asperger’s and Autistic Disorders); Attention Deficit/Hyperactivity Problems (including Hyperactive-Impulsive and Inattentive types); and Oppositional Defiant Problems. For each instrument, a DSM-oriented scale comprises the items from that instrument which were rated as being very consistent with the respective diagnostic category. The scales are displayed on profiles for scoring children in relation to normative samples of peers. The profiles show raw scale scores (sum of the 0-1-2 ratings of items comprising a scale); T scores; percentiles; and cutpoints for normal, borderline, and clinical ranges. Windows software for scoring the profiles can provide comparisons among DSM-oriented scale scores obtained from up to 8 CBCL/1½-5/LDS and C-TRF forms per child.

Introduction

Questions often arise about relations between formal diagnostic systems, such as the DSM, and empirically based instruments, such as the Child Behavior Checklist for Ages 1½-5 (CBCL/1½-5/LDS) and Caregiver-Teacher Report Form (C-TRF). Studies have shown significant associations between DSM diagnoses and scores on empirically based syndrome scales (e.g., Edelbrock & Costello, 1988; Kasius, Ferdinand, van den Berg, & Verhulst, 1997). However, the specific criteria for DSM diagnoses differ from the items of the empirically based scales. Furthermore, the associations that are found between diagnoses and scale scores may vary according to the training and orientation of the diagnosticians, the diagnostic procedures, ages of the children, sources of data, and other factors.

Purpose of this Project

The purpose of this project was to determine the degree to which experienced mental health professionals would judge particular CBCL/1½-5/LDS and C-TRF items as being consistent with particular DSM-IV diagnostic categories (American Psychiatric Association, 1994). To obtain sophisticated judgments, we asked highly experienced child psychiatrists and psychologists who had published research on children’s behavioral/emotional problems to rate the consistency of each CBCL/1½-5/LDS and C-TRF problem item with DSM diagnostic categories of behavioral/emotional disorders that are potentially relevant to preschoolers. If ratings of CBCL/1½-5/LDS and C-TRF items were found to be consistent, the ratings would be used to construct DSM-oriented scales for scoring the CBCL/1½-5/LDS and C-TRF. The DSM-oriented scales would accompany new versions of our empirically based syndrome scales for scoring these instruments.

Method

We identified the following DSM-IV diagnostic categories that are defined largely in terms of behavioral/emotional problems and that are potentially applicable to preschoolers: Asperger’s Disorder, Autistic Disorder, Attention-Deficit/Hyperactivity Disorder (ADHD), Hyperactive-Impulsive and Inattentive types, Dysthymia, Generalized Anxiety Disorder (GAD), Major Depressive Disorder, Oppositional Defiant Disorder (ODD), Separation Anxiety Disorder (SAD), Specific Phobia. Because the criteria for Asperger’s Disorder and Autistic Disorder are identical except for communication skills, we combined these two disorders into a single category. We then did the following:

  1. We carried out a pilot study in which child psychiatrists, psychologists, and trainees were asked to rate the CBCL/1½-5/LDS and C-TRF items for consistency with the DSM categories and to provide feedback on the procedures.
  2. We revised the instructions and rating forms on the basis of the results and feedback from participants in the pilot study.
  3. We invited participation by experienced child psychiatrists and psychologists who had published research on children’s behavioral/emotional problems.
  4. Those who agreed to participate were sent the following materials:
    1. Copies of the criteria for the nine DSM-IV diagnostic categories.
    2. The instructions that are presented in Appendix A.
    3. Rating forms on which the 116 items of the CBCL/1½-5/LDS and C-TRF were listed. (The two instruments have 82 problem items in common, plus 17 problem items that are specific to each instrument, for a total of 82 + 17 + 17 = 116 items.) For each of the nine DSM-IV categories, raters were asked to score each of the 116 problem items as 0 = not consistent, 1 = somewhat consistent, and 2 = very consistent with the DSM category.

Appendix B lists the 16 child psychiatrists and psychologists from ten cultures who submitted ratings.

Results

We based our selection of items for DSM-oriented scales on a criterion of at least 10 raters out of 16 (63%) scoring an item 2 (very consistent)with a diagnostic category. We used a criterion of >10 ratings of 2because it was high enough to require considerable agreement among raters, while still allowing for the effects of differences among the raters in culture, professional training, theoretical orientation, and the kinds of children served. Because of overlaps in the DSM criteria and in the items found to meet the criterion of >10 ratings of 2, the following diagnoses were combined into single categories: Affective Disorders: Dysthymia, Major Depression; Anxiety Disorders: GAD, SAD, Specific Phobia; PDD: Asperger’s Disorder, Autistic Disorder; ADHD: Hyperactive-Impulsive and Inattentive types. ODD remained a separate category.

Table 1 lists abbreviated versions of the CBCL/1½-5/LDS and C-TRF items that met the criterion of >10 ratings of 2. The only item that met this criterion for more than one of the final five DSM categories was item 71. Shows little interest in things around him/her, which was scored 2 by 10 raters for both the Affective and PDD categories. To avoid inflating correlations between these two categories, this item was assigned only to the Affective category, because only one rater scored it 0 for that category, whereas two raters scored it 0 for the PDD category.

Discussion

The five DSM-oriented scales are displayed on CBCL/1½-5/LDS and C-TRF hand-scored and computer-scored profiles analogous to the profiles for the empirically based scales (Achenbach & Rescorla, 2000). Normative distributions of scores, percentiles, and T scores are based on a new U.S. national sample. The scale scores are not equivalent to diagnoses. Instead, to make DSM diagnoses, users should apply the DSM criteria with respect to specific symptoms, impairment, age of onset, duration, and exclusion of other disorders.

Empirically based and DSM-oriented scales scored by the same respondents for the same children on the same pool of items can facilitate assessment that takes account of both the patterns of co-occurring problems reflected in the empirically based syndromes and groupings of problems that are consistent with DSM diagnostic categories. Both types of scales can be quantitatively scored in terms of normed, gender-specific, standard scores, and also in terms of absolute levels of problems. This offers many possibilities for comparing and combining the empirically based and DSM-oriented scales for assessment of initial problems, evaluation of outcomes and differential treatment efficacy, epidemiological studies, genetic research, cross-cultural comparisons, and testing of correlates of psychopathology. TheManual for the ASEBA Preschool Forms & Profiles (Achenbach & Rescorla, 2000) provides further details of the development and application of the empirically based and DSM-oriented scales.

References

Achenbach, T.M., & Rescorla, L.A. (2000). Manual for the ASEBA Preschool Forms & Profiles. Burlington, VT: University of Vermont Department of Psychiatry.

American Psychiatric Association. (1994). Diagnostic & statistical manual of mental disorders (4th ed.). Washington, D.C.: American Psychiatric Association.

Edelbrock, C., & Costello, A.J. (1988). Convergence between statistically derived behavior problem syndromes and child psychiatric diagnoses. Journal of Abnormal Child Psychology,16, 219-231.

Kasius, M.C., Ferdinand, R.F., van den Berg, H., & Verhulst, F.C. (1997). Associations between different diagnostic approaches for child and adolescent psychopathology. Journal of Child Psychology & Psychiatry, 38, 625-632.

Table 1

CBCL/1½-5/LDS and C-TRF Items Rated by > 10/16 Experts as Very Consistent with DSM-IV Categories

Affective Disorders
Anxiety Disorders
PDD
ADHD
ODD
13. Cries 10. Clings 3. Afraid to try new 5. Can’t concentrate 15. Defiant
43. Looks unhappy 32. Fears 4. Avoids eye contact 6. Can’t sit still 20. Disobedient
50. Overtired 37. Upset by separation 7. Can’t stand things out of place 8. Can’t stand waiting 44. Angry moods
71. Little interest 47. Nervous 21. Disturbed by change 16. Demands must be met 81. Stubborn
89. Underactive 87. Fearful 23. Doesn’t answer 36. Gets into everything 85. Temper
90. Sad 99. Worries 25. Doesn’t get along with peers 59. Quickly shifts 88. Uncooperative
    63. Rocks head, body    
CBCL Only
CBCL Only
67. Unresponsive to affection
C-TRF Only
C-TRF Only
24. Doesn’t eat well 22. Doesn’t want to sleep alone 70. Little affection 19. Daydreams 22. Mean
38. Trouble sleeping 28. Doesn’t leave home 76. Speech problem 24. Difficulty with directions  
49. Overeating 48. Nightmares 80. Strange behavior 28. Disturbs others  
74. Sleeps little 51. Panics 92. Upset by new 48. Fails to carry out  
    98. Withdrawn 51. Fidgets  
C-TRF Only
C-TRF Only
  64. Inattentive  
12. Apathetic 49. Fears daycare/school   75. Overactive  

Note: Items are designated in the table with the numbers they bear on the CBCL/1½-5/LDS and C-TRF and summaries of their content.

Appendix A

Instructions to DSM Raters:
Rating the Diagnostic Consistency of Specific Problems with DSM-IV Categories

Purpose

To determine whether problems listed on the CBCL/1½-5/LDS and C-TRF are diagnostically consistent with DSM disorders that might be found at ages 18 months through 71 months. Accompanying this instruction sheet are:

  1. DSM-IV criteria for disorders that might be found at ages 18-71 months. In practice, you might not use some of the DSM diagnoses for children of these ages. However, we wish to see which problem items might be diagnostically consistent with the symptom criteria for the disorders, even if other factors argue against actually using the DSM diagnoses for ages 18-71 months.
  2. A list of the problem items with spaces for rating the diagnostic consistency of each item with each DSM category. The number beside the item is the number that it bears on the CBCL/1½-5/LDS and/or C-TRF. The first items are on the CBCL/1½-5/LDS or both forms, followed by items that are only on the C-TRF.

Instructions

If you are willing, please follow these steps:

  1. Starting with the first problem on the list, consider its consistency with the first category of disorders, which includes Asperger and Autistic disorders. Consult the accompanying DSM-IV criteria for Asperger and Autistic disorders.
  2. Decide whether you think the first problem is diagnostically consistent with either Asperger or Autistic disorder or both.
    1. Please use the DSM-IV symptom criteria as a basis for deciding whether a problem is consistent with a category. 
    2. You may feel that some problem items are appropriate diagnostic indicators of particular disorders, but that they do not have precise counterparts among the DSM-IV symptom criteria. Feel free to rate these problem items as being consistent with the categories, according to the scoring rules listed in #3.
    3. The DSM-IV lists the same criteria for Asperger and Autistic disorders, except that qualitative impairments in communication are listed only for Autistic disorder. If you feel that any problem items are diagnostically consistent with the impairments in communication listed for Autistic disorder, rate these items as diagnostically consistent with the Asperger-Autistic category. 
  3. After you decide whether the first problem is consistent with Asperger or Autistic disorder, please rate how consistent the problem is with the Asperger-Autistic category, as follows:0 = Not consistent with the category.
    1 = Somewhat consistent with the category.
    2 = Very consistent with the category.
  4. After you have rated the consistency of the first problem item with the Asperger- Autistic category, rate the consistency of each other problem item with each category specified on the rating form. You may prefer to rate the first item for all categories before proceeding to the second item, i.e., work from left to right. Or you may prefer to rate all items for the first category before rating any items for the second category, i.e., proceed from top to bottom.
  5. Feel free to rate an item 0, 1, or 2 for any category, regardless of the ratings you give that item for the other categories. For example, you can give an item a rating of 0 for three categories, 1 for four categories, and 2 for two categories. In other words, do not spend time choosing a single category for your highest rating of an item. Instead, just consider each category alone when rating each problem item. You may decide that some problem items should be rated 0 for all categories, whereas other problem items should be rated 2 for several categories. 
  6. After you have finished your ratings, please indicate whether you would like a report of the findings, and e-mail the rating form to Thomas.Achenbach@uvm.edu or fax it to 802-656-5131. We will then mail your check for $50. 

Thanks very much for your help.


Appendix B

Child Psychiatrists and Psychologists Who Rated CBCL/1½-5/LDS and C-TRF Items for Consistency With DSM-IV Categories 

Country or Cultural Background 

Australia

Sawyer, Michael, MD., Ph.D.
Associate Professor
Research & Evaluation Unit
Women’s and Children’s Hospital
72 King William Road
North Adelaide
South Australia 5006

China 

Leung, Patrick, Ph.D.
Associate Professor
Department of Psychology
3/F, Sino Building
The Chinese University of Hong Kong
Shatin, New Territories, Hong Kong

Germany

Heubeck, Bernd, Dipl. Psych.
Senior Lecturer in Clinical Psychology
Division of Psychology
The Australian National University
Canberra, ACT 0200

Greece

Roussos, Alexandra, M.D.
Attiki Child Psychiatry Hospital
4 Garefi Street
11525 Athens, Greece

Israel

Auerbach, Judith, Ph.D.
Senior Lecturer
Department of Behavioral Sciences
Ben-Gurion University
Beer Sheva 84105

Jamaica

Lambert, Michael, Ph.D.
Associate Professor
Michigan State University
East Lansing, MI 38824-1117

Netherlands

Ferdinand, Robert, M.D., Ph.D., Director
Outpatient Child and Adolescent Psychiatry
Sophia Children’s Hospital
Erasmus University
Dr. Molewaterplein 60
3015 GJ Rotterdam<

Verhulst, Frank, M.D., Ph.D.
Professor and Director
Department of Child Psychiatry
Sophia Children’s Hospital
Erasmus University
Dr. Molewaterplein 60
3015 GJ Rotterdam

Puerto Rico

Bird, Hector, M.D., Professor
Columbia University and NY State Psychiatric Institute
1051 Riverside Drive, Unit 78
New York City, NY 10032

Turkey

Erol, Nese, Ph.D., Professor
Ankara University
Faculty of Medicine
Child Psychiatry Department
Dikimevi, Ankara Turkey

USA

Campbell, Susan, Ph.D., Professor
Department of Psychology
University of Pittsburgh
4015 O’Hara Street
Pittsburgh, PA 15260

Carlson, Gabrielle, M.D., Professor
Department of Psychiatry
Putnam Hall
SUNY Stony Brook
Stony Brook, NY 11794-8790

Gordon, Michael, Ph.D., Professor
Department of Psychiatry
SUNY Upstate Medical University
750 East Adams Street
Syracuse, NY 13210

Phares, Vicky, Ph.D., Associate Professor
University of South Florida
Department of Psychology
4202 E. Fowler Avenue, BEH 339
Tampa, FL 33620

Shaw, Daniel, Ph.D., Associate Professor
Department of Psychology
University of Pittsburgh
4015 O’Hara Street, 604 OEH
Pittsburgh, PA 15260-0001

Speltz, Matthew, Ph.D., Professor
Psychiatry & Behavioral Sciences
University of Washington School of Medicine
Clinical Director, Child & Adolescent Outpatient Psychiatry Clinic
Children’s Hospital & Regional Medical Center
Seattle, WA 98105-0371