The often-reported low informant agreement about child psychopathology between multiple informants has lead to various suggestions about how to address discrepant ratings. Among the factors that may lower agreement that have been discussed is informant credibility, reliability, or psychopathology, which is of interest in this paper. We tested three different models, namely, the accuracy, the distortion, and an integrated so-called combined model, that conceptualize parental ratings to assess child psychopathology. The data comprise ratings of child psychopathology from multiple informants (mother, therapist and kindergarten teacher) and ratings of maternal psychopathology. The children were patients in a preschool psychiatry unit (N=247). The results from structural equation modeling show that maternal ratings of child psychopathology were biased by maternal psychopathology (distortion model). Based on this statistical background, we suggest a method to adjust biased maternal ratings. We illustrate the maternal bias by comparing the ratings of mother to expert ratings (combined kindergarten teacher and therapist ratings) and show that the correction equation increases the agreement between maternal and expert ratings. We conclude that this approach may help to reduce misclassification of preschool children as ‘clinical’ on the basis of biased maternal ratings.
The assessment of child psychopathology and associated risk factors is important to achieve an understanding of emotional and behavioral problems in children (Goodman and Gotlib, 1999); however, this assessment also presents a methodological challenge because the validity of child psychopathology ratings from informants is controversial (Richters, 1992 and Gartstein et al., 2009). Currently, there are diverse recommendations for guiding clinicians in dealing with differing ratings from multiple informants in a single case (Smith, 2007 and Carlson and Youngstrom, 2011). Among these recommendations, three approaches seem to be particularly noteworthy. In the first approach, researchers consider each informant as a source of valid (and full) information, and consequently search for an algorithm to combine all ratings to improve predictability or to decide which informant's rating should be preferred in practice (e.g., Fitzmaurice et al., 1995 and Shemmassian and Lee, 2012). This approach includes the assumption that the validity of rating may depend on the specific situation (Rettew et al., 2011) or the specific disorder (Carlson and Blader, 2011).
The second approach basically puts the validity of informants in question and uses additionally collected information about an informant's characteristics, for example, their credibility (Youngstrom et al., 2011) or interviewers' impressions of the reliability of informants (De Los Reyes et al., 2011). Consequently, this approach aims to select a proportion of valid informants.
The third, and possibly most radical, approach is to dismiss the ideal of integrating differing informants' ratings and conceptualize the main criteria of child psychiatric disorders as situation- and informant-specific phenomena (Offord et al., 1996). With this approach, the widely shared basic assumption that psychiatric disorders are, to some extent, situation- and observer-independent diagnostic categories is given up. In other words, this approach no longer aims to achieve a shared understanding of child psychopathology and is therefore at risk for interpreting all facets of an assessment as phenomena-related true and valid information.
From a conceptual viewpoint, the presented approaches do not aim to incorporate different ratings into one situation-independent measure of child psychopathology because they do not integrate informants into one measurement model (such models are presented in paragraph 1.3). Instead, the approaches prefer to select a part of the data or use them as multiple criteria to hold the assumption of valid informants. This practice renders study results difficult to compare. Moreover, the three approaches cannot be concurrently evaluated because there are no shared golden criteria (i.e., an informant-independent measure of child psychopathology).
However, considered from a statistical viewpoint, the second approach may be particularly interesting, as it introduces moderator variables that can be used to weight the association between parental rating and child psychopathology by informant characteristics. This approach can be formalized in a structural equation model and allows the simultaneous inclusion of several informant characteristics to clarify whether, for example, the credibility, reliability, and parental psychopathology of informants are uncorrelated constructs. The structural equation model offers the opportunity to attribute variance to different causes. Parental psychopathology may make an especially large contribution to explaining disagreements in ratings of child psychopathology. Therefore, we will take a closer look at this issue before presenting our own approach, which is in essence a measurement model approach that includes parental psychopathology as a moderator variable.
Assessing child psychopathology using parental ratings implies assessing parental psychopathology. We have developed and evaluated methods to correct for distorted ratings from mothers with increased psychopathology, a technique that may enhance agreement with other informants. Alternatively, a completely different assessment method, such as direct behavioral observation, may be considered to obtain a valid estimation of a child's psychopathology.
