Stop using the Adverse Childhood Experiences Checklist to make claims about trauma causing physical and mental health problems

Scores on the adverse childhood experiences (ACE) checklist (or ACC) are widely used in making claims about the causal influence of childhood trauma on mental and physical health problems. Does anyone making these claims bother to look at how the checklist is put together and consider what a summary score might mean?

 

mind the brain logo

Scores on the adverse childhood experiences (ACE) checklist (or ACC) are widely used in making claims about the causal influence of childhood trauma on mental and physical health problems. Does anyone making these claims bother to look at how the checklist is put together and consider what a summary score might mean?

In this issue of Mind the Brain, we begin taking a skeptical look at the ACE checklist. We ponder some of the assumptions implicit in what items were included and how summary scores of the number of items checked are interpreted. Readers will be left with profound doubts that the ACE is suitable for making claims about trauma.

This blog will eventually be followed by another that presents the case that scores on the ACC do not represent a risk factor for health problems, only a relatively uninformative risk marker. In contrast to potentially modifiable risk factors, risk markers are best interpreted as calling attention to the influence of some combination of other risk factors, many of as yet unspecified, but undoubtedly of an entirely different nature than what is being studied. What?!! You will have to stay tuned, but I’ll give some hints about what I am talking about in the current blog post.

Summary of key points

 The ACE checklist is a collection of very diverse and ambiguous items that cannot be presumed to necessarily represent traumatic experiences.

Items variously

  • Represent circumstances that are not typically traumatic.
  • Reflect the respondent’s past or current psychopathology.
  • Make equivalent and traumatic vastly different experiences, many neutral and some that are positive.
  • Reinterpret a personal vulnerability due to familial transmission of psychopathology, either direct or indirect, rather than simply an exposure to events.
  • Ignore crucial contextual information, including timing of events.

There is reason not to assume that higher summed scores for the ACE represent more exposure to trauma than lower scores.

Are professionals misinterpreting the ACE checklist just careless or are they ideologues selectively identifying “evidence” for their positions which don’t depend on evidence at all?

ace-7Witness claims based on research with the ACE that migraines are caused by sexual abuse   and that psychotherapy addressing that abuse should be first line treatment. Or claims that childhood trauma is as strong a risk factor for psychosis and schizophrenia as smoking is for lung cancer [* ] and so psychotherapy is equivalent to medication in its effects. Or claims that myalgic encephalomyelitis, formerly known as chronic fatigue syndrome, is caused by childhood trauma and the psychological treatments can be recommended as the treatment of choice. These claims share a speculative, vague neo-cryptic pseudopsychoanalytic set of assumptions that is seldom articulated or explicitly confronted with evidence. Authors typically leap from claims about childhood trauma causing later problems to non sequitur claims about the efficacy of psychological intervention in treating these problems by addressing trauma. These claims about efficacy of trauma-focused treatment are not borne out in actually examining effects observed in randomized controlled trials.

Rather than attempting to address a provocative question about investigator motivation without a ready way of answering it, I will show most claims about trauma causing mental and physical health problems are, at best, based on very weak evidence, if they depend solely on the ACE checklist.

I will leave for my readers to decide if some authors who make such a fuss about the ACE have bothered to look at the instrument or care that is so inappropriate for the purposes to which they put it.

The ACE is reproduced at the bottom of this post and it is a good idea to compare what I’m saying about it to the actual checklist.

e5fc302ac1fabf0757e62a935b27800d
What “science” is behind such speculations?

The ACE was originally intended for educational purposes, not as a scientific instrument. Perhaps that explains its gross deficiencies as a key measure of psychological and epidemiological constructs.

The ACE checklist is a collection of very different and ambiguous items that cannot be presumed to represent traumatic experiences.

The ACE consists of ten dichotomous items for which the respondent is asked to indicate no/yes whether an experience occurred before the age of 18.  However, for six of the 10 items, the respondent is given further choices  that often differ greatly in the kind of experience to which the items refer. Scoring of the instrument does not take which of these experiences is the basis of a response. For example,

5. Did you often feel that … You didn’t have enough to eat, had to wear dirty clothes, and had no one to protect you? or

Your parents were too drunk or high to take care of you or take you to the doctor if you needed it?

Yes   No     If yes enter 1     ________

This item treats some very different circumstances as equivalent. The first half is complex, but largely covers the experience of living in poverty, but combines that with “having no one to protect you.” In contrast, the second half refers to substance abuse on the part of parents. In neither case, is there any room for interpreting what mitigating circumstances in the respondent’s life might have influenced effects of exposure. Presumably, the timing of this exposure would be important. If the exposure only occurred at the end of the 18 year period covered by the checklist, effects could be mitigated by other individual and social resources the respondent had.

Single items that are added together in a summary score.  We have to ask whether there is an equivalency between the two halves of the item that will be treated as the same. This will be an accumulating concern as we go through the 10 item questionnaire

The items vary greatly in the likelihood that they refer to an experience that was traumatic. Seldom do any of the researchers who use the ACE explain what they mean by trauma. If they did, I doubt that they could make a good argument that in endorsing many of these items would indicate that a respondent had faced a trauma.

From the third edition of the American Psychiatric Association Diagnostic and Statistical Manual (DSM-III) onward to DSM-5, the assumption has been that a traumatic event is a catastrophic stressor outside the range of usual human experience.

With that criteria in mind we have to ask if items are likely to represent a traumatic experience for most people. In answering this question, we also have to ask how we willing to consider a particular item is equivalent to other items in arriving at an overall score reflecting exposure to trauma before age 18. Yet, if summary scores are to be meaningful, assumption has to be made that items contribute equally if they are endorsed

6. Were your parents ever separated or divorced?

Yes   No     If yes enter 1     ________

The item refers to a highly prevalent and complex event, the nature and consequences of which are likely to unfold over time. Importantly, we need a sense of context to judge whether the event is traumatic and, if so how severe. Presumably, it would matter greatly when, across the 18 year span, the event that occurred. No timing or other information is asked of the respondent, only whether or not this event occurred. Neither the respondent nor anyone interpreting a score on the inventory has further information as to what is meant.

Other problems with ambiguous items.

Questions can be raised about the validity of all the individual items and the wisdom of combining them as equivalent in creating a summary score.

Items 1 and 2: Items raise questions about what role the respondent played eliciting the event.

 Did an event simply befall a respondent? Was it related to some pre-existing characteristic of the respondent? Or did the respondent have an active role in generating the event?

Did a parent or other adult member of the household often…

Swear at you, insult you, put you down, or humiliate you?

or

Act in a way that made you afraid that you might be physically hurt?

Yes   No     If yes enter 1     ________

And

Did a parent or other adult in the household often …

Push, grab, slap, or throw something at you?

or

Ever hit you so hard that you had marks or were injured?

Yes   No     If yes enter 1     ________

 Here, as throughout the rest of the checklist, questions can be raised about whether these items refer simply to an environmental exposure in epidemiological terms, say, equivalent to asbestos or tobacco. We don’t know the frequency, intensity or context of a the behavior in question, all of which may be crucial in evaluating whether a trauma occurred. For instance, it matters greatly if the behavior happened frequently when the respondent as a toddler or was limited to a struggle that occurred when the respondent was a teen high on drugs  attempting to take the car keys and go for a after midnight drive.

Like most of the rest of the questionnaire, there is the question of timing.

Item 3: There is so much ambiguity in endorsments of (ostensible) sexual abuse. Maybe it was a positive, liberating experience.

This is a crucial item and discussions of the ACE often assume that it is endorsed and represents a traumatic experience:

Did an adult or person at least 5 years older than you ever…

Touch or fondle you or have you touch their body in a sexual way?

or

Try to or actually have oral, anal, or vaginal sex with you?

Note that this is a complex item for which endorsement could be on the basis of a single instance of a person at least 5 years older touching or fondling the respondent. What if the presumed “perpetrator” is the 20 year old boyfriend or girlfriend of a 14 year old?

Are we willing to treat as equivalent “touch” or ‘fondle you” and “having anal sex” in all instances?

Arguably, the event which construed as trauma could actually be quite positive, as in the respondent  forming a secure attachment with a somewhat older, but nonetheless appropriate partner. All that is unconventional is not traumatic. What if the respondent and  alleged “perpetrator” were in a deeply intimate relationship or already married?

The research that attempts to link endorsement of such an item to lasting mental and physical health problems is remarkably contradictory and inconsistent 

Item 4:  Does this  item reflect the respondent’s serious clinical depression or other mental disorder before age 18 or currently, when the checklist is being completed?

Did you often feel that …  No one in your family loved you or thought you were important or special?    or

Your family didn’t look out for each other, feel close to each other, or support each other?

Yes   No     If yes enter 1     ________

As elsewhere in the checklist, there is no place for the respondent or someone interpreting a “yes” response for taking into account timing or contextual factors that might mitigate or compound effects of this “exposure.”

Item 5: Is this a  traumatic exposure or an enduring set of circumstances conferring multiple known risks to mental and physical health?

Did you often feel that …

You didn’t have enough to eat, had to wear dirty clothes, and had no one to protect you?

or

Your parents were too drunk or high to take care of you or take you to the doctor if you needed it?

Yes   No     If yes enter 1     ________

This item has already been discussed above, but is worth revisiting in terms of raising issues whether particular items refer either directly or indirectly to enduring sets of circumstances that pose their own enduring threat. The relevant question is whether items which ostensibly represent “traumatic events” and risk for subsequent problems are not risk factors, but only risk indicators, and not particularly informative ones.

Item 7: Could an ostensibly a traumatic exposure actually be no actual exposure?

Was your mother or stepmother:

Often pushed, grabbed, slapped, or had something thrown at her?    or

Sometimes or often kicked, bitten, hit with a fist, or hit with something hard?    or

Ever repeatedly hit over at least a few minutes or threatened with a gun or knife?

Yes   No     If yes enter 1     ________

Like item four, which refers to ostensible sexual abuse, this item seems to be one of the least ambiguous in terms of representing exposure to risk. But does it? We don’t know the timing, duration, or context. For instance, the mother might no longer be in the home and the respondent might not have known what happened at the time. There is even the possibility that the respondent was the “perpetrator” of such violence against the mother.

Items 8 and 9: Are traumatic exposures or indications of familial transmission of psychopathology?

Did you live with anyone who was a problem drinker or alcoholic or who used street drugs?

Yes   No

If yes enter 1     ________

And

Was a household member depressed or mentally ill or did a household member attempt suicide?    Yes   No     If yes enter 1     ________

These items are highly ambiguous. They don’t take in consideration whether the person was a biological relative, or whether they were a parent, sibling, or someone not biologically related. They don’t take into account timing. There may not have even been any direct exposure to the substance misuse or the attempted suicide, but the respondent only later learned of something that was closeted.

Item 10: traumatic exposure or relief from exposure?

Did a household member go to prison?

Yes   No

If yes enter 1     ________

The implications of endorsement of this item depend greatly on whom the household member was and the circumstances of them going to prison.

There may be a familial relationship with this person, but it could have been an abusive stepparents or stepsiblings, with the incarceration representing a lasting relief from some impressive situations. Or the person who became incarcerated was not an immediate family member, but somewhat more transient, maybe someone who was just renting a room or given a place to stay. We just don’t know.

Does adding up all these endorsements in a summary score clarify or confuse further?

Now add up your “Yes” answers:   _______   This is your ACE Score

 It would be useful to briefly review the assumptions involved in summing across items of a checklist and entering the summary score as a continuous variable in statistical analyses.

Classical test theory recognizes that the individual items may imperfectly reflect the underlying construct, in this case, traumatic exposure. However, in constructing a sum, the expectation is that the imperfections or errors of measurement in particular items cancel each other out. The summed score becomes a purer a representation of the underlying construct than any of the original items. Thus, the summary score will be more reliable and valid than any of the individual items would be.

There are a number of problems in applying this assumption to a summary ACE score. The items are quite heterogeneous, i.e., they vary wildly in whether they are likely to represent a traumatic exposure, and if so, the severity of that exposure. More importantly, there is a huge amount of variation in what these brief items would represent for particular individuals in the contexts they found themselves in the first 18 years of their lives. Undoubtedly, most endorsements of these items would represent false positives, if we hold ourselves to any strict definitions of trauma. If we don’t do so, we risk equating the only normative experiences that may have neutral or even positive effects on the respondent with serious exposures to traumatic events with lasting consequences

We are not in a position to know whether a score of five or even eight necessarily represents more traumatic exposure than a score of one.

Moreover, there is important empirical research of the clustering of events. We certainly cannot consider them random and unrelated. One classic study found 

In our data, total CCA was related to depressive symptoms, drug use, and antisocial behavior in a quadratic manner. Without further elucidation, this higher order relationship could have been interpreted as support for a sensitization process in which the long-term impact of each additional adversity on mental health compounds as childhood adversity accumulates. However, further analysis revealed that this acceleration effect was an artifact of the confounding of high cumulative adversity scores with the experience of more severe events. Thus, respondents with higher total CCA had disproportionately poorer emotional and behavioral functioning because of both the number and severity of the adversities they were exposed to, not the cumulative number of different types of adversities experienced.

And

Because low-impact adversities did not present a cumulative hazard to young adult mental health, they functioned as suppressor events in the total sum score, consistent with Turner and Wheaton’s (1997) expectation. Their inclusion increased the “noise” in the score and greatly watered down the influence of high-impact events. Thus, in addition to decreasing efficiency, total scores may seriously underestimate the cumulative effects of severe forms of childhood adversity, such as abuse and serious neglect.

But what if many or most of the high scores in a particular sample represent only a clustering of low- or no-impact adversities?

Another large-sample, key study cautioned:

Significant effects of parental separation}divorce in predicting subsequent mood disorders and addictive disorders are powerfully affected by whether or not there was parental violence and psychopathology in the household prior to the break-up and whether exposure to these adversities was reduced as a result of the separation (Kessler et al. 1997a). There are some situations – such as one in which the father was a violent alcoholic – where our data suggest that parental divorce and subsequent removal of the respondent from exposure to the father might actually be associated with a significant improvement in the respondent’s subsequent disorder risk profile, a possibility that has important social policy implications.

Finding Your ACE Score-page-0

NOTE

*Richard Bentall commonly interprets summed ACE scores in peer reviewed articles  as having a traditional dose-response association with mental health outcomes, and therefore as representing a modifiable causal factor in psychosis. In books and in social media, his claims become simply absurd.

bentall

I don’t think his interpretations withstand a scrutiny of the items and what a summed score might conceivably represent.

eBook_Mindfulness_345x550Preorders are being accepted for e-books providing skeptical looks at mindfulness and positive psychology, and arming citizen scientists with critical thinking skills. 

I will also be offering scientific writing courses on the web as I have been doing face-to-face for almost a decade. I want to give researchers the tools to get into the journals where their work will get the attention it deserves.

Sign up at my website to get advance notice of the forthcoming e-books and web courses, as well as upcoming blog posts at this and other blog sites. Get advance notice of forthcoming e-books and web courses. Lots to see at CoyneoftheRealm.com.
 

 

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s