Last year, on CNN’s “Anderson Cooper 360” show ((“Anderson Cooper 360,” November 22 2005.)), Elizabeth Marquardt, author of Between Two Worlds – which is being re-released in a trade paperback edition this month – had this exchange with Cooper:
Anderson Cooper: Elizabeth, one of the questions that you asked the participants is whether or not they would describe their family as stressful. Fifty-one percent of children of — of what might be termed good divorces agreed with that statement, as compared to only 35 percent of those in unhappy marriages. What do you make of that?
Elizabeth Marquardt: That’s right. One of the striking findings to come out of our study was that children of good divorces often fare worse than those from unhappy marriages, so long as the marriage is low-conflict. And most marriages that end in divorce now are low-conflict.
This is fairly typical of how the Between Two Worlds study was reported – compared to the control group (young adults raised in intact marriages), young adults whose parents divorced fare worse. Elizabeth relies on this comparison often; in the Anderson Cooper interview, she criticizes The Good Divorce author Constance Abram’s research for having no control group of nondivorced parents.
Here’s another example, from a New York Times article about the Between Two Worlds study: ((“Poll Says Even Quiet Divorces Affect Children’s Paths,” New York Times, November 5 2005, p. A13.))
The new survey, based on the first nationally representative sample of young adults, highlights the many ways that divorce shapes the emotional tenor of childhood.
For example, those who grew up in divorced families were far more likely than those with married parents to say that they felt like a different person with each parent, that they sometimes felt like outsiders in their own home and that they had been alone a lot as a child.
Those with married parents, however, were far more likely to say that children were at the center of their family and that they generally felt emotionally safe.
Unfortunately, the methodology of the Between Two Worlds study is fatally flawed, and the research cannot support any of these comparisons. Why? Because respondents were asked different questions depending on if their parents had divorced or not.
For instance, take the finding, reported in the Times, that adults raised by married parents were far more likely to sat “that they generally felt emotionally safe” when they were growing up. This finding is also reported in Between Two Worlds (page 59):
In our national survey, most young adults from intact families strongly agree that when they were grouping up, “I generally felt emotionally safe.” But fewer than half of those from divorced families say the same thing. Young people who grew up in “bad divorces” are less likely to say they felt emotionally safe, but I was surprised to find that even those of us from “good divorces” felt significantly less safe than our peers from intact families with unhappy but low-conflict marriages.”
This finding is based on asking young adults whose parents stayed married to agree or disagree with the statement “I generally felt emotionally safe.” In contrast, those whose parents divorced were instead asked “After my parents’ divorce, I generally felt emotionally safe.” ((Source: Appendix “B” of Between Two Worlds, page 117, available online as a pdf file.))
So the respondents in the control group were asked about their childhood in general. In contrast, the respondents from divorced families were asked to focus specifically on a major family trauma. Given the biased questioning, it would have been a miracle if Elizabeth didn’t find major differences between the two groups. But it doesn’t tell us anything about outcomes, or about long-range trauma. The study design cannot distinguish between those who were unhappy for a while in the post-divorce period, but on the whole recovered; and those who were left with long-term, ongoing trauma due to their parents’ divorce.
In the Times article, Robert Emory (of the University of Virginia’s “Center for Children, Families and the Law”) said ”The key is to separate pain from pathology.” This is the distinction that the Between Two Worlds study failed to make.
To see why this matters, imagine doing a similar study, this time dividing respondents into those who had pets who died, and those who didn’t. If you asked the “no pet” group if they felt emotionally safe as kids, but asked the “pet died” group if they felt emotionally safe after their pet’s death, probably many more of the latter group would say they didn’t feel emotionally safe. But would that tell us anything about the long-range outcomes for these two groups? Would a direct comparison of the two groups, as if they had been asked the same question, be appropriate? Of course not. A question about a particular, traumatic period in childhood cannot be used to characterize a respondent’s upbringing on the whole.
According to Elizabeth Marquardt, the respondents from divorced families were “periodically” reminded to focus on the period after their parents’ divorce throughout the survey. This means that almost none of the comparisons between divorced and non-divorced families in Between Two Worlds have any validity. For instance, the finding that Anderson Cooper focused on, that those from divorced families are more likely to “describe their family as stressful,” seems unremarkable when you keep in mind that the respondents were asked only about stress “after my parents divorce.” What child wouldn’t find that time stressful? But it doesn’t prove that their upbringing was more stressfel as a whole.
Even results from questions that didn’t include a “after my parents’ divorce” provision are dubious, in my eyes, because the context of the survey as a whole is changed by constantly asking those with divorced parents – and only those with divorced parents – to focus on a painful memory. The poor methodology taints the entire survey.
One finding from Between Two Worlds that Elizabeth emphasized is that “grown children from unhappy, low-conflict marriages generally fare better than those from ‘good’ divorces.” I doubt this finding would have been as strong — or existed at all — if this survey had asked the control group the same questions. Even the current, flawed results show some cases in which those raised by “good divorce” parents appear to be better off than those who parents stayed together. For instance, contrary to the New York Times’ reporting, those raised in “good divorce” families were slightly less likely to feel “like an outsider in my home,” than those whose parents stuck with unhappy marriages.
How much “extra” unhappiness, doubt and misery did Between Two Worlds measure by asking those with divorced parents, over and over, to think of “after my parents divorce?” There’s no way of knowing, but it could have been a large effect. Even a small effect could have seriously changed the finding that “grown children from unhappy, low-conflict marriages generally fare better than those from ‘good’ divorces,” because in many cases the statistical difference between those two groups was fairly small.
* * *
When I asked her about the difference in questions on her blog, Elizabeth responded:
Since we were studying post-divorce childhood, we asked those from divorced families to answer the question in regard to “after your parents’ divorce” and reminded them of that periodically through the interview. …If you want to study post-divorce childhood you have to ask people about, well, post-divorce childhood.
I thought they were studying the long-term effects of divorce. If it’s true that a person whose parents divorced when they were a child is significantly more likely to recall their upbringing as emotionally unsafe, stressful, etc., then that effect should show up without loading the question by asking subjects to focus specifically on the divorce.
If Elizabeth fears that subjects whose parents divorced when they were older – say, 10 or 15 – would have put too much weight on their pre-divorce time, then she should have asked her questions bounded by period (“When you were between 10 and 15 years old….” and so on). Then all respondents could have been asked the same questions, and the results examined for differences pre- and post- divorce. Failing to ask the control group the same questions is not a small design flaw; it’s a catastrophic error that invalidates all results based on comparisons to the control group. ((It’s notable that this study was put together by an anti-divorce think tank, and has never been subjected to peer review. It’s natural to wonder if the peer review process could have forced the study’s authors, Elizabeth Marquardt and Professor Norval Glenn, to improve their methodology.))
Between Two Worlds proves that many young adults whose parents divorced recall the period “after my parents divorce” as painful and confusing. However, nearly all ((I say “nearly all” because some of the findings — such as the greater likelihood of divorce among children of divorced parents — would not have been effected by the bad study design)) of the study’s conclusions — including the conclusion that children are better off if their “low conflict” unhappy parents stay together, rather amicably divorcing — are based on an invalid study design. In particular, most findings from Between Two Worlds based on comparisons to the control group should be regarded as invalid and unproved.