One of the major problems in nutritional epidemiology is that we have a hard time measuring things that we are supposed to be measuring in order to say anything meaningful about relationships between diet and chronic disease. You know, things like how much people are actually eating or how much physical activity they really do get.
Today I had the opportunity to hear Walter Willett, king daddy of the field of nutritional epidemiology, speak on just this dilemma (and yes, he still has that sweet ‘stache). Introduced as having received “too many awards to mention,” W began his talk–“Energy Balance and Beyond: The Power and Limits of Dietary Data”–by addressing the recent unpleasantness raised by researchers who have suggested that the dietary data that we collect simply isn’t worth analyzing (Archer, Pavela, & Lavie 2015; Dhurandhar et al. 2015).
Having been recently immersed in my rhetoric of science readings, I noted that W started right off with some perfunctory boundary work, as way of indicating who was “in” and who was “out” when it came to credibility: He noted that the investigators questioning the value of dietary self-reports are “funded by Coca-cola,” and even the ones that are “pretty good scientists” are “not epidemiologists” and therefore “a little bit naive.” So much for evaluating the data and the arguments on their own merits.
Then he trotted out the “slippery slope” argument: If we throw out self-reported dietary data because it is wildly inaccurate, then not only do we have to “throw out the Dietary Guidelines” (heaven forfend!), but we’d have to throw out occupational safety and drug trial data also–because these are often based on self-reports. Certainly, there’s very little difference between reporting on events in your workplace or what side effects you might have in response to a pill you’re taking and reporting on what you remember eating over the course of the past year.
Then he got down to the nitty-gritty. The reason your Average American is fat is, to put it bluntly, because of math:
2500 kcals/day x 1% = 25 kcals/day
25 kcals/day x 365 days/year = 9125 kcals/year
9125 kcals/year = about 1kg of weight gain/year
Of course, the Average American only gains (on average) about 0.5 kg/year, but W easily explained the discrepancy: We gain weight, but then we have to expend more energy dragging our fat asses around–my words, not his–so we don’t gain as much weight as we would, but then the increased energy expenditure makes us hungry, so we eat more, so we gain weight, ad infinitum, only not, because we seem to plateau, but then, well, there’s that.
So here’s the $64,000 (or really a few hundred million in grant money) question: How good are we at measuring the 1% (or less) difference between “energy in” and “energy out” that we see expressed as weight gain in the population?
The answer, according to the man himself? Not very.
W then showed a comparison of a set of methods to measure “energy in,” along with their coefficient of variation (you can get the technical explanation here, but it is–simplistically–a measure of the amount of variability in your data; long story short: larger numbers = more variability, less precision and smaller numbers = less variability, more precision).
It looked something like this:
|Method||Co-efficient of variation|
|Food frequency questionaire (FFQ)||15%|
|24-hour diet recall interview||28%|
|Doubly-labeled water (DLW)||9%|
What W made clear is that, if you want to know whether–or to what extent–Americans are indeed eating more and moving less, you can’t measure “energy in” using FFQs, diet records, and interviews and expect to get anywhere near that “1% of calories” accuracy you’re looking for. You can’t even rely on doubly-labeled water, typically considered to be a biomarker measurement with a high degree of precision. In fact, W made the point that DLW samples sent out to different labs would come back with results that differed by up to 50%.
W went on to explain that we have similar difficulties measuring “energy out,” with our very best measurements having a coefficient of variation of nearly 20%.
So in other words, or actually in W’s exact words:
“Weight is the best measure of energy balance.”
As far as I can figure it, using weight as a way of “measuring” (and I use that term loosely) energy balance creates a theoretical–if not a methodological–situation that is, in a word, unfalsifiable. Or–in another word–bogus.
“Weight gain results when the things that we think cause weight gain happen, thus proving that those things have happened.”
At least one of the reasons for attempting to measure “energy in” and “energy out” is to find out whether or not it makes sense to attribute weight gain (or loss) to the differences between them. Instead, W is saying, we can know all we really need to know about how much people eat or move or both, because, voila, weight.
This is like saying, even though traces of tooth fairy fingerprints, footprints, or fiber samples are extremely difficult to obtain, we can reliably determine the existence of tooth fairies by the presence of quarters under your pillow.
Not only does this completely disregard the ever-growing list of things that may also contribute to weight gain/loss over time,* but it contradicts W’s own assertion just moments later that–even though we can’t do it very accurately–a good reason for attempting to measure total energy intake is that it can act as a “crude measure of physical activity.”
One the one hand, W is saying we can estimate how active you are by finding out how much you eat because people who eat more are also likely to be more active–that’s why they eat more–and people who eat less are typically less active–that’s why they don’t eat as much–with the implication being that “energy in” and “energy out” are positively related; as one changes, the other changes in the same direction.
On the other hand, he’s saying these two variables are completely independent of each other. Eating more doesn’t imply you move more; moving less doesn’t imply you eat less. And the way we know that these two variables are disconnected in any given Average American is, voila again, weight.
At this point, it was hard not to be completely distracted by the cognitive dissonance ringing through the room.
I did hang on long enough to hear W say that we shouldn’t really be worried about energy intake anyway because what really matters is diet quality, which, by the way, we can’t measure accurately either.
With all due respect, all I could think of is that while Emperor W may not be completely without clothes, he was definitely down to boxer shorts today.
In an auditorium full of really smart people, I cannot have been the only person thinking that W and his data looked a little over-exposed. But–as we saw with the circumstances revealed by the Ramsden and Zamora paper last week–it can be hard to contradict a famous colleague, and in nutritional epidemiology, no one is famouser than W. It may be even harder, I suppose, when he is an invited guest and an apparently nice fellow. The Q&A was respectful and polite. Difficult as it is to believe, even I kept my mouth shut.
I know science sometimes advances one funeral at a time, and I truly wish W a long and happy life. But maybe he’ll start to get a little chilly there in his boxers and start thinking about retiring someplace warm. Soon.
Non-exercise activity levels
Other unknown factors