“Nutrition is for real people. Statistical humans are of little interest.”
Roger J. Williams, PhD
Nutritional epidemiology has many shortcomings when it comes to acting as a basis for public health nutrition policy. But you don’t have to take Walter Willett’s word for it. Apart from the weaknesses in the methodology, there is one great big elephant in the nutrition epidemiology room that no one really wants to talk about: our current culture-wide “health prescription.”
(Thanks to Utopia Theory!)
You don’t have to care about or read about nutrition to know that “fat is bad” and “whole grains are good” [1,2]. Whether or not you follow the nutrition part of the current “health prescription” is likely to depend on a host of other factors related to general “health prescription” adherence, which in turn may have a much larger impact on your health than your actual nutritional choices. This is especially true because variation in intake and/or variation in risk related to intake are frequently quite small.
For example, in a study relating French fry consumption to type 2 diabetes, the women who ate the least amount of French fries ate 0 servings per day while the women who ate the most ate 0.14 servings per day or about 5 French fries per day (i.e. not a big difference in intake) . The risk of developing type 2 diabetes among 5-fries a day piggies was observed to be .21 times greater than the risk among the no-fry zone ladies (i.e. not a big variation in risk).
Okay, everyone knows that French fries are “bad for you.” But these ladies ate them anyway. Were there other factors related to general “health prescription” adherence which may have had an impact on their risk of diabetes?
The French fry eaters also “tended to have a higher dietary glycemic load and higher intakes of red meat, refined grain, and total calories. They were more likely to smoke but were less likely to take multivitamins and postmenopausal hormone therapy.” (They also exercised less.) In other words, the French fry eaters, within a context of a known “health prescription” had chosen to ignore a number of healthy lifestyle recommendations, not just the ones related to French fries.
“As a general rule, noncompliant patients will usually have worse outcomes than compliant patients. In fact, there is solid evidence that patients who fail to comply with a placebo have worse outcomes than patients who comply with a placebo [4, 5] . . . . Patients who comply poorly with a placebo probably have other poor self-care habits.”
[Also, see Gary Taubes’ characteristically exhaustive discussion of the compliance effect. Pack a lunch.]
If you think of our current default diet recommendation as the “placebo” (although its effects may not be exactly benign), it is clear that people who fail to comply with dietary prohibitions against red meat, saturated fats, and “junk” food like French fries may also be more likely to have other poor self-care habits, like smoking and not exercising. That poor health care habits are related to poor health is of no surprise to anyone.
In their statistical manipulation of a dataset, nutritional epidemiologists attempt to “control” for confounding variables (confounders), such as differences in health behavior. A confounder is something that may be related to both the hypothesized cause under investigation (i.e. French fry eating) and the outcome (i.e. type 2 diabetes). As such, it muddies the water when you are trying to figure out exactly what causes what.
When statisticians “control” or “adjust” for these confounders in a data set, they essentially “pretend” (that’s the exact word my biostats professor used) that the other qualities that any given individual brings to a data set are now equalized and that the specific factor under investigation—diet—has been isolated. Well, it has and it hasn’t. The “statistical humans” created by computer programs that now have equalized risk factors are a mirage; these people do not exist. The people who contributed the data that ostensibly demonstrates that “French fries increase risk of type 2 diabetes” are the exact same people who had other behaviors that may also contribute to increased risk of diabetes. (Please note: I chose this example, rather than “red meat causes heart disease” because there are many plausible explanations for French fries causing type 2 diabetes, it is just that you aren’t going to find evidence for them using this approach.)
(Thanks and apologies to Baloocartoons.com.)
Most nutritional epidemiology articles contain some version the following statement in their conclusions:
“We cannot rule out the possibility of unknown or residual confounding.”
Meaning: We can not rule out the possibility that our results can be explained by factors that we failed to fully take into account. Like the elephant in the room.
That this is actually the case becomes apparent when hypotheses that seem iron-clad in observational studies are put to the test in experimental conditions.
Lack of experimental confirmation
If ever there was a field about which you could say “for every study there is an equal and opposite study,” it is nutritional epidemiology–although experimental results are generally considered “more equal” than observational data. Associations that link specific nutrients to the prevention of specific diseases can be (relatively) strong and consistent in the context of nutritional epidemiology observational data, but absent in experimental situations. Epidemiological studies suggested that beta carotene could prevent cancer; experimental evidence suggested just the opposite and in fact, smokers given beta carotene supplements had increased risk of cancer . Epidemiological studies suggest that low-fat, high-carb diets are related to a healthy weight. This may be the case, but experimental evidence shows that reducing carbs and increasing fat is more effective for weight loss [7, 8]. In one study, when experiment participants added carbs back into their diet (the increase in calories from 2 months to 12 months is entirely accounted for–and then some–by carbohydrate), they regained the weight they had lost.*
Data from 
Kenneth Rothman, in his book Epidemiology: An Introduction, emphasizes the importance of applying Karl Popper’s philosophy of refutationism to epidemiology:
“The refutationist philosophy postulates that all scientific knowledge is tentative in that it may one day need to be refined or even discarded. Under this philosophy, what we call scientific knowledge is a body of as yet unrefuted hypotheses that appear to explain existing observations.” 
Rothman makes the point that there is an asymmetry when it comes to refuting hypotheses based on observations: a single contrary observation carries more weight in judging whether or not a hypothesis is false than a hundred observations that suggest that it is true.
In the case of the current “low fat, whole grain diets will prevent chronic disease” hypothesis, there is not just one contrary observation, but scores of them, including the results of applying this hypothesis in a 30-year, population-wide experiment in the US.
If the current nutrition paradigm needs to be “refined or even discarded,” how will we acquire the knowledge we need to create a better system? How can we move away from “statistical people” towards a perspective that encompasses the individual variations in genetics, culture, and lifestyle that have such a tremendous impact on health?
Tune in next time for the final episode of N of 1 nutrition when I ask the all-important question: What the heck does n of 1 nutrition have to do with public health?
*This doesn’t mean that carbs are evil–some of my best friends are carbs–but that the conditions in a population that are associated with a healthy weight and the conditions in an experiment to that lead to increased weight loss are very different.
1. Eckel RH, Kris-Etherton P, Lichtenstein AH, Wylie-Rosett J, Groom A, Stitzel KF, Yin-Piazza S. Americans’ awareness, knowledge, and behaviors regarding fats: 2006-2007. J Am Diet Assoc. 2009 Feb;109(2):288-96.
2. Marquart L, Pham AT, Lautenschlager L, Croy M, Sobal J. Beliefs about whole-grain foods by food and nutrition professionals, health club members, and special supplemental nutrition program for women, infants, and children participants/State fair attendees. J Am Diet Assoc. 2006 Nov;106(11):1856-60.
3. Halton TL, Willett WC, Liu S, et al. Potato and french fry consumption and the risk of type 2 diabetes in women. Am J Clin Nutr. 2006 Feb;83(2):284-90.
4. Coronary Drug Project Research Group. Influence of adherence to treatment and response of cholesterol on mortality in the coronary drug project. N Engl J Med. 1980 Oct 30;303(18):1038-41.
5. Horwitz RI, Viscoli CM, Berkman L et al. Treatment adherence and risk of death after a myocardial infarction. Lancet. 1990 Sep 1;336(8714):542-5.
6. Willett, W. Nutrition Epidemiology, 2nd ed. New York: Oxford University Press, 1998.
7. Gardner C, Kiazand A, Alhassan, et al. Weight Loss Study: A Randomized Trial Among Overweight Premenopausal Women: The A TO Z Diets for Change in Weight and Related Risk Factors .Comparison of the Atkins, Zone, Ornish, and LEARN. Journal of the American Medical Association. 2007;297(9):969-977
8. Shai I, Schwarzfuchs D, Henkin Y, Shahar DR, et al; Dietary Intervention Randomized Controlled Trial (DIRECT) Group. Weight loss with a low-carbohydrate, Mediterranean, or low-fat diet. N Engl J Med. 2008 Jul 17;359(3):229-41.
9. Rothman, K. Epidemiology: An Introduction. New York: Oxford University Press, 2002.
11 thoughts on “N of 1 Nutrition Part 4: The Elephant in the Room”
I love the asterisk – “the conditions in a population that are associated with a healthy weight and the conditions in an experiment to that lead to increased weight loss are very different.” I’ve always understood intuitively that just because healthy people do x thing, it doesn’t mean that doing x thing will make an unhealthy person healthy (or substitute anything else for “healthy”). I see attempts to link behaviors and outcomes so often in diet/food commercials: “women who eat Special K for breakfast have smaller waist measurements than women who don’t” or something along those lines.
There are so many aspects that don’t make sense – I’ve spent the last 10 years mimicking the supposed behavior of normal-weight people, and it hasn’t caused me to be normal weight. At the same time, when I observe my friends and acquaintances who ARE normal weight, I see that they eat junk food and fast food, rarely eat anything like vegetables or oatmeal, drink a lot of soda, smoke cigarettes for breakfast, and exercise one week per year right before they go on a beach vacation. (I’m kind of mixing up “health” and weight, but you see what I mean.). If that isn’t a case of “other factors” being involved, I don’t know what is.
If a+b=c, then c-a=b and c-b=a *
this seems to be not necessarily true when it comes to nutrition.
*that’s as science-y as I get.
LOL, That’s probably as science-y as anybody needs to get! Equating a relationship with cause & effect is a such common and misleading occurrence in reporting nutrition findings–I literally wish there were a law against it (and I’m not a big fan of laws in general, plus my husband would say that you can’t legislate intelligence). Proving true cause & effect in chronic disease and obesity is so complicated & not even necessarily helpful. We’ve got to learn to do a better job of getting objective information to people so they can figure it out for themselves. Getting health claims off of boxes of cereal would be an excellent place to start.
The real problem here is that what is called “controlling” is not control. Backing noise out of a statistical model is not anything like control. Hey, I hope you will visit my blog today 🙂
LOL that is TOO funny! I’d been working on this post for a week–when I start writing about the flaws in nutrition epi, I get carried away–and finally managed to get out there. We must be channeling the same cosmic force: “Why look! Elephant!”
And yes, you are right about “controlling.” I kept waiting for this concept to make sense to me in class, and when my professor finally used the word “pretend,” it did.
so funny, Sunday I checked to see if you had posted something new. Oh, must be busy….. I read something else into your prof’s comment. As in, “lets pretend that we are really dealing with a fully-populated orthogonal design space here, with an actual design that will allow us to deal with each factor independent of the other.” Fat chance to get a “design” like that unless it is actually designed that way.
What I kept running across was that nutrition epidemiologists use data from studies that weren’t even designed to investigate the factor of interest they were trying to look at, much less separate that factor out from the factors with which it would be intertwined.
no problem. Add in a bunch of junk data and make up for it in volume. This is why I am so wary of meta-analyses.
LOL–“make up for it in volume” I thought that’s what bad musicians did!
“The risk of developing type 2 diabetes among 5-fries a day piggies was observed to be .21 times the risk among the no-fry zone ladies”
Eating fries reduces your risk by a factor 5? Or do you mean the “additional risk” is 0.21?
Thanks for the catch. I left out the “greater than.”