“Nutrition is for real people. Statistical humans are of little interest.”
Roger J. Williams, PhD

Nutritional epidemiology has many shortcomings when it comes to acting as a basis for public health nutrition policy.   But you don’t have to take Walter Willett’s word for it.  Apart from the weaknesses in the methodology, there is one great big elephant in the nutrition epidemiology room that no one really wants to talk about:  our current culture-wide “health prescription.”

(Thanks to Utopia Theory!)

You don’t have to care about or read about nutrition to know that “fat is bad” and “whole grains are good” [1,2]. Whether or not you follow the nutrition part of the current  “health prescription” is likely to depend on a host of other factors related to general “health prescription” adherence, which in turn may have a much larger impact on your health than your actual nutritional choices. This is especially true because variation in intake and/or variation in risk related to intake are frequently quite small.

For example, in a study relating French fry consumption to type 2 diabetes, the women who ate the least amount of French fries ate 0 servings per day while the women who ate the most ate 0.14 servings per day or about 5 French fries per day (i.e. not a big difference in intake) [3]. The risk of developing type 2 diabetes among 5-fries a day piggies was observed to be .21 times greater than the risk among the no-fry zone ladies (i.e. not a big variation in risk).

Okay, everyone knows that French fries are “bad for you.” But these ladies ate them anyway. Were there other factors related to general “health prescription” adherence which may have had an impact on their risk of diabetes?

The French fry eaters also “tended to have a higher dietary glycemic load and higher intakes of red meat, refined grain, and total calories. They were more likely to smoke but were less likely to take multivitamins and postmenopausal hormone therapy.” (They also exercised less.) In other words, the French fry eaters, within a context of a known “health prescription” had chosen to ignore a number of healthy lifestyle recommendations, not just the ones related to French fries.

“As a general rule, noncompliant patients will usually have worse outcomes than compliant patients. In fact, there is solid evidence that patients who fail to comply with a placebo have worse outcomes than patients who comply with a placebo [4, 5] . . . . Patients who comply poorly with a placebo probably have other poor self-care habits.”

[Also, see Gary Taubes’ characteristically exhaustive discussion of the compliance effect. Pack a lunch.]

If you think of our current default diet recommendation as the “placebo” (although its effects may not be exactly benign), it is clear that people who fail to comply with dietary prohibitions against red meat, saturated fats, and “junk” food like French fries may also be more likely to have other poor self-care habits, like smoking and not exercising. That poor health care habits are related to poor health is of no surprise to anyone.

Statistical people

In their statistical manipulation of a dataset, nutritional epidemiologists attempt to “control” for confounding variables (confounders), such as differences in health behavior. A confounder is something that may be related to both the hypothesized cause under investigation (i.e. French fry eating) and the outcome (i.e. type 2 diabetes).  As such, it muddies the water when you are trying to figure out exactly what causes what.

When statisticians “control” or “adjust” for these confounders in a data set, they essentially “pretend” (that’s the exact word my biostats professor used) that the other qualities that any given individual brings to a data set are now equalized and that the specific factor under investigation—diet—has been isolated. Well, it has and it hasn’t. The “statistical humans” created by computer programs that now have equalized risk factors are a mirage; these people do not exist. The people who contributed the data that ostensibly demonstrates that “French fries increase risk of type 2 diabetes” are the exact same people who had other behaviors that may also contribute to increased risk of diabetes. (Please note: I chose this example, rather than “red meat causes heart disease” because there are many plausible explanations for French fries causing type 2 diabetes, it is just that you aren’t going to find evidence for them using this approach.)

(Thanks and apologies to Baloocartoons.com.)

Most nutritional epidemiology articles contain some version the following statement in their conclusions:

“We cannot rule out the possibility of unknown or residual confounding.”

Meaning: We can not rule out the possibility that our results can be explained by factors that we failed to fully take into account. Like the elephant in the room.

That this is actually the case becomes apparent when hypotheses that seem iron-clad in observational studies are put to the test in experimental conditions.

Lack of experimental confirmation

If ever there was a field about which you could say “for every study there is an equal and opposite study,” it is nutritional epidemiology–although experimental results are generally considered “more equal” than observational data. Associations that link specific nutrients to the prevention of specific diseases can be (relatively) strong and consistent in the context of nutritional epidemiology observational data, but absent in experimental situations. Epidemiological studies suggested that beta carotene could prevent cancer; experimental evidence suggested just the opposite and in fact, smokers given beta carotene supplements had increased risk of cancer [6]. Epidemiological studies suggest that low-fat, high-carb diets are related to a healthy weight. This may be the case, but experimental evidence shows that reducing carbs and increasing fat is more effective for weight loss [7, 8]. In one study, when experiment participants added carbs back into their diet (the increase in calories from 2 months to 12 months is entirely accounted for–and then some–by carbohydrate), they regained the weight they had lost.*

Data from [7]

Kenneth Rothman, in his book Epidemiology: An Introduction, emphasizes the importance of applying Karl Popper’s philosophy of refutationism to epidemiology:

“The refutationist philosophy postulates that all scientific knowledge is tentative in that it may one day need to be refined or even discarded. Under this philosophy, what we call scientific knowledge is a body of as yet unrefuted hypotheses that appear to explain existing observations.” [9]

Rothman makes the point that there is an asymmetry when it comes to refuting hypotheses based on observations: a single contrary observation carries more weight in judging whether or not a hypothesis is false than a hundred observations that suggest that it is true.

In the case of the current “low fat, whole grain diets will prevent chronic disease” hypothesis, there is not just one contrary observation, but scores of them, including the results of applying this hypothesis in a 30-year, population-wide experiment in the US.

If the current nutrition paradigm needs to be “refined or even discarded,” how will we acquire the knowledge we need to create a better system? How can we move away from “statistical people” towards a perspective that encompasses the individual variations in genetics, culture, and lifestyle that have such a tremendous impact on health?

Tune in next time for the final episode of N of 1 nutrition when I ask the all-important question: What the heck does n of 1 nutrition have to do with public health?

*This doesn’t mean that carbs are evil–some of my best friends are carbs–but that the conditions in a population that are associated with a healthy weight and the conditions in an experiment to that lead to increased weight loss are very different.

References:

1. Eckel RH, Kris-Etherton P, Lichtenstein AH, Wylie-Rosett J, Groom A, Stitzel KF, Yin-Piazza S. Americans’ awareness, knowledge, and behaviors regarding fats: 2006-2007. J Am Diet Assoc. 2009 Feb;109(2):288-96.

2. Marquart L, Pham AT, Lautenschlager L, Croy M, Sobal J. Beliefs about whole-grain foods by food and nutrition professionals, health club members, and special supplemental nutrition program for women, infants, and children participants/State fair attendees. J Am Diet Assoc. 2006 Nov;106(11):1856-60.

3. Halton TL, Willett WC, Liu S, et al. Potato and french fry consumption and the risk of type 2 diabetes in women. Am J Clin Nutr. 2006 Feb;83(2):284-90.

4. Coronary Drug Project Research Group. Influence of adherence to treatment and response of cholesterol on mortality in the coronary drug project. N Engl J Med. 1980 Oct 30;303(18):1038-41.

5. Horwitz RI, Viscoli CM, Berkman L et al. Treatment adherence and risk of death after a myocardial infarction. Lancet. 1990 Sep 1;336(8714):542-5.

6. Willett, W. Nutrition Epidemiology, 2^nd ed. New York: Oxford University Press, 1998.

7. Gardner C, Kiazand A, Alhassan, et al. Weight Loss Study: A Randomized Trial Among Overweight Premenopausal Women: The A TO Z Diets for Change in Weight and Related Risk Factors .Comparison of the Atkins, Zone, Ornish, and LEARN. Journal of the American Medical Association. 2007;297(9):969-977

8. Shai I, Schwarzfuchs D, Henkin Y, Shahar DR, et al; Dietary Intervention Randomized Controlled Trial (DIRECT) Group. Weight loss with a low-carbohydrate, Mediterranean, or low-fat diet. N Engl J Med. 2008 Jul 17;359(3):229-41.

9. Rothman, K. Epidemiology: An Introduction. New York: Oxford University Press, 2002.