What Makes Nutrition Research So Difficult to Conduct and Interpret?

Research Study Designs Are Important

When designing a nutrition research study, it is crucial to determine the best study design to answer the research question (1,2). Several research designs are commonly used to assess the impact of a dietary intervention on health and disease. Some of these include tightly controlled feeding studies, randomized clinical trials, and observational studies. Each design has advantages and disadvantages.

A feeding trial involves providing study participants with all meals, snacks, and beverages to control what they consume. Such trials provide the most control over participants’ dietary intake. Provided meals are carefully prepared, ingredients are weighed and measured, and participants are instructed to eat all the food they are provided (or to bring back what they did not eat). As an example, Bell et al. (3,4) used single-meal feeding trials to examine insulin dosing for protein and fat intake in participants with type 1 diabetes.

Feeding trials are expensive, and special facilities are needed to prepare and provide food for the participants. Feeding trials are usually short-term, not only because of the costs involved, but also because participants have to greatly modify their lives to adhere to eating only the foods and beverages provided by the trial. These trials usually enroll people who are quite different from the general public. For example, if a nutrition researcher wanted to study the impact of different amounts of dietary fiber on A1C levels, and wanted to include women with prediabetes who are 30–40 years of age, recruitment could prove to be difficult, and retaining participants could be even more challenging. The participants would need to be in the trial for at least 3 months to get meaningful changes in A1C; however, women in this age-group are likely to be married, have children, and be employed. To participate in such a trial, they would have to be willing to avoid eating any of the food prepared for the rest of their family, they would not be able share their trial-provided food with anyone else, and they would need to visit the research unit nearly every day to pick up meals.

The burden on participants enrolled in feeding trials is great, and the risk for nonadherence increased as the trial goes on. If participants do not strictly adhere to eating the foods they are provided, the findings of the trial can be seriously affected. Yet, it is difficult to precisely determine participants’ degree of adherence because they might eat foods other than those they were provided, share their foods with others, or not follow the protocol requiring them to bring back any foods they do not eat.

Feeding trials are not appropriate for answering questions about longer-term dietary exposures. Many diseases develop over time, so fully examining whether a particular dietary pattern affects health requires a long follow-up period and a large sample size. Randomized controlled intervention trials have been conducted to examine the effects of long-term dietary modification (5–7). Although these trials are not feeding trials, they still require a large budget and come with their own complexities. Research teams who have been able to conduct long-term dietary modification trials have relied on many different research methods and tools. Having close, ongoing contact with participants, carefully monitoring their adherence, and asking them to self-monitor what they eat are just a few of the key components of such trials.

These trials are labor-intensive for both researchers and participants, and in most cases, adherence has been shown to decrease after several months. Participant adherence to the intervention (and the control condition) is essential, but not always realized. Although nonadherence to protocol-specified behaviors is not a problem specific to nutrition research, the act of selecting, preparing, and eating foods is complex and repeated multiple times each day, providing many opportunities for nonadherence to the diet modification specified by the trial.

The Diabetes Prevention Program (DPP) trial is a good example of a large, randomized controlled intervention trial (8). Registered dietitians instructed participants in the intervention group to follow a healthful eating pattern, but meals were not provided. Participants had to learn approaches to reducing their fat and calorie intake to achieve a weight loss goal of 7% of baseline body weight and then to maintain their weight loss (8). They were required to self-monitor their behaviors, and dietitians reviewed this documentation regularly and provided constructive feedback. Participants attended weekly sessions during the first 24 weeks of the study. However, during the maintenance phase, contacts with the study team decreased to about twice per month. DPP participants had to remain adherent over a long period of time, so when contact frequency decreased, the research team had to work even more diligently to assist participants in meeting their study goals. If they had not done so, it would have been impossible to test the study hypothesis that lifestyle intervention would delay or prevent the development of type 2 diabetes in people with impaired glucose tolerance who were at high risk for developing the disease.

Observational studies have been used to track dietary intake in large numbers of participants and can be used to track such data over many years. Participants do not have to change their dietary behaviors to participate in an observational study. These studies are not as expensive to conduct, and the burden on participants and investigators is much lower.

Observational studies are not carefully controlled like clinical trials, so their results may be less reliable. Observational studies compare individuals who self-select to consume either a healthful or unhealthful diet, and the two groups may differ in other characteristics that could influence health (9,10). Nonetheless, such studies are vitally important for identifying possible connections between diet and health that can be further tested in randomized clinical trials.

Food Is Complicated

Nutrition researchers must be aware of and account for numerous food-related issues when collecting and analyzing dietary data. A few such issues are described below.

Foods and food products are available in different varieties, brands, and flavors. There are many options of the same type of food, yet the ingredients in each option may differ in ways that matter greatly to nutrition researchers. One brand of microwavable popcorn may contain butter, whereas another may contain buttery flavoring (butter-flavored coconut oil).

How food is cooked can change its nutrient profile. If vegetables are eaten fresh, many of their nutrients are preserved; however, if vegetables are boiled, many water-soluble nutrients can be lost. Many nutrients, including vitamin C, thiamin, and folic acid, are sensitive to temperature (11).

Where a food is prepared can make a substantial difference in the nutrient content because the ingredients used in their preparation can vary widely. Home-cooked foods versus restaurant-prepared foods can greatly differ in their nutrient composition.

Seasonal variations can make a difference in the nutrient content of fruits and vegetables. Also, those purchased directly from local farms have different levels of nutrients than those picked before they have ripened and been transported many miles to grocery stores.

The food environment is ever-changing, and food ingredients will continue to change, although consumers may not notice these changes at all. For example, until relatively recently, synthetic sources of trans fats were ubiquitous in bakery products as an emulsifier to improve creaming and reduce the rancidity of frying oil (12). Now, artificial trans fats are no longer contained in bakery products, or they have been greatly reduced (13). If a researcher wanted to examine the intake of cookies over time on health outcomes, using data overlapping this important change in the trans fat content of the cookies would greatly complicate this evaluation.

People Are Also Complicated

People vary in many ways, including by sex, race/ethnicity, BMI, economic status, metabolic rate, food preferences, exercise patterns, and fitness levels, among others. All of these differences could affect what study participants eat, how they metabolize what they eat, and how much they remember about what they eat.

The ability to assess dietary intake to explore diet-disease connections is essential to nutrition research but remains problematic for a number of reasons. Dietary data are typically obtained from participants who self-report what they eat (1,14). Common approaches used to collect dietary data include food records, food frequency questionnaires, and 24-hour recalls (1,14). To complete food records, participants are asked to carefully describe everything they eat or drink over a period of a few days, typically 3–4 days. Food records are burdensome to complete because participants must weigh and measure all of their foods and beverages and write all of this information down. To complete food frequency questionnaires, participants are asked to reflect on what they have typically eaten during the past 6 or 12 months, and it is known that participant recall is fallible. The 24-hour recall method collects dietary intake over a much shorter period of time, but participants can misreport what they eat.

Such misreporting can vary by participants’ personal characteristics. For example, a number of studies have shown that women and individuals who are heavier tend to underreport foods consumed (15–18). Underreporting of food intake by participants may reflect social desirability (i.e., not wanting to be judged by what they eat) (16,17). Underreporting may also reflect an inability to accurately estimate how much they actually consume. Extensive research has been conducted to identify approaches to mathematically adjust for misreported intake. Unfortunately, these corrections are not a perfect solution to fully address the problem of misreported food intake. Biomarkers have been studied to assist in the assessment of foods consumed. However, they carry with them a number of different, but still complicating, issues, including individual differences in nutrient absorption (19).

Food and Nutrition Databases Have Limitations

Food and nutrition databases are used to convert a consumed food to the specific nutrients contained in the food (20). These databases contain information about foods’ sources, profiles, nutrients, and dietary components.

It is not hard to imagine the complexity and difficulty involved in attempting to capture all of the different types and varieties of foods available to the public in one database. Each nutrient contained in each food is quantified. Significant effort goes into annually updating the content of food and nutrient databases as new foods are introduced to the marketplace almost daily. The databases are enormous because the intention is usually to include as many foods (and their variations) as possible. For example, there are more than 40 chicken soup options from which to choose in the U.S. Department of Agriculture/Agricultural Research Service Food and Nutrient Database for Dietary Studies (21).

Food and nutrient databases cannot include every component of every food. For example, nutrients are not the only components contained in foods, and the nonnutritive components in foods are usually not included in these databases. Vegetables and fruits contain many nonnutritive compounds, including dithiolthiones, monoterpenes, indoles sterols, and sulfhydryls. Although these components are not nutritional, they are in the foods research participants eat, and they may be important to health.

It is also known that food packaging may have an impact on health. Bisphenol A (BPA) is a good example of a nonnutritive but important exposure that was linked to food packaging. BPA is considered an endocrine disruptor, and the primary source of exposure to BPA is through the diet (22). BPA has been known to leach into food from internal epoxy resin coatings on cans used for canning foods such as soups and vegetables. BPA is not included in food and nutrition databases.

Reporting of Nutrition Research Findings Can Be Confusing

Once a research study is published, its results are likely to be picked up by the lay press. Such reporting is where a good deal of nutrition confusion is created. Nutrition is a topic of great interest to people, particularly those who have been diagnosed with diabetes, who often inquire about which diet is best to help them achieve target glucose levels or to lose or maintain body weight. Where they get answers to such questions matters; unfortunately, most people learn about nutrition research findings in the popular press or rely on the Internet for nutrition information. Although a wealth of nutrition information is available, some of it may be unreliable.

The online newsletter “Obesity and Energetics Offerings,” from the Indiana School of Public Health in Bloomington and the University of Alabama–Birmingham, provides some excellent examples of headlines that have failed to accurately describe the results of nutrition studies. The newsletter’s “Headline vs. Study” section on 11 October 2019 (23), for example, provided a link to an article in Women’s Health magazine titled, “Snacking on Nuts Found to Help Prevent Weight Gain” (24), followed by a link to the actual research report on which the article was based. The study was not a randomized controlled trial, the nuts were not provided to participants, and the participants self-reported both their nut intake and their weight. Although consumers reading the magazine headline might have thought that snacking on nuts would help them control their weight, all the research really found was an association—not necessarily causal—between self-reported nut intake and self-reported weight (25).

Similarly, data from the Women’s Health Initiative study cohort were used to explore links between chocolate consumption and the risk of diabetes (26). The authors cautiously presented their finding of an inverse relationship between chocolate intake and incident diabetes at moderate levels of chocolate consumption. However, press reports about the study simply suggested that eating chocolate would stave off diabetes (27).

Because patients may not be able to interpret the results of nutrition-related studies they see reported in the news, health care providers should be prepared to discuss these issues when patients inquire. Most Americans likely do not understand the nuances of such research or how links between foods or diets and chronic disease outcomes are stronger when supported by consistent evidence from basic science research, feeding studies, observational cohort studies, and randomized controlled clinical trials. People do not always realize that findings from one study cannot stand alone and that results from multiple studies are usually needed to create the robust evidence necessary for nutrition recommendations. Health care providers can help patients better understand nutrition research findings reported in the popular media by asking them to think about how many times they have read or heard about so-called “miracle foods.”

Importantly, when patients ask questions about nutrition-related news articles, they are opening the door to discussing their own dietary habits. Health care providers might consider providing simple suggestions for improving dietary intake (e.g., drink water instead of sugary beverages or eat vegetables at every meal), referral to a dietitian, and information about Internet sites that support evidence-based dietary recommendations.

To address conflicting data and nutrition recommendations, the American Diabetes Association (ADA) routinely reviews diabetes-related nutrition research and determines whether the evidence is strong enough to issue guidelines or recommend changes to the standards of care. The recently published, “Nutrition Therapy for Adults With Diabetes or Prediabetes: A Consensus Report” (28), for example, summarized the evidence supporting a new recommendation acknowledging that several different dietary patterns are acceptable for people who have prediabetes.