Research reporting fidelity refers to the integrity involved with making decisions about how to analyze and present data in scientific papers, talks, and press releases, such that our data and the methods used to generate them are described without artifice, without exaggeration, and with sufficient detail for other scientists t judge and replicate our methods. A current concern within health research is compromised research- reporting fidelity in many domains of inquiry. However, the literature on dietary weight-loss supplements may contain some of the most pervasive and egregious breaches. As a nation where the majority of individuals struggle with obesity, dietary weight-loss supplements are a popular resource used to lose weight. Any gap between evidence-based science and conclusions drawn about weight-loss supplements is of concern to biomedical research and to public health, given the number of studies conducted on these supplements and the number of persons taking such supplements. The objective of this proposal is to focus on two particular concerns: (1) a phenomenon commonly referred to as p-hacking defined as reanalyzing data using various tests until a p-value below the nominal alpha level is obtained, thus inflating the chances of obtaining false positive results, and (2) common errors in the calculation of effect sizes in meta-analytic research. This proposal aims to assess the magnitude of these problems, thereby laying a baseline for the evaluation of future progress in mitigating these problems, and to test an educational program to mitigate one of these problems.
The first aim i s to determine the prevalence of p-hacking within weight-loss supplement literature with unprecedented magnitude, speed, and efficiency by combining sophisticated statistical methods, such as multiple imputation and double sampling, as well as by employing the use of crowdsourcing.
The second aim i s to determine the frequency of effect size estimation errors that occur within peer-reviewed meta-analyses conducted on dietary weight-loss supplements, by systematically reviewing meta-analyses conducted on weight-loss supplements.
The third aim i s to develop and test the efficacy of three brief training modules for early career investigators designed to enhance effect-size estimation data reproducibility using weight-loss supplement meta-analyses, by conducting a simple randomized controlled trial. The methodological advances this proposal aims to foster will enhance the technical capabilities of future researchers who wish to employ the use of multiple imputation, double sampling, and crowdsourcing to investigate questions requiring the extraction of large amounts of detailed information from existing scientific literature. Additionally, the results obtained from this project will help improve the accuracy and reproducibility of dietary weight-loss supplement research, making accurate scientific information more accessible to the novice, the broader scientific community, regulatory agencies, medical decision makers, and for public health to help improve the health and life quality of persons taking, or considering taking, dietary supplements for weight loss.
The integrity and value of science depends on the rigorous implementation of research methods and faithful reporting of procedures and results, yet in practice, many failings in implementation and reporting occur, which undermine the value of the research conducted. In the proposed fellowship training, the candidate will learn methods to detect such failings and monitor their frequency, and to test methods to train others to minimize such errors, by focusing on peer-reviewed literature investigating dietary supplements used for weight loss. As a nation where many individuals struggle with obesity and use dietary supplements to lose weight, this will have great public health value by contributing to the validit of health information provided to the public.
| Trepanowski, John F; Kroeger, Cynthia M; Barnosky, Adrienne et al. (2018) Effects of alternate-day fasting or daily calorie restriction on body composition, fat distribution, and circulating adipokines: Secondary analysis of a randomized controlled trial. Clin Nutr 37:1871-1878 |
| Kroeger, Cynthia M; Allison, David B; Thomas, Diana M et al. (2018) TO THE EDITOR. Spine (Phila Pa 1976) 43:E492-E493 |
| Miranda, E R; Fuller, K N Z; Perkins, R K et al. (2018) Endogenous secretory RAGE increases with improvements in body composition and is associated with markers of adipocyte health. Nutr Metab Cardiovasc Dis 28:1155-1165 |
| Trepanowski, John F; Kroeger, Cynthia M; Barnosky, Adrienne et al. (2017) Effect of Alternate-Day Fasting on Weight Loss, Weight Maintenance, and Cardioprotection Among Metabolically Healthy Obese Adults: A Randomized Clinical Trial. JAMA Intern Med 177:930-938 |
