The link between biomarkers and diseases: don't trust and check!
According to the results of a study conducted by John Ioannidis, a research planning expert from Stanford University, the associations between genes or other biomarkers and diseases identified during more than two dozen widely cited studies are significantly exaggerated. The result of this may be inadequate medical recommendations that are not supported by data from larger studies.
The works under discussion include studies that have linked the BRCA1 mutation with colon cancer, the level of C-reactive protein with diseases of the cardiovascular system and the level of homocysteine with vascular diseases.
According to Ioannidis, these exaggerations are most likely the result of the vagaries of statistics in combination with the peculiarities of human nature and the competitive nature of scientific publications. No scientific result is unambiguous, there are always fluctuations. This is not about cheating or poor research planning, but about statistical expectation. Some of the results are always stronger, while the other part is weaker, but scientific journals and researchers like to publish data on high levels of correlation.
After the publication appears, confidence in the existence of a relationship between the marker and the disease often persists, partly due to the common practice of quoting earlier studies in each new article. Due to repeated citations, the results of these studies are beginning to be perceived as indisputable facts even in the presence of later works demonstrating less pronounced or even statistically insignificant connections between the studied indicators.
As part of their research, Ioannidis and his colleague Orestis Panagiotou from Ioannina University, Greece, analyzed 35 widely cited studies published in reputable scientific journals between 1991 and 2006. Each of the works was cited in later publications at least 400 times, some of them were referenced several thousand times once. All studies analyzed the relationship between biomarkers, such as variants of nucleotide sequences of certain genes, pathogenic microorganisms, blood protein levels, etc., with the likelihood of developing various diseases, including cancer and diseases of the cardiovascular system.
As a result, the authors found that, subsequently, when conducting larger-scale studies or meta-analysis (generalizing the results of dozens of papers on one topic), only less than half of the biomarkers they selected were statistically significantly associated with the risks of developing diseases. In more than 80% of cases, the correlation between biomarkers and diseases in the first or more cited publications was significantly higher than in subsequent and less cited articles
This pattern also extended to negative results. For example, in one frequently cited article published in the New England Journal of Medicine, it was stated that infection with penicillin-resistant bacteria does not increase the risk of death of a patient from pneumococcal pneumonia. This conclusion, which seemed illogical to many clinicians, was refuted in a later study, which demonstrated in this case an increase in the risk of death by 50%.
Ioannidis notes that, in addition to statistical aberrations, the universally practiced "creative" approach to the analysis of statistical data, which consists in applying various methods of analysis to obtain the desired or expected results, is of great importance. He believes that one of the possible methods of solving this problem is to create a system of checks and re-evaluations of each proposed association between a biomarker and a disease, and urges specialists to believe the data obtained during large-scale meta-analyses, and not the results of studies that first revealed a particular relationship.
The article Comparison of Effect Sizes Associated With Biomarkers Reported in Highly Cited Individual Articles and in Consequent Meta-analyses was published on June 1 in the Journal of the American Medical Association.
Evgeniya Ryabtseva
Portal "Eternal youth" www.vechnayamolodost.ru based on Stanford University School of Medicine: Association between biomarkers and disease often overstated, researcher finds.
03.06.2011