Comment on: “Is There Evidence for the Development of Sex-Specific Guidelines for Ultramarathon Coaches and Athletes? A Systematic Review”
Sports Medicine - Open volume 9, Article number: 81 (2023)
We read with great interest the work done by Kelly , who conducted a systematic review (SR) that aimed to review the current knowledge of sex differences in ultramarathon runners and determine if sufficient evidence exists for providing separate guidelines for males and females. First, we want to compliment the author on the relevance of this investigation. In this letter, concerns are raised regarding the methodological aspects of the review, which may have led to misleading conclusions.
The author declared that the review was conducted following the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA)-Protocol guidelines . At this point, some mistakes in writing the review results may have been committed since the reference used concerns the writing of SR protocols  and not the SR itself .
In addition, critical principles of the guidelines for conducting SR were violated:
Despite the author mentioning in the methodology that “The methods were specified in advance and documented in a detailed protocol,” we identified that the SR protocol was not previously registered in a database (e.g., International Prospective Register of Systematic Reviews [PROSPERO]), which reduces the transparency and reproducibility of the study.
Only one reviewer conducted the review, which is at odds with the recommendations of having at least two independent-blinded reviewers between identification and eligibility steps . In this sense, the internal validity of the review is reduced due to possible selection bias of studies .
The inclusion and exclusion criteria are outlined according to the specifications that were used to find the selected studies. However, the study flowchart (Figure 1 in the article ) only reports the excluded studies as “Ineligible outcome” and “Ineligible population.” It is unclear to the reader why the author excluded such studies, compromising the transparency of how they were included/excluded from the study.
The research question based on the P-population/I-intervention/C-comparison/O-outcome (PICO) model is inaccurate. In addition, the types of studies included in the review were erroneously described in the acronym I-intervention. At this point, the author could create the acronym S-Study Design to provide specific information about the observational studies that will be included in the SR (e.g., cross-sectional/cohort).
The use of the Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) framework is worrying, as some criteria for classifying the certainty of the evidence need to classify the magnitude of the effect of the intervention and the assessment of the risk of bias in the primary studies . Given that most of the included studies are observational and the review does not present a meta-analysis calculation, it is not possible to understand how the reviewer classified the evidence certainty of the primary studies. At this point, no tool for ranking the risk of bias of the primary studies was used, which makes it unclear what quality of evidence the SR conclusions were based on.
In conclusion, it was suggested that sex-specific recommendations for female ultramarathon runners might improve health and performance outcomes. However, the author reports that the evidence base found in the review  was of poor quality, making it challenging to infer conclusions regarding the potential benefits of sex-specific approaches to ultramarathon training, racing, nutrition, and recovery.
Due to the lack of methodology information about the questions mentioned above, we invite the author to provide transparent information about that.
Availability of Data and Materials
Preferred Reporting Items for Systematic Reviews and Meta-Analysis
International Prospective Register of Systematic Reviews
Grading of Recommendations, Assessment, Development, and Evaluations
Kelly CPMG. Is there evidence for the development of sex-specific guidelines for ultramarathon coaches and athletes? A systematic review. Sports Med Open. 2023;9:6.
Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4:1.
Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372: n71.
Charrois TL. Systematic reviews: what do you need to know to get started? Can J Hosp Pharm. 2015;68:144–8.
Higgins J, Thomas J, Chandler J, Cumpston M, Li T, Page M, et al. Cochrane handbook for systematic reviews of interventions. Cochrane Handbook for Systematic Reviews of Interventions; 2019.
Drucker AM, Fleming P, Chan A-W. Research techniques made simple: assessing risk of bias in systematic reviews. J Invest Dermatol. 2016;136:e109–14.
We thank the scholarship offered by Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP) (scholarship number 2022/07280-8), Coordenação de Aperfeiçoamento de Pessoal de Nível Superior—Brazil (CAPES)—Finance Code 001, and by the Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq—Brazil/ Process Number: 200595/2022-9).
No sources of funding were used to assist in the preparation of this letter.
Ethics Approval and Consent to Participate
Consent for Publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
da Silva, L.S.L., Benjamim, C.J.R. Comment on: “Is There Evidence for the Development of Sex-Specific Guidelines for Ultramarathon Coaches and Athletes? A Systematic Review”. Sports Med - Open 9, 81 (2023). https://doi.org/10.1186/s40798-023-00615-2