Skip to main content
  • Letter to the Editor
  • Open access
  • Published:

Comment on: “Is There Evidence for the Development of Sex-Specific Guidelines for Ultramarathon Coaches and Athletes? A Systematic Review”

A Reply to this article was published on 08 September 2023

The Original Article was published on 25 January 2023

Dear Editor,

We read with great interest the work done by Kelly [1], who conducted a systematic review (SR) that aimed to review the current knowledge of sex differences in ultramarathon runners and determine if sufficient evidence exists for providing separate guidelines for males and females. First, we want to compliment the author on the relevance of this investigation. In this letter, concerns are raised regarding the methodological aspects of the review, which may have led to misleading conclusions.

The author declared that the review was conducted following the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA)-Protocol guidelines [2]. At this point, some mistakes in writing the review results may have been committed since the reference used concerns the writing of SR protocols [2] and not the SR itself [3].

In addition, critical principles of the guidelines for conducting SR were violated:

Despite the author mentioning in the methodology that “The methods were specified in advance and documented in a detailed protocol,” we identified that the SR protocol was not previously registered in a database (e.g., International Prospective Register of Systematic Reviews [PROSPERO]), which reduces the transparency and reproducibility of the study.

Only one reviewer conducted the review, which is at odds with the recommendations of having at least two independent-blinded reviewers between identification and eligibility steps [4]. In this sense, the internal validity of the review is reduced due to possible selection bias of studies [5].

The inclusion and exclusion criteria are outlined according to the specifications that were used to find the selected studies. However, the study flowchart (Figure 1 in the article [1]) only reports the excluded studies as “Ineligible outcome” and “Ineligible population.” It is unclear to the reader why the author excluded such studies, compromising the transparency of how they were included/excluded from the study.

The research question based on the P-population/I-intervention/C-comparison/O-outcome (PICO) model is inaccurate. In addition, the types of studies included in the review were erroneously described in the acronym I-intervention. At this point, the author could create the acronym S-Study Design to provide specific information about the observational studies that will be included in the SR (e.g., cross-sectional/cohort).

The use of the Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) framework is worrying, as some criteria for classifying the certainty of the evidence need to classify the magnitude of the effect of the intervention and the assessment of the risk of bias in the primary studies [6]. Given that most of the included studies are observational and the review does not present a meta-analysis calculation, it is not possible to understand how the reviewer classified the evidence certainty of the primary studies. At this point, no tool for ranking the risk of bias of the primary studies was used, which makes it unclear what quality of evidence the SR conclusions were based on.

In conclusion, it was suggested that sex-specific recommendations for female ultramarathon runners might improve health and performance outcomes. However, the author reports that the evidence base found in the review [1] was of poor quality, making it challenging to infer conclusions regarding the potential benefits of sex-specific approaches to ultramarathon training, racing, nutrition, and recovery.

Due to the lack of methodology information about the questions mentioned above, we invite the author to provide transparent information about that.

Availability of Data and Materials

Not applicable.



Systematic review


Preferred Reporting Items for Systematic Reviews and Meta-Analysis


International Prospective Register of Systematic Reviews




Grading of Recommendations, Assessment, Development, and Evaluations


  1. Kelly CPMG. Is there evidence for the development of sex-specific guidelines for ultramarathon coaches and athletes? A systematic review. Sports Med Open. 2023;9:6.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4:1.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372: n71.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Charrois TL. Systematic reviews: what do you need to know to get started? Can J Hosp Pharm. 2015;68:144–8.

    PubMed  PubMed Central  Google Scholar 

  5. Higgins J, Thomas J, Chandler J, Cumpston M, Li T, Page M, et al. Cochrane handbook for systematic reviews of interventions. Cochrane Handbook for Systematic Reviews of Interventions; 2019.

  6. Drucker AM, Fleming P, Chan A-W. Research techniques made simple: assessing risk of bias in systematic reviews. J Invest Dermatol. 2016;136:e109–14.

    Article  CAS  PubMed  Google Scholar 

Download references


We thank the scholarship offered by Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP) (scholarship number 2022/07280-8), Coordenação de Aperfeiçoamento de Pessoal de Nível Superior—Brazil (CAPES)—Finance Code 001, and by the Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq—Brazil/ Process Number: 200595/2022-9).


No sources of funding were used to assist in the preparation of this letter.

Author information

Authors and Affiliations



The authors read and approved the final manuscript.

Corresponding author

Correspondence to Leonardo Santos Lopes da Silva.

Ethics declarations

Ethics Approval and Consent to Participate

Not applicable.

Consent for Publication

Not applicable.

Competing Interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

da Silva, L.S.L., Benjamim, C.J.R. Comment on: “Is There Evidence for the Development of Sex-Specific Guidelines for Ultramarathon Coaches and Athletes? A Systematic Review”. Sports Med - Open 9, 81 (2023).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: