- Short communication
- Open Access
Diagnostic Utility of New SCAT5 Neurological Screen Sub-tests
Sports Medicine - Open volume 7, Article number: 14 (2021)
The Sports Concussion Assessment Tool (SCAT) is recommended to screen for concussion following head impact events in elite sport. The most recent 5th edition (SCAT5) included a ‘rapid neurological screen’ which introduced new subtests examining comprehension, passive neck movement, and diplopia. This study evaluated the additional diagnostic value of these new subtests.
A prospective cohort study was performed in the Pro14 elite Rugby Union competition between September 2018 and January 2020. The SCAT5 was administered by the team doctor to players undergoing off-field screening for concussion during a medical room assessment. Sensitivity, specificity, false negatives, and positives were examined for SCAT5 comprehension, passive neck movement, and diplopia subtests. The reference standard was a final diagnosis of concussion, established by serial standardised clinical assessments over 48 h.
Ninety-three players undergoing off-field screening for concussion were included. Sensitivity and specificity of the comprehension, passive neck movement, and diplopia subtests were 0, 8, 5% and 0, 91, 97%, respectively (concussion prevalence 63%). No players had any abnormality in comprehension. No players had abnormal passive neck movement or diplopia in the absence of abnormalities in other SCAT5 sub-components.
The new SCAT5 neurological screen subtests are normal in the majority of players undergoing off-field concussion screening and appear to lack diagnostic utility over and above other SCAT5 subtests.
The Sports Concussion Assessment Tool 5th edition (SCAT5) includes a ‘rapid neurological screen’ with new subtests examining comprehension, passive neck movement, and diplopia.
In elite Rugby Union players undergoing off-field concussion screening, no players had any abnormality in comprehension; or had diplopia, or painful passive neck movement, in the absence of abnormalities in other SCAT5 sub-components.
The neurological screen subtests did not affect the diagnostic accuracy of the SCAT5 and did not detect any additional concussed players, or independently result in any false-positive cases.
Concussion is a common injury in collision sports . Given the diverse array of presenting symptoms and signs that can occur following concussion, a standardised multi-modal diagnostic approach has been recommended . The International Consensus Conference for Concussion in Sport therefore developed the Sports Concussion Assessment Tool (SCAT) to screen for suspected concussion, standardise evaluation of sports-related concussion, track player recovery, and serve as a tool for player education .
Development of the SCAT has been an iterative process. The latest version, the SCAT5, included a ‘rapid neurological screen’, adding new subtests evaluating comprehension, passive neck movement and diplopia. Although the rationale for the new sub-tests was not fully elaborated, the authors state that they will ‘increase the utility of the tool’ . The performance of other SCAT components has been previously evaluated in detail ; however, there is an absence of evidence for the diagnostic accuracy of the newer neurological subtests in off-field screening for concussion .
This study therefore aimed to determine if the new SCAT5 neurological screen subtests are effective additions to the SCAT in off-field screening for sport-related concussion. Specific objectives were to describe the distribution of subtest results; evaluate diagnostic accuracy in Rugby Union players undergoing off-field concussion screening; and describe the additional value compared to other SCAT subtests.
Study Design and Population
A prospective cohort study was conducted in the male, adult, elite-level Rugby Union ‘PRO14’ competition players over two seasons between September 2018 and January 2020. The source population consisted of players from Scottish (2 teams), Irish (4), South African (2), Welsh (4), and Italian (2) elite level rugby union teams. The study population comprised players undergoing off-field concussion screening with the SCAT5 after identification of a meaningful head impact event with the potential to cause concussion during competitive PRO14 matches.
The World Rugby Head Injury Assessment (HIA) process for management of head impacts during competitive matches has been described previousl y. Briefly, players demonstrating clear signs of concussion (e.g. loss of consciousness, tonic posturing, ataxia, seizures) are immediately and permanently removed from the remainder of the match, without undergoing further off-field concussion screening. Players sustaining head impacts where the consequences were unclear, e.g. dangerous mechanism, undergo an off-field concussion screening assessment (termed the HIA-1 assessment). The assessment is conducted by the team doctor in a medical room during a 15-min player interchange. In this study, off-field screening was performed using the SCAT5 instrument, with results interpreted in comparison to pre-season player-specific baseline results. All players entering the HIA process receive repeat clinical assessments made by the team doctor post-match (termed the HIA-2 assessment) and after 48 h rest (termed the HIA-3 assessment), supported by cognitive assessments (typically a computerised neuro-cognitive tool such as CogSport) .
Index Tests and Reference Standard
The index tests under consideration were the component subtests of the new ‘rapid neurological screen’ in the SCAT5, performed during off-field concussion HIA-1 screening assessments, namely: comprehension (normal = ability to read aloud and follow instructions without difficulty), passive neck movement (normal = full range of pain-free passive cervical spine movement), and diplopia (normal = no diplopia in any plane of eye movement). Subtest values were interpreted in comparison to player-specific pre-season baseline SCAT5 results.
The reference standard, against which performance of each index test was compared, was a clinical diagnosis of concussion following completion of the HIA process.
HIA process data are routinely recorded at the point of assessment by physicians using a tablet based, web-hosted platform, with standardised data collection forms (CSx Systems, Auckland, New Zealand) . Any missing CSx outcome data was collected by direct communication with team doctors.
The analysis proceeded in three stages. Firstly, player flow through the study and demographic characteristics were described. Secondly, the proportion of abnormal index test results was described overall, and separately for concussed/non-concussed players. Thirdly, the sensitivity and specificity were calculated for each index test against the reference standard of a final clinical diagnosis of concussion. Finally, the additional value of each subtest, over and above other SCAT5 components, was investigated by examining extra false positives (non-concussed players with abnormal index test findings, but otherwise normal SCAT5 subtest results) and extra true positives (concussed players with abnormal index test finding, but otherwise normal SCAT5 subtest results).
Sample Size, Statistics, Ethics, and Funding
A census sample of head impact events undergoing off-field HIA-1 concussion screens over two PRO14 seasons were included. The width of confidence intervals indicates the precision of results. Statistical analyses were carried out in Stata version 13.1 (StataCorp., College Station, USA). Available case analyses were performed with a conventional significance level (α) of 0.05 used. The unit of analysis was consecutive significant head impact events, and individual players could be included in the sample more than once if subject to recurrent HIA-1 concussion screens. Ethical approval was provided for analysis of the data from the University of Sheffield. All players provided informed consent for use of data prior to the start of the season. All data were anonymised. The study was not funded.
A total of 119 head impact events, where the consequences were unclear, were detected in 107 players who underwent off-field concussion screening. Overall SCAT5 HIA-1 screening assessment results were recorded for all players, with outcome data absent in 8 players. SCAT5 sub-test results were unavailable for a further 22 players. A flow chart describing derivation of the study sample is presented in Fig. 1. The median age was 28 years (Interquartile range 25–31) with 79% of players having sustained a previous career concussion (median 2, interquartile range 1–3).
Overall, 64 players undergoing off-field screening were diagnosed with concussion, giving a prevalence of 57.6% (n = 111). Of these concussed players, 57 were correctly removed from play following their HIA-1 assessment. Of the non-concussed players, 38/47 were correctly returned to play. Sensitivity and specificity of the SCAT5 HIA-1 off-field screening assessment were therefore 89.1% (95%CI 78.8–95.5%) and 80.9% (66.7–90.9%, n = 111) respectively.
New neurological screen subtests were normal in the majority of players undergoing off-field concussion screening. No players had any abnormality in comprehension; passive neck movement was painful in 8 players (8.6%); and 4 cases had diplopia (4.3%, n = 93). There were no significant differences in the proportion of concussed v non-concussed players with painful passive neck movement or diplopia (p = 0.9 and 0.6, n = 93). Sensitivity and specificity of the comprehension, passive neck movement, and diplopia subtests were 0, 8, 5% and 0, 91, 97%, respectively (concussion prevalence 63.4% n = 93). Table 1 summarises the results and diagnostic accuracy of new SCAT5 neurological screen subtests.
No players’ painful passive neck movement or diplopia in the absence of abnormalities in other SCAT5 sub-components compared to baseline values. The passive neck movement and diplopia subtests therefore did not affect the diagnostic accuracy of the SCAT5, and did not detect any additional concussed players, or independently result in any false-positive cases.
New subtests included in the SCAT5 rapid neurological screen were normal in the majority of players undergoing off-field concussion screening. Sensitivity and specificity of the comprehension, passive neck movement, and diplopia subtests were 0, 8, 5% and 0, 91, 97%, respectively. The new neurological screen subtests did not affect the diagnostic accuracy of the SCAT5. No players had any abnormality in comprehension, or had diplopia, or painful passive neck movement, in the absence of abnormalities in other SCAT5 sub-components.
The SCAT5 is the fourth iteration of the Sports Concussion Assessment Tool . The original version had a ‘neurological screening’ domain, including subtests similarly examining speech and eye motion . However, these were subsequently removed from the SCAT2 and SCAT3 [8, 9], prior to re-introduction in the SCAT5 . ‘Neck pain’ has been part of the symptom checklist in all SCAT editions, with a neck examination introduced from the SCAT3 onwards . However, specific evaluation of full range of pain-free passive cervical spine movement is specific to the SCAT5.
Concussion is considered a subset of mild traumatic brain injury largely reflecting a functional disturbance of brain disturbance . The new SCAT5 neurological subtests of comprehension and diplopia are blunt examinations, which might be expected to be normal in the absence of other grossly abnormal neurological symptoms or signs. The findings that these subtests were largely normal in a population without overt signs of concussion, and only abnormal in conjunction with other SCAT5 subtest abnormalities, may therefore be unsurprising. Previous studies have demonstrated visual problems associated with traumatic brain injury and more detailed neuro-ophthalmological testing might have utility in off-field concussion screening .
Pain on passive neck movement is not a traditional neurological examination, and neck pain would not be expected in an isolated functional brain injury. However, neck examination could provide useful information during concussion screening. Concussion is commonly associated with concomitant neck injury including muscle strain, ligamentous sprains or, rarely, arterial dissection or bony injuries, which could confound the detection of concussion . Moreover, despite limited evidence, if the presence of a neck sprain is also predictive of concussion, passive neck movement could potentially be a useful screening test. However in this study, passive neck examination did not appear to add any value over and above other SCAT5 subtests.
The SCAT5 consist of multiple subtests applied concurrently. In this testing paradigm, as further subtests are added, sensitivity will increase and specificity fall. Ideally, to maximise diagnostic accuracy, the optimal combination of individual subtests would be chosen that individually demonstrate reasonable sensitivity to detect new cases of concussion over and above other subtests; but that also have satisfactory specificity to minimise false-positive cases. The performance of individual SCAT components for off-field concussion screening, and their optimal combination, has been examined previously . In the current study, the new comprehension, passive neck movement, and diplopia subtests demonstrated worse performance than other individual SCAT subtests, with no additional value. Their very low sensitivities (0, 8, and 5%, respectively) also indicate that they could not be substituted for other SCAT5 components. These results should be generalizable throughout Rugby and, as the subtests tests are relatively simple, are likely to have external validity in other elite sports. It is essential to note that ‘the diagnosis of concussion relies on a clinical synthesis of complex, non-specific and at times contradictory information’ . It has been shown previously that team doctors often use expert judgement when interpreting off-field screening tests results compared to baseline or normative thresholds . Despite not providing direct information, it is therefore possible that the new neurological subtests provide global information to inform an overall clinical assessment. Importantly, the SCAT5 is also used in other contexts, such as diagnosis or tracking recovery, and the new neurological subtests may be more useful in these applications, which were outside the scope of the current study.
This study has several limitations. Firstly, the sample size is relatively small, although the 95% confidence intervals for sensitivity and specificity are not consistent with a clinically significant ability of new subtests to discriminate concussed players. Secondly, there was missing data in some players which could have introduced selection bias. These were predominantly non-concussed cases with normal HIA-1 off-field screening results (n = 17/26), where the neurological sub-tests are highly likely to be normal, suggesting that our results are conservative and are unlikely to have underestimated sensitivity. Thirdly, team doctors administered the SCAT5 in both index test and reference standard assessments, there is a risk of incorporation and diagnostic review bias. Finally, there are no convincing and objective gold standard criteria for the diagnosis of concussion which could lead to outcome misclassification and information bias.
The new SCAT5 neurological screen subtests are normal in the majority of players undergoing off-field concussion screening and appear to lack diagnostic utility over and above other SCAT5 subtests. If corroborated, our findings suggest that these components might be safely omitted from side-line concussion screening assessments.
Availability of Data and Materials
The datasets generated during and/or analysed during the current study are not publicly available due to the terms of athlete consent and ethical approvals. However, anonymised data may be available from the corresponding author on reasonable request.
Harmon KG, Drezner JA, Gammons M, et al. American Medical Society for Sports Medicine position statement: concussion in sport [published correction appears in Br J Sports Med. 2013 Feb;47(3):184]. Br J Sports Med. 2013;47(1):15–26.
Echemendia RJ, Meeuwisse W, McCrory P, et al. The Sport Concussion Assessment Tool 5th Edition (SCAT5): Background and rationale. Br J Sports Med. 2017;51(11):848–50.
Fuller GW, Tucker R, Starling L, Falvey E, Douglas M, Raftery M. The performance of the World Rugby Head Injury Assessment Screening Tool: a diagnostic accuracy study. Sports Med Open. 2020;6(1):2.
Patricios J, Fuller GW, Ellenbogen R, et al. What are the critical elements of sideline screening that can be used to establish the diagnosis of concussion? A systematic review. Br J Sports Med. 2017;51(11):888–94.
Collie A, Maruff P, Makdissi M, McCrory P, McStephen M, Darby D. CogSport: reliability and correlation with conventional cognitive tests used in postconcussion medical evaluations. Clin J Sport Med. 2003;13(1):28–32.
CSX; 2020. Available at: http://csx.co.nz/. Accessed 3 Feb 2020.
McCrory P, Johnston K, Meeuwisse W, et al. Summary and agreement statement of the 2nd International Conference on Concussion in Sport, Prague 2004. Br J Sports Med. 2005;39(4):196–204.
McCrory P, Meeuwisse W, Johnston K, et al. Consensus statement on concussion in sport—the Third International Conference on Concussion in Sport held in Zurich, November 2008. Phys Sportsmed. 2009;37:141–59.
Guskiewicz KM, Register-Mihalik J, McCrory P, et al. Evidence-based approach to revising the SCAT2: introducing the SCAT3. Br J Sports Med. 2013;47:289–93.
Ventura RE, Balcer LJ, Galetta SL. The neuro-ophthalmology of head trauma. Lancet Neurol. 2014;13(10):1006–16.
Cheever K, Kawata K, Tierney R, Galgon A. Cervical injury assessments for concussion evaluation: a review. J Athl Train. 2016;51(12):1037–44.
We would like to acknowledge and sincerely thank the PRO14 team doctors, medical practitioners, and CSx for their help in facilitating collection of Head Injury Assessment data.
No financial support was received for the research or authorship. World Rugby funded publication of the article.
Ethics Approval and Consent to Participate
All research procedures were in accordance with the ethical standards of World Rugby, University of Sheffield, and the 1964 Helsinki Declaration and its later amendments. Ethical approval was provided for analysis of the data from the University of Sheffield. All players provided informed consent for use of data prior to the start of the season.
Consent for Publication
Ross Tucker, Eanna Falvey, Marc Douglas, and Martin Raftery are employed by World Rugby. Jon Miles and Prabhat Mathema are employed by Wales Rugby Union. Gordon Fuller has received travel expenses to attend scientific conferences from World Rugby and was a member of the expert panel developing the SCAT5 concussion assessment tool. The authors, Gordon Ward Fuller, John Miles, Ross Tucker, Marc Douglas, Martin Raftery, Eanna Falvey and Prabhat Mathema declare that they have no other conflicts of interest.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Fuller, G.W., Miles, J., Tucker, R. et al. Diagnostic Utility of New SCAT5 Neurological Screen Sub-tests. Sports Med - Open 7, 14 (2021). https://doi.org/10.1186/s40798-021-00303-z
- Brain Concussion
- Diagnostic accuracy