Evidence of Concussion Signs in National Rugby League Match Play: a Video Review and Validation Study
© The Author(s). 2017
Received: 12 February 2017
Accepted: 11 August 2017
Published: 22 August 2017
Many professional sports have introduced sideline video review to help recognise concussions. The reliability and validity of identifying clinical and observable signs of concussion using video analysis has not been extensively explored. This study examined the reliability and validity of clinical signs of concussion using video analysis in the National Rugby League (NRL).
All 201 professional NRL matches from the 2014 season were reviewed to document six signs of possible concussion (unresponsiveness, slow to get up, clutching/shaking head, gait ataxia, vacant stare, and seizure).
A total of 127,062 tackles were reviewed. Getting up slowly was the most common observable sign (2240 times in the season, 1.8% of all tackles) but only 223 times where it appeared to be a possible concussion (0.2% of all tackles and 10.0% of the times it occurred). Additionally, clutching/shaking the head occurred 361 times (on 212 occasions this sign appeared to be due to a possible concussion), gait ataxia was observed 102 times, a vacant stare was noted 98 times, unresponsiveness 52 times, and a possible seizure 4 times. On 383 occasions, one or more of the observable signs were identified and deemed associated with a possible concussion. There were 175 incidences in which a player appeared to demonstrate two or more concussion signs, and 54 incidences where a player appeared to demonstrate three or more concussion signs. A total of 60 diagnosed concussions occurred, and the concussion interchange rule was activated 167 times. Intra-rater reliability (κ = 0.65–1.00) was moderate to perfect for all six video signs; however, the inter-rater reliability was not as strong (κ = 0.22–0.76). Most of the signs had relatively low sensitivity (0.18–0.75), but high specificity (0.85–1.00).
Using video replay, observable signs of concussion appear to be sensitive to concussion diagnoses when reviewing known injuries among professional rugby league players. When reviewing an entire season, however, certain signs occur very commonly and did not identify concussion. Thus, the implementation of video review in the NRL is challenging, but can provide a useful addition to sideline concussion identification and removal from play decisions.
KeywordsConcussion Video analysis Injury management Return to play
The addition of video review to the assessment of concussion injury events may help improve consistency in the management of players, as well as assist in diagnostic decision-making in cases where signs may be transient and resolved by the time of the medical assessment.
The signs of concussion appear to be quite sensitive to concussion when reviewing known injuries; however, when reviewing an entire season, some signs occur very commonly and usually do not reflect a concussive injury.
Most signs of concussion had high specificity but low sensitivity when examining all tackles across a sporting season.
Participation in many full contact and collision sports, such as rugby league, carries with it a risk of concussion . In-game concussion diagnosis, however, remains a highly challenging task for the athletic trainer and sports medicine physician. On-field or sideline clinical assessments can be challenging due to the heterogeneous presentation of an athlete following a head impact, the non-specific nature of many of the clinical signs and symptoms of concussion , the absence of a reliable concussion biomarker , and the regularity with which some concussion signs emerge and evolve over time . Recognising a potential concussion and removing an athlete from play is understood to be an important intervention for reducing the risk of a worse clinical outcome following injury . However, it is acknowledged that in some instances, concussions may be missed from the sideline . This may occur for a variety of reasons, but commonly the transient early physical signs may resolve before the player can be removed from play and assessed .
Some prior studies suggest that worse outcomes following concussion are associated with on-field signs and symptoms, such as loss of consciousness , amnesia [8, 28], mental status change for more than 5 min , and dizziness . It is important to appreciate that the literature on the association between on-field signs and symptoms is mixed. For example, loss of consciousness has been associated with worse clinical outcomes in some [1, 4, 15, 28, 35, 39], but not in most studies [2, 3, 5, 8, 9, 16, 18, 22, 27, 30, 32, 36–38, 42, 43]. The vast majority of studies examining loss of consciousness base this finding on a questionnaire or interview completed with the athlete, not video review of the injury event for confirmation. Similarly, post-traumatic amnesia has been associated with worse clinical outcomes in some [15, 24, 28], but not in most studies [1–3, 8, 16, 18, 23, 27, 30, 34, 37, 38, 42]. Dizziness has been observed as an on-field symptom associated with a protracted recovery of greater than 21 days (6.34 time more likely) , but assessing dizziness is subjective and may or may not manifest as an objective sign (e.g., gait ataxia). Thus, video review may allow for the quantification of objective concussion signs, but not subjective symptoms.
In the sport of rugby league, the concussion incidence rates have been reported to vary widely depending on the level of competition . In one study of three National Rugby League (NRL) clubs, a concussion incidence rate of 14.8 concussions per 1000 player match hours was reported , while a rate of 28.3 concussion per 1000 player match hours were reported from one NRL club over a 15-year (1998–2012) period .
The use of video footage on the sideline for reviewing a concussion has been introduced in a number of professional sports as a method to improve the recognition of a possible concussion that may be missed by on-field medical personnel . Video studies have now been conducted in a variety of sports such as rugby league [11–13], rugby union , hockey , and Australian rules football [6, 25, 26]. In addition to the introduction of sideline video review, the governing body of the sport in Australia (the National Rugby League) implemented a new the “concussion interchange rule” (CIR). The CIR requires the mandatory removal of any player suspected of having sustained a concussion. The CIR allows a player to be removed from play for 15 min to be assessed by the club medical officer, including completing the Sports Concussion Assessment Tool third edition (SCAT-3). Following the assessment, if the player was not diagnosed with concussion, they are permitted to return to play without using an interchange. The incidence of use of the CIR was 24.0 (95% CI 20.7–27.9) uses of the CIR per 1000 NRL player match hours , and 44.9 (95% CI 38.5–52.3) uses of the CIR per 1000 National Youth Competition player match hours .
The primary aim of this study was to determine the rate of six objective concussion signs that occurred during the 2014 NRL season, as well as the sensitivity and the specificity of these signs to classify whether or not a diagnosed concussion resulted from the event. The secondary objective of this study was to analyse the intra- and inter-rater reliability of these signs using the video recordings of each incident that activated the CIR.
Participants for this study were the entire league of NRL players involved in match play during the 2014 season. Each team is comprised of a squad of approximately 25 players, and there are 16 teams in the league. There are 13 players on the field during the match.
This study conducted a video analysis of every game during the 2014 National Rugby League (NRL) season. The full game video analysis for all games during the season was conducted retrospectively, at the conclusion of the season by the lead author. The video review was conducted independent of any club trainer or team medical staff. The video was used to examine all on-field signs. There was no prerequisite for recording the presence of a sign (e.g., contact with the head was not required). The lead researcher obtained information on the use of the CIR (i.e., the player’s name and round in which the CIR was used). Each NRL team physician provided a list of players who had been diagnosed with concussion during the season. The club physician made the final diagnosis of concussion, based on conventional clinical examination techniques. For 162/167 (97%) uses of the CIR , and for all diagnosed concussions (n = 60), the Observational Review and Analysis of Concussion (ORAC) form  was used to record the game circumstances and details of the event as well as the presence or absence of six concussive signs (clutching or shaking head, slow to get up, gait ataxia, blank/vacant stare, unresponsiveness, and seizure). There were five uses of the CIR where the video of the incident was not located; none of these cases were diagnosed with a concussion. There were 162 (97%) cases where the CIR incident was identified, and 60 (100%) cases were medically diagnosed with a concussion. Two reviewers conducted the video analysis of these cases. To assess intra-rater reliability, one of the reviewers also completed a second review of 162 video clips of uses of the CIR, which included a review of all cases of medically diagnosed concussion.
All diagnosed concussions and 97% of the uses of the CIR were independently reviewed by two raters. The video raters were blinded to the club medical staff’s documentation (i.e., they conducted their retrospective review of the video signs completely independently of the club medical staff). Both raters have experience in the identification of concussion on the sideline and with retrospective video review. The two raters determined whether any of six signs of concussion were present, absent, or indeterminable based on the available footage of the incident for every case. When there was disagreement between the two raters, both raters reviewed and discussed those cases in an effort to reach consensus. In the cases where consensus could not be achieved, ratings from an experienced third rater were used. Consent was obtained from all participants who used the CIR. This study was approved by the University of Newcastle Human Ethics Committee (H-2012-0344).
Definition of video signs
Clutch or shake head
The player holds his head or face in the palm of his hand or hands, or the player rubs or shakes his head in a manner that appears to demonstrate they are experiencing discomfort (i.e., not just wiping his face).
Slow to return to feet/play
Player took longer than usual to return to his feet (e.g., remained on the ground, got to his knees or to his haunches, and waited momentarily before standing), in the absence of another player holding him down.
Gait ataxia (wobbly legs)
Unable to stand steadily unaided or walk normally. Appears “clumsy”. Upon standing and walking the player has unsteadiness, wobbly legs, balance problems, stumbles or falls over, drags his feet, or cannot walk straight independently.
The player is not visually focused on doctor/trainer when being spoken to, or assessed and asked to attend, and/or the player appears to be looking off into the distance. The player’s face is expressionless.
Evidence of unresponsiveness
The player’s body goes limp/floppy, the player loses control of muscles, the player does not protect (i.e., brace) himself when falling (e.g., ragdoll-like appearance). The player remains/lies motionless on the ground for a period of time longer than expected, or the player shows a lack of visible responsiveness to verbal stimuli.
Tonic posturing—stiffening of limbs; or clonic movements—involuntary repetitive contraction and relaxation of the muscles, could include jerky movements, or shaking/convulsion of upper and/or lower limbs, or body.
Descriptive analyses are presented as means (standard deviation) for continuous variables; categorical variables are presented as numbers included (n) or percentage of the total. Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated for each of the six concussion signs using 2 × 2 contingency tables. The number of times a player demonstrated one or more of these signs that looked like a possible concussion sign (n = 383) during match play for the entire season was used as the denominator for calculating sensitivity, specificity, PPV, and NPV.
Intra- and inter-rater reliability analyses using Cohen’s kappa (κ) statistics  were used for 162 (97%) cases where the CIR was used and for all of those cases that were medically diagnosed with concussion, to determine consistency among the two raters for each of the six individual signs. Unlike the total percent agreement, Cohen’s kappa considers the proportional agreement that could occur simply by chance. The κ coefficients are calculated by considering the proportion of rater agreement and the expected proportion . Using the interpretations of κ described by McHugh , κ agreement was categorised as almost perfect (>.90), strong (.80–.90), moderate (.60–.79), weak (.40–.59), minimal (.21–.39), and none (0–.20). All analyses were performed using IBM SPSS Statistics V.23.0  and used two-sided tests for significance at the 0.05 level, with either 90 or 95% confidence intervals (CIs).
Data summary of video analysis from the 2014 National Rugby League season
Total number of tackles in the season
Average number of tackles per match
Average concussion interchange rule user per match
Concussion interchange rule (CIR) uses
Returned to play
No return to play
CIR, no return to play, and no concussion
Summary of the rates of concussion signs for the entire season, concussion-related signs, use of the CIR, and medically diagnosed concussions
Entire season total
Possible concussion sign
Use of the CIR
CIR missing dataa
Medically Diagnosed concussion
Medically Diagnosed concussion missing data
(n = 162)a
(n = 60)
Slow to get up
Vacant stare b
Rates, sensitivity, specificity, positive predictive value, and negative predictive value for concussion signs
Medically Dx concussion with sign
Medically Dx concussion without sign
Clutching or shaking head
Slow to get up
Blank or vacant stare
Summary of video analysis findings for those situations in which the concussion interchange rule was used
Signs of concussion
Medically diagnosed concussion (n = 60)
No diagnosed concussion and no return to playa (n = 16)
Returned to playa (n = 80)
Diagnosed concussion vs. returned to playa
RR (90% CI)
Slow to get up
Total signs observed (n)
The players who were medically diagnosed with a concussion were significantly more likely (1.52 times) to exhibit unresponsiveness and a vacant stare (2.35 times) than players who were removed from play under the CIR, not diagnosed with a concussion, and were returned to play. Clutching the head and gait ataxia were not significantly different signs between those who were and were not medically diagnosed with a concussion (see Table 1).
A sign that was always present in cases of diagnosed concussion was slow to get up (sensitivity = 100%), although it had low specificity (50%). A possible seizure was observed only four times during the season and on three occasions those athletes were medically diagnosed as having a concussion (3/60 diagnosed concussions; sensitivity = 5%, specificity = 100%). A blank or vacant stare had fairly high sensitivity and specificity (75% and 84%, respectively; see Table 4).
Intra-rater reliability and inter-rater reliability for the concussion signs
Intra-rater (κ) (95% CIs)
κ agreement classification
Inter-rater (κ) (95% CIs)
κ agreement classification
Absolute disagreement (CIR cases, n = 162)
Clutch of shake head
Slow to get up
To expand previous video analysis work in collision sports, this study explored the rate of six observable signs of concussion, as well as their sensitivity and specificity during match play of a National Rugby League season. The results of the current study revealed moderate to perfect intra-rater reliability or better for all video signs. For inter-rater reliability, clutching or shaking head, slow to get up, and unresponsiveness had moderate inter-rater reliability, whereas post-impact seizure had weak inter-rater reliability, and gait ataxia and blank or vacant stare had poor inter-rater reliability. These findings are relatively consistent with those reported from a video review study of concussion signs during Australia Football League (AFL) match play, which found that intra-rater reliability was generally better than inter-rater reliability, and blank or vacant stare was the sign with the lowest agreement .
Blank or vacant stare was the only sign that had reasonably high sensitivity (75%) and specificity (84%); all other signs did not have both high sensitivity and specificity for a concussion diagnosis. Some of the signs had high sensitivity but low specificity, suggesting that those signs may be a useful marker in flagging a potential concussion, but not helpful in confirming the diagnosis. Therefore, medical personnel working in tandem on the sideline and with individuals reviewing video may be an appropriate strategy to achieve a sensitive and specific approach to concussion diagnosis. In a video review of concussion signs during match play in the Australian Football League (AFL), slow to get up had the highest sensitivity (87%), but low specificity (19%). All other video concussion signs examined in this prior study had high specificity but low sensitivity, such as blank and vacant look [specificity 100%, sensitivity 9%], motor incoordination [specificity 95%, sensitivity 29%], impact seizure [specificity 93%, sensitivity 7%], and rag doll appearance [specificity 91%, sensitivity 16%]). The highest PPV was found with the “blank and vacant look” (100%), and “motor incoordination” (81%) signs . However, in the current study, we found that the highest PPV was for seizure (75%), while slow to get up (100%), blank or vacant stare (95%), and gait ataxia (91%) were the signs with the highest NPV.
The sign with the highest relative risk of diagnosed concussion was a blank or vacant stare (2.35 times more likely; 90% CI = 1.40–4.30), while unresponsiveness was 1.52 times more likely (90% CI = 1.06–2.10). Gait ataxia (1.46 times more likely; 90% CI = 0.94–2.38) and clutching or shaking head (0.77 times more likely; 90% CI = 0.55–1.12) were not statistically different. Blank or vacant stare appears to be a conceptually different sign to the other five. The detection of a blank or vacant stare tends to be through a secondary screening process, in that it is often only observed if, and when, a player is injured and the camera zooms in closely. It is also arguably the most subjective of the six signs and therefore vulnerable to greater variation between raters when classifying this sign. This notion is supported by the weaker inter-rater reliability of this sign not only in the current study but also in previous video review studies [12, 13, 25]. In our study, the low inter-rater reliability was due to one rater seeing the sign more often than the other, recording it as present, and then both raters ultimately agreeing that it was present. Observing a blank or vacant stare was rare in a recently published NHL video review study . We do not know why the sign was so uncommon in that study compared to ours. It might be more difficult to see the sign through a hockey helmet, and it might also be more difficult to see it based on camera angles when players are moving off the ice. Among the other signs, inter-rater reliability was the strongest for clutch or shake head, slow to get up, and unresponsiveness, each with a moderate agreement classification. These signs may be more easily seen during field-side evaluations; their intra-rater reliability was strong to perfect. In contrast, the minimal agreement between raters for gait ataxia or blank/vacant stare warrants caution with their use.
There were several limitations of the current study. In some instances, the available video footage was not sufficiently clear to code all signs (i.e., the view from the available camera angle was obscured, or a close up of the incident was not available). This ‘missing data’ was excluded from the analyses, which might have slightly improved the support for the utility of some of the visible signs. This was particularly true for the blank or vacant stare sign, where missing data on this sign was common. Despite being the most sensitive and specific sign for concussion diagnosis, blank or vacant stare had the worst inter-rater reliability (0.22, with 71 absolute disagreements between raters). For these reasons, if video review within professional sports are implemented, then access to high quality reviews with the capability of multi-angle and slow motion replays to allow for close ups would be optimal  and would reduce the likelihood of missing data. Although the video reviewer was blinded to the sideline assessment results and the medical diagnosis of concussion for this study, they were only partially blinded to the use of the CIR. Given that the process for enacting a use of this rule requires the trainer to provide a signal to the sideline, and the official on the sideline identifies the interchange with a green card, the video reviewer was able to identify many instances where the CIR was used. Another study limitation was that there were no concussions that were subsequently (beyond match day) diagnosed by the club medical and reported to the researchers, which is inconsistent with other video review studies that have reported a numbers of cases of post-game diagnosis of concussion . A further limitation of the current study pertains to the generalizability of the current findings to other levels of rugby league. The current study was a post-game review of an adult, male, professional league, and as such the results may not necessarily be generalizable to in-match real-time analysis, or other levels of match play. Our use of an operational definition of exclusion to categorise the observed signs into ‘plausible concussion signs’ versus signs that were more likely attributable to other factors, was also a limitation in terms of its subjectivity and reproducibility for future work in video review of concussion signs. Finally, only one reviewer completed the coding of the entire game, for every game in the season; the inter-rater reliability of that type of coding is unknown.
A conservative approach to sideline concussion management would be to remove a player from play based on any evidence of possible concussion signs. This approach is encouraged as the best management strategy and in the best interest of the welfare of the player . However, there is limited information available on the reliability and validity of identifying the objective signs of concussion when using video analysis, and indeed not all instances of observed concussion signs occur as a result of the player having sustained a concussion. The signs of concussion appear to be quite sensitive to concussion when reviewing known (i.e., medically diagnosed concussions) or suspected (i.e., players who have used the CIR) injury. When reviewing an entire season of match play, specific objective concussion signs such as slowness in getting up and clutching/shaking the head occurred commonly during professional rugby league match play, but did not typically reflect concussion occurrence. For these reasons, video injury surveillance can be difficult to interpret, but may provide a useful adjunct to the clinical assessment of potential concussion. With improved access to video replays, clear definitions and education regarding the observable signs, and improved communication between video observers and sideline medical personnel, the detection of concussion may improve.
The authors would like to thank Dr Jinho Lee (School of Human Movement and Nutrition Sciences, University of Queensland) for completing the video review of all uses of the CIR.
Funding was provided by Hunter Medical Research Institute (HMRI) supported by Anne Greaves.
Availability of data and materials
AG conceived the design of the study, reviewed all game video, collected all of the study data, and drafted, revised, and finalised the manuscript. He had final veto on the submission. DH assisted with the design of the study, conducted the statistical analysis, provided expert editorial comment for all drafts of the manuscript, and had final veto on the submission. CL assisted with the design of the study and provided editorial comment to the final draft of the manuscript. GI assisted with the design of the study, assisted with the statistical analysis of the data, provided expert editorial comment for all drafts of the manuscript, and had final veto on the submission. All authors read and approved the final manuscript.
Ethics approval and consent to participate
The study was approved by the University of Newcastle Human Ethics Committee (Ref No. H-2012-0344). Consent to participant was provided by all participants.
Consent for publication
Andrew Gardner has a clinical practice in neuropsychology involving individuals who have sustained sport-related concussion (including current and former athletes). He has operated as a contracted concussion consultant to the Australian Rugby Union (ARU) from July 2016. He has received travel funding from the Australian Football League (AFL) to present at the Concussion in Football Conference in 2013 and 2017. Previous grant funding includes the NSW Sporting Injuries Committee, the Brain Foundation (Australia), and the Hunter Medical Research Institute (HMRI), supported by Jennie Thomas. He is currently funded through the HMRI, supported by Anne Greaves, and the University of Newcastle’s Priority Research Centre for Stroke and Brain Injury.
David Howell and Christopher Levi have no competing interests to declare.
Grant Iverson has been reimbursed by the government, professional scientific bodies, and commercial organisations for discussing or presenting research relating to mild TBI and sport-related concussion at meetings, scientific conferences, and symposiums. He has a clinical and consulting practice in forensic neuropsychology involving individuals who have sustained mild TBIs (including athletes). He has received research funding from several test publishing companies, including ImPACT Applications, Inc., CNS Vital Signs, and Psychological Assessment Resources (PAR, Inc.). He acknowledges unrestricted philanthropic support from the Mooney-Reed Charitable Foundation and ImPACT Applications, Inc.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
- Asplund CA, McKeag DB, Olsen CH. Sport-related concussion: factors associated with prolonged return to play. Clin J Sport Med. 2004;14:339–43.View ArticlePubMedGoogle Scholar
- Bock S, Grim R, Barron TF, Wagenheim A, Hu YE, Hendell M, Deitch J, Deibert E. Factors associated with delayed recovery in athletes with concussion treated at a pediatric neurology concussion clinic. Childs Nerv Syst. 2015;31:2111–6.View ArticlePubMedGoogle Scholar
- Brown NJ, Mannix RC, O’Brien MJ, Gostine D, Collins MW, Meehan WP. Effect of cognitive activity level on duration of post-concussion symptoms. Pediatrics. 2014;133:e299–304.View ArticlePubMedPubMed CentralGoogle Scholar
- Chermann JF, Klouche S, Savigny A, Lefevre N, Herman S, Bohu Y. Return to rugby after brain concussion: a prospective study in 35 high level rugby players. Asian J Sports Med. 2014;5:e24042.View ArticlePubMedPubMed CentralGoogle Scholar
- Collins MW, Iverson GL, Lovell MR, McKeag DB, Norwig J, Maroon J. On-field predictors of neuropsychological and symptom deficit following sports-related concussion. Clin J Sport Med. 2003;13:222–9.View ArticlePubMedGoogle Scholar
- Davis G, Makdissi M. Use of video to facilitate sideline concussion diagnosis and management decision-making. J Sci Med Sport 2016; online first.Google Scholar
- Echemendia RJ, Bruce JM, Meeuwisse W, Hutchison MG, Comper P, Aubry M. Can visible signs predict concussion diagnosis in the National Hockey League? Br J Sports Med 2017; bjsports-2016-097090.Google Scholar
- Ellis MJ, Ritchie LJ, Koltek M, Hosain S, Cordingley D, Chu S, Selci E, Leiter J, Russell K. Psychiatric outcome after pediatric sports-related concussion. J Neurosurg Pediatr. 2015;16:709–18.View ArticlePubMedGoogle Scholar
- Erlanger D, Kaushik T, Cantu R, Barth JT, Broshek DK, Freeman JR, Webbe FM. Symptom-based assessment of the severity of a concussion. J Neurosurg. 2003;98:477–84.View ArticlePubMedGoogle Scholar
- Gardner A, Iverson GL, Levi CR, Schofield PW, Kay-Lambkin F, Kohler RMN, Stanwell P. A systematic review of concussion in rugby league. Br J Sports Med 2014; bjsports-2013-093102.Google Scholar
- Gardner AJ, Iverson GL, Quinn TN, Makdissi M, Levi CR, Shultz SR, Wright DK, Stanwell P. A preliminary video analysis of concussion in the national rugby league. Brain Inj. 2015;29:1182–5.View ArticleGoogle Scholar
- Gardner AJ, Iverson GL, Stanwell P, Ellis J, Levi CR. A video analysis of use of the new “concussion interchange rule” in the national rugby league. Int J Sports Med. 2016;Google Scholar
- Gardner AJ, RMN K, Levi CR, Iverson GL. Usefulness of video review of possible concussions in National Youth Rugby League. Int J Sports Med. 2016; in pressGoogle Scholar
- Gardner AJ, Levi CR, Iverson GL. Observational review and analysis of concussion: a method for conducting a standardised video analysis of concussion in rugby league. Sport Med - Open. 2017;3:26.View ArticleGoogle Scholar
- Guskiewicz KM, McCrea M, Marshall SW, Cantu RC, Randolph C, Barr W, Onate JA, Kelly JP. Cumulative effects associated with recurrent concussion in collegiate football players: the NCAA concussion study. JAMA. 2003;290:2549–55.View ArticlePubMedGoogle Scholar
- Guskiewicz KM, Ross SE, Marshall SW. Postural stability and neuropsychological deficits after concussion in collegiate athletes. J Athl Train. 2001;36:263–73.PubMedPubMed CentralGoogle Scholar
- Hallgren KA. Computing inter-rater reliability for observational data: an overview and tutorial. Tutor Quant Methods Psychol. 2012;8:23–34.View ArticlePubMedPubMed CentralGoogle Scholar
- Hinton-Bayre AD, Geffen G. Severity of sports-related concussion and neuropsychological test performance. Neurology. 2002;59:1068–70.View ArticlePubMedGoogle Scholar
- Hutchison MG, Comper P, Meeuwisse WH, Echemendia RJ. An observational method to code concussions in the National Hockey League (NHL): the heads-up checklist. Br J Sports Med. 2014;48:125–9.View ArticlePubMedGoogle Scholar
- IBM Corp . IBM SPSS statistics for Macintosh, version 22.0. Armonk: IBM Corp.Google Scholar
- Kohler R, Makdissi M, McDonald W, Partridge B, Gardner AJ. A preliminary video review of in-game head injury incidents (HII) and use of the head injury assessment (HIA) from the 2015 super rugby season. Br J Sports Med. 2017;51:A78–9.Google Scholar
- Lau B, Lovell MR, Collins MW, Pardini J. Neurocognitive and symptom predictors of recovery in high school athletes. Clin J Sport Med. 2009;19:216–21.View ArticlePubMedGoogle Scholar
- Lau BC, Kontos AP, Collins MW, Mucha A, Lovell MR. Which on-field signs/symptoms predict protracted recovery from sport-related concussion among high school football players? Am J Sports Med. 2011;39:2311–8.View ArticlePubMedGoogle Scholar
- Lovell MR, Collins MW, Iverson GL, Field M, Maroon JC, Cantu R, Podell K, Powell JW, Belza M, Fu FH. Recovery from mild concussion in high school athletes. J Neurosurg. 2003;98:296–301.View ArticlePubMedGoogle Scholar
- Makdissi M, Davis G. The reliability and validity of video analysis for the assessment of the clinical signs of concussion in Australian football. J Sci Med Sport 2016; online first.Google Scholar
- Makdissi M, Davis G. Using video analysis for concussion surveillance in Australian football. J Sci Med Sport 2016; online first.Google Scholar
- Makdissi M, Davis G, Jordan B, Patricios J, Purcell L, Putukian M. Revisiting the modifiers: how should the evaluation and management of acute concussions differ in specific groups? Br J Sports Med. 2013;47:314–20.View ArticlePubMedGoogle Scholar
- McCrea M, Guskiewicz K, Randolph C, Barr WB, Hammeke TA, Marshall SW, Powell MR, Woo Ahn K, Wang Y, Kelly JP. Incidence, clinical course, and predictors of prolonged recovery time following sport-related concussion in high school and college athletes. J Int Neuropsychol Soc. 2013;19:22–33.View ArticlePubMedGoogle Scholar
- McCrea M, Iverson GL, Echemendia RJ, Makdissi M, Raftery M. Day of injury assessment of sport-related concussion. Br J Sports Med 2013; 47: 272–284 Available from: http://www.ncbi.nlm.nih.gov/pubmed/23479484.
- McCrea M, Kelly JP, Randolph C, Cisler R, Berger L. Immediate Neurocognitive effects of concussion. Neurosurgery. 2002;50:1032–40.PubMedGoogle Scholar
- McCrory P, Meeuwisse WH, Aubry M, Cantu B, Dvorák J, Echemendia RJ, Engebretsen L, Johnston K, Kutcher JS, Raftery M, Sills A, Benson BW, Davis GA, Ellenbogen RG, Guskiewicz K, Herring SA, Iverson GL, Jordan BD, Kissick J, McCrea M, McIntosh AS, Maddocks D, Makdissi M, Purcell L, Putukian M, Schneider K, Tator CH, Turner M. Consensus statement on concussion in sport: the 4th international conference on concussion in sport held in Zurich, November 2012. Br J Sports Med. 2013;47:250–8.View ArticlePubMedGoogle Scholar
- McDevitt J, Tierney RT, Phillips J, Gaughan JP, Torg JS, Krynetskiy E. Association between GRIN2A promoter polymorphism and recovery from concussion. Brain Inj. 2015;29:1674–81.View ArticlePubMedGoogle Scholar
- McHugh ML. Interrater reliability: the kappa statistic. Biochem Med. 2012;22:276–82.View ArticleGoogle Scholar
- Meehan WP, Mannix RC, Stracciolini A, Elbin RJ, Collins MW. Symptom severity predicts prolonged recovery after sport-related concussion, but age and amnesia do not. J Pediatr. 2013;163:721–5.View ArticlePubMedPubMed CentralGoogle Scholar
- Miller JH, Gill C, Kuhn EN, Rocque BG, Menendez JY, O’Neill JA, Agee BS, Brown ST, Crowther M, Davis RD, Ferguson D, Johnston JM. Predictors of delayed recovery following pediatric sports-related concussion: a case-control study. J Neurosurg Pediatr. 2016;17:491–6.View ArticlePubMedGoogle Scholar
- Morgan CD, Zuckerman SL, Lee YM, King L, Beaird S, Sills AK, Solomon GS. Predictors of postconcussion syndrome after sports-related concussion in young athletes: a matched case-control study. J Neurosurgery Pediatr. 2015;15:589–98.View ArticleGoogle Scholar
- Nelson LD, Guskiewicz KM, Marshall SW, Hammeke T, Barr W, Randolph C, McCrea MA. Multiple self-reported concussions are more prevalent in athletes with ADHD and learning disability. Clin J Sport Med. 2016;26:120–7.View ArticlePubMedGoogle Scholar
- Nelson LD, Tarima S, LaRoche AA, Hammeke TA, Barr WB, Guskiewicz K, Randolph C, McCrea MA. Preinjury somatization symptoms contribute to clinical recovery after sports-related concussion. Neurology. 2016;86:1856–63.View ArticlePubMedPubMed CentralGoogle Scholar
- Pellman EJ, Viano DC, Casson IR, Arfken C, Powell J. Concussion in professional football: injuries involving 7 or more days out-part 5. Neurosurgery. 2004;55:1100–19.View ArticlePubMedGoogle Scholar
- Putukian M, Raftery M, Guskiewicz K, Herring S, Aubry M, Cantu RC, Molloy M. Onfield assessment of concussion in the adult athlete. Br J Sports Med. 2013;47:285–8.View ArticlePubMedGoogle Scholar
- Savage J, Hooke C, Orchard J, Parkinson R. The incidence of concussion in a professional Australian rugby league team, 1998–2012. J Sports Med 2013; online first.Google Scholar
- Terwilliger VK, Pratson L, Vaughan CG, Gioia GA. Additional post-concussion impact exposure may affect recovery in adolescent athletes. J Neurotrauma. 2016;33:761–5.View ArticlePubMedPubMed CentralGoogle Scholar
- Zuckerman SL, Yengo-Kahn AM, Buckley TA, Solomon GS, Sills AK, Kerr ZY. Predictors of postconcussion syndrome in collegiate student-athletes. Neurosurg Focus. 2016;40:E13.View ArticlePubMedGoogle Scholar