A video review of multiple concussion signs in National Rugby League match play

Over the past few years, sideline video review and analysis has become common practice. Its use ranges from professional leagues worldwide to some National Collegiate Athletic Association (NCAA) conferences, with the goal of improving the recognition of a possible concussion [13]. Several studies have reviewed the usefulness and limitations of sideline and post-game video review [2, 5–7, 12, 13]. Our group [2–4] and others [1–3, 13, 14] have conducted several video reviews in professional leagues and reported that video injury surveillance can be difficult to interpret but may provide a useful adjunct to the recognition of concussion [6]. For example, although the presence of a sign after a concussion is sensitive to identifying the injury when reviewing known or suspected injuries, when reviewing an entire season of match play, some concussion signs occur commonly but do not typically reflect the presence of a concussion [4].

Because athletes may demonstrate several signs of a concussion simultaneously following a head impact, one way to maximize the application of sideline video review systems is to examine the risk for concussion associated with the presence of various combinations of concussion signs observed with video replay. In addition, reviewing these combinations of signs in the context of match play characteristics (such as anticipating the oncoming impact, tackler versus ball carrier, tackle height, and the number of defenders involved in the tackle) may also assist with maximizing the usefulness of sideline video review systems. For example, in professional hockey, a risk prediction model using a combination of visual signs and mechanism of injury improved the ability to identify players who may require further evaluation for possible concussion [1]. The current study builds on the work that examined the rate of six concussion signs during the 2014 National Rugby League (NRL) season [4–7], by evaluating various combinations of signs for predicting cases of medically diagnosed concussion. The purposes of this study were to determine (i) the combination of observable signs occurring in the NRL using video review and (ii) the combination of observable signs and match play characteristics that were associated with a concussion diagnosis when the concussion interchange rule (CIR) was activated.

During the 2014 season, there were 167 observed incidences where the CIR was used, and we located and reviewed 162 (97%). Of these cases, return to play data was available for 156 (96%) of incidences, which was the final number of cases used in our analyses. Of these events, athletes were subsequently medically diagnosed with a concussion 38% of the time by their club physician (n = 60). All 60 cases of medically diagnosed concussion are included in the analysis, along with the 96 cases where the CIR was used but the athlete was not diagnosed with a concussion following the suspected event.

To expand on previous work examining the use of video analysis in concussion identification [4–8, 10], we explored the rates of six observable signs of concussion, match play characteristics, and how the combinations of these signs were able to predict a concussion diagnosis when the CIR was activated. Many variations of observable signs were present following a suspected concussion in NRL match play. Considered individually, positive anticipation of the oncoming hit, the number of defenders involved in the tackle, unresponsiveness or loss of consciousness at the time of injury, clutching or shaking the head at the time of injury, vacant stare at the time of injury, and seizure at the time of injury were potential predictor variables that led to a concussion diagnosis. Considered in combination, only positive anticipation of the oncoming hit, a fewer number of tacklers involved in the collision, and a vacant stare were significantly independently associated with a medical diagnosis of concussion. Using a cutoff threshold of three out of six observable on-field concussion signs to distinguish between those who were or were not medically diagnosed with concussion resulted in low sensitivity (0.30). Therefore, it was anticipated that further insight might be gained by examining which combinations of these types of signs may lead to improved identification accuracy over simply the total number of signs. However, no single combination of signs led to high levels of discriminative ability, likely due, at least in part, to the low number of observations for each category.

It is important to note that prior work reviewing an entire NRL season [4] identified that the six concussion signs occur commonly in NRL match play in the absence of the CIR being activated and in the absence of a diagnosed concussion. For example, the sign slow to get up was observed 2240 times over the course of the entire season, but of those 2240 times, only 223 instances looked like they may have been a concussion-related sign (10.0%) [4]. Of the 223 instances, 153 players were removed under the CIR and 60 were medically diagnosed as having a concussion [4]. In other words, all players medically diagnosed with a concussion that season (N = 60) were slow to get up, but this sign is very common in the absence of concussion. Similarly, prior work has reported that clutching the head is the most common observable sign among NRL players who used the CIR [6]; however, clutching the head also occurs commonly in the absence of the CIR being activated or a diagnosed concussion [4]. In addition, clutching the head was not significantly associated with a concussion diagnosis following CIR use in the current study. In contrast, the presence of a blank or vacant stare was a significant predictor of an eventual diagnosis of concussion in our study. Additionally, positive anticipation of an oncoming collision was independently associated with sustaining a concussion in those for whom the CIR was activated. It may be that the nature of the game and the nature of collisions and tackles differ between rugby league and other sports such as ice hockey that have demonstrated an association between unanticipated collisions and increased risk for concussion [16]. Anticipation of the impact is an interesting variable to consider across these sports. Unlike hockey, which is a 360° sport, rugby league tends to be a 180° sport, meaning a ball carrier does not often get “blindsided” by a tackler, and a tackler (almost all of the time) is engaged in making the contact and therefore is almost always anticipating contact. So despite there being few examples of a player not anticipating a hit in professional rugby league, very few of those events led to a subsequent diagnosed concussion. It is also possible that the significant anticipation of the hit is a spurious finding. It is also important to note that nearly two out of three tackles in the NRL are made by one or two defenders (i.e., 63% of tackles involved one or two defenders; 17 and 46%, respectively) [11]. Therefore, offensive players are exposed to a greater number of tackles, per season, by one or two players, so this greater exposure could underlie the independent association. Similarly, we speculate that a greater percentage of tackles over the course of the season also involve the offensive player anticipating the hit (versus not), and so the greater overall number of tackles in this scenario might underlie the association. In contrast, the technique used in a one-on-one tackle or a two-man tackle differs from that of a tackle involving three or more defenders, potentially resulting in an altered injury risk.

The sole reliance on video signs to identify a potential concussion can be problematic. In the National Hockey League (NHL) for example, visual signs have been found to differ in their ability to predict a diagnosis of concussion, and approximately 53% of the concussions were not associated with visual signs [3]. Therefore, examining other potential variables (i.e., mechanism of injury) in combination with various video signs may improve the sensitivity and specificity of identifying or predicting concussion [1]. Specifically, Bruce and colleagues [1] reported that suspected LOC and balance problems each account for unique variance in subsequent concussion diagnoses. Initial contact with the shoulder and secondary contact with the ice increased the risk of concussion diagnosis among athletes who exhibit a visual sign. The current study found that vacant stare, positive hit anticipation, and fewer number of tacklers involved in the tackle predicted concussion diagnoses in professional rugby league players.

There were several limitations of the current study. Not all signs were identified in every incident due to variability in the quality of the available video footage (i.e., in some instances, the view of the incident was obscured, or there was no closeup footage). These missing data were excluded from the analyses, which might have slightly improved the support for the utility of some of the visible signs. This issue was most commonly encountered when attempting to identify the blank or vacant stare sign. For these reasons, it is important to ensure that high-quality footage with the capability of multi-angle, slow motion replay, and narrow zoom options are available when reviewing video footage on the sideline [13]. Only one reviewer completed the coding of the events surrounding the use of the CIR for every game in the season; the inter-rater reliability of that type of coding is unknown. Additionally, the video reviewer was only partially blinded to the use of the CIR but completely blinded to the sideline assessment results and the medical diagnosis of concussion when reviewing the matches. Another study limitation was that there were no concussions that were subsequently diagnosed after the match day by the club medical personnel and reported to the researchers, which is inconsistent with other video review studies that have reported a numbers of cases of post-game diagnosis of concussion [3]. Our use of an operational definition of exclusion to categorize the observed signs into “plausible concussion signs” versus signs that were more likely attributable to other factors was also a limitation in terms of its subjectivity and reproducibility for future work. Further, in 16 instances, players were removed under the CIR, not diagnosed with a concussion, but did not return to play. Although data were not available regarding the reason for this, potential reasons may have been that they sustained an injury other than concussion that required their removal from gameplay or that the injury occurred near the end of the match. Importantly, the six signs that were reviewed in this study were signs that the researchers chose to observe, not necessarily the specific signs that the team medical personal who are making the in-game decisions on the sideline are also using to make their decisions to use the CIR. Finally, the current study was a post-game review of a men’s professional rugby league, and as such the results may not necessarily be generalizable to other levels of match play.

Video replay analysis is useful for sports medicine professionals, but it also can be difficult to interpret and presents challenges in identifying those who have sustained a concussion. Advancements in the in-game detection of concussion may be improved by establishing clear definitions, providing education pertaining to observable signs of concussion, having an independent concussion expert on the sideline or in the stands who can identify players to be removed from play, and improving access to video replays, particularly systems that allow for multiple angle reviews, in addition to improving the communication between video observers and sideline medical personal.