Participants and Procedures
Canadian National Team carded athletes (N = 199) from both senior national teams and lower-level national teams participated in the study. The athletes were between the ages of 18–36 from 23 different summer and winter sports. The core questionnaire was the same as previously published [13]; however, an additional question on caffeine consumption was added, and a subset of athletes had one additional question on electronic device use, see Additional file 1. The consent form was located on the first page of the online survey (www.surveymonkey.com), and if participants continued to complete the survey, it was an indication of their consent to participate. The study was performed in accordance with the standards of ethics outlined in the Declaration of Helsinki and approved by the University of Calgary Conjoint Health Research Ethics Board.
Clinical Validation Sub-study
Following the completion of the initial ASSQ at time 1 (T1), 65 athletes were randomized to the clinical validation sub-study. Seventy-one percent of the athletes (N = 46) could comply with the requirements of the protocol. The protocol included 2 weeks of wrist-watch actigraphy, which is not presented here (Readiband, Fatigue Science, Canada; [17]), an ASSQ completed at time 2 (T2), a standardized clinical sleep interview with the SMP, a rating of the category of clinical sleep problem from the SMP, a more detailed follow-up interview if required, recommendations for sleep interventions, and an ASSQ completed at time 3 (T3), see Fig. 1.
Once an athlete was randomized to the clinical validation sub-study, they were contacted to determine the actigraphy recording period which started after at least 1 day for every time zone traveled recently to accommodate for any circadian misalignment from recent travel. After the recording period, athletes completed the ASSQ T2 to assess test-retest reliability which occurred (102 ± 66 days) after T1 but prior (1 ± 3 days) to the structured clinical sleep interview with the SMP. The structured clinical sleep interview was performed online through videoconferencing or over the phone. The SMP followed a standardized interview sheet which was based on prior clinical experience working with athletes. It included questions about sleep history (e.g., do they think they have a sleep problem), estimated sleep parameters (e.g., sleep duration, naps, sleep latency, wake after sleep onset), sleep disorders (e.g., insomnia, delayed sleep phase syndrome, sleep-disordered breathing, periodic limb movement, parasomnias, and bruxism), timing of sleep (e.g., preferred bedtime and wake time, do they see themselves as a morning-type or evening-type), and if the athlete had issues with sleep or performance during travel. After the interview was completed (average duration 9.9 ± 3 min), the SMP who was blinded to the results from the ASSQ at T1 and T2 rated the category of clinical sleep problem based on how severe the athletes sleep disturbances were (none, mild, mild to moderate, moderate, moderate to severe, and severe) and indicated which of the standardized recommendations were to be communicated to the subject.
Sleep Intervention Recommendations
Athletes were emailed results based on the recommendations from the SMP after the questionnaires were examined (those athletes not randomized N = 153) or after the interviews (N = 46). In those who the SMP classified as moderate to severe (N = 16), a second more detailed follow-up interview to evaluate the problem further and to discuss the recommendations occurred. The recommendations were standardized based on the SMP’s rating of clinical sleep problem and the responses to the modifier questions. The recommendations included a general sleep education sheet on sleep quantity, quality, timing, and proper sleep hygiene for all the athletes. Also included were individualized recommendations depending on their responses to sleep duration, napping activity, insomnia, and sleep-disordered breathing symptoms. If indicated, a travel and jet lag fatigue management education sheet, recommendations for an insomnia self-help book or standardized online CBT-Insomnia program, a circadian re-entrainment program using light therapy and melatonin, and recommendations for further sleep testing or treatment in their local area were provided. When possible and with the athlete’s consent, the sport physician and lead integrated support team members would also get the results to help monitor the athlete’s progress on a more frequent basis. All but two athletes in the clinical validation sub-study completed an ASSQ T3 approximately 150 ± 67 days after the clinical interview to assess if the sleep recommendations helped improve sleep as reflected in a reduction in the SDS.
Statistical Analyses
To assess the sample characteristics, descriptive statistics were determined for age, sex, sport, status on the senior national team or lower-level national team, years at the current level, and training season the questionnaire was completed in. The median time taken to complete the ASSQ (generated from www.surveymonkey.com) was also assessed.
The internal consistency of the ASSQ items was estimated using Cronbach’s alpha. Test-retest correlations compared the stability of the scale from the T1 to T2, prior to the recommendations taking place.
To determine if the questions for each scale could be summed for a score, unidimensionality of the latent trait being measured (sleep difficulty and chronotype) was assessed using principal component analysis (PCA) [18]. Exploratory factor analysis (EFA) was performed on both the sleep difficulty score and chronotype score. Pearson’s correlation coefficients were estimated between the original sleep difficulty score and the new sleep difficulty score. Confirmatory factor analysis (CFA) was performed on both sleep difficulty and chronotype scores.
To determine if the sleep difficulty score and chronotype score were consistent across different groups, the data from the Canadian National Team athletes was compared to the data from a separate study in N = 1074 competitors from the London Virgin Money Marathon who completed an expanded version of the ASSQ (manuscript in preparation). For data from Canada and the UK, the CFA path coefficients were compared, as well as the item correlations.
Comparisons of level of clinical sleep problem between the ASSQ and the SMP were performed using weighted kappa [19,20,21]. Weighted kappa was used as kappa (unweighted) does not consider the degree of disagreement. Reliability was estimated for the ASSQ scoring system with Cronbach’s alpha. Sensitivity, specificity, positive predictive value, and negative predictive value [22] were estimated from the ASSQ scoring system sleep problem categories to the SMP categories after the clinical interview. Sensitivity was estimated as the number of subjects determined by the ASSQ scoring system as needing a clinical intervention from the SMP (moderate and severe) divided by the number of subjects needing a clinical intervention as determined by the SMP. Specificity was estimated by the number of subjects determined by the ASSQ scoring system as not needing a clinical sleep intervention (none, mild categories) divided by the number of subjects not needing a clinical intervention as determined by the SMP. The positive predictive value was estimated by dividing the subjects for which there was agreement on needing a clinical intervention (moderate, severe categories) by the actual number of subjects needing a clinical intervention as determined by the SMP. The negative predictive value was estimated by dividing the subjects for which there was agreement on not needing a clinical intervention by the actual number of subjects not needing a clinical intervention as determined by the SMP.
To assess the impact of the sleep recommendations, simple paired t tests were performed at T1 (baseline) and T3 (post-recommendations) for the SDS and the categories of clinical sleep problem.
All statistical analyses were performed with R 3.3.3 (R Core Team, Vienna, Austria). Descriptive statistics, Cronbach’s alpha exploratory factor analysis and correlation estimates were performed with the Psych package [23]. CFA were performed with the Lavaan package [24]. Inter-rater estimates and weighted kappa were performed with the IRR package [25].