Table 2 Inter-rater reliability and overall agreement of the TRUST form
Item name | Relative inter-rater reliability (intraclass correlation coefficient (2,1)) | Absolute reliability (standard error of measurement) |
|---|---|---|
1a. Data citation | 0.49 [0.33 to 0.64] | 0.31/1 |
1b. Code citation | 0.00 [− 0.17 to 0.18] | 0.09/1 |
2a. Data transparency | 0.00 [− 0.13 to 0.39] | 0.23/1 |
3a. Code transparency | 0.00 [− 0.14 to 0.18] | 0.09/1 |
5a. Research material transparency | 0.23 [0.07 to 0.42] | 0.16/1 |
6a. Design and analysis | 0.86 [0.80 to 0.91] | 0.19/1 |
7a. Study registration | 0.78 [0.69 to 0.86] | 0.23/1 |
8a. Registration of analysis plan | * | * |
9a. Acceptance of replication studies | 0.80 [0.70 to 0.87] | 0.09/1 |
9b. Registered Reports | 1.00 [1.00 to 1.00] | 0.00/1 |
9c. Submission of background and methods alone | 0.00 [− 0.14 to 0.18] | 0.09/1 |
Overall | 0.68 [0.55 to 0.79] | 1.17/27 |