Table 4 Analysis of agreement
Variable | Mean of differences | 90% CI | Chances of true differences (%) | Effect size Cohen’s dadj (90% CI) | Magnitude of effect | p-value |
|---|---|---|---|---|---|---|
Traditional talk test: last stage where the answer was “yes” | ||||||
WTTT/LY vs WVT1 | − 22.4 | − 31.3/− 13.3 | Unlikely equivalent (87.3/12.7/0.0) | − 0.85 (− 1.22/− 0.48) | Moderate | 0.001 |
WTTT/LY vs WATT/2–3 | − 21.1 | − 27.5/− 14.5 | Unlikely equivalent (90.3/9.7/0.0) | − 0.78 (− 1.01/− 0.55) | Moderate | 0.000 |
WTTT/LY vs WATT/4–5 | 7.1 | 0.4/13.7 | Very likely equivalent (0.0/97.1/2.9) | 0.31 (0.07/0.56) | Trivial | 0.039 |
Traditional talk test: first stage where the answer was “no” | ||||||
WTTT/FN vs WVT2 | − 2.9 | − 10.9/5.1 | Most likely equivalent (0.1/99.9/0.0) | − 0.10 (− 0.34/0.13) | Trivial | 0.455 |
WTTT/FN vs WATT/4–5 | − 41.6 | − 49.7/− 33.3 | Most unlikely equivalent (100.0/0.0/0.0) | − 1.36 (− 1.60/− 1.12) | Large | 0.000 |
WTTT/FN vs WATT/6–7 | − 13.9 | − 18.7/− 9.1 | Very likely equivalent (1.9/98.1/0.0) | − 0.41 (− 0.56/− 0.25) | Trivial | 0.000 |
Alternative talk test: visual analog scale | ||||||
WATT/2–3 vs WVT1 | − 1.3 | − 8.2/5.6 | Very likely equivalent (0.7/99.1/0.2) | − 0.06 (− 0.40/0.28) | Trivial | 0.763 |
WATT/4–5 vs WVT1 | − 29.5 | − 37.6/− 21.2 | Most unlikely equivalent (99.8/0.2/0.0) | − 1.34 (− 1.73/− 0.95) | Large | 0.000 |
WATT/4–5 vs WVT2 | 38.7 | 27.1/50.2 | Most unlikely equivalent (0.0/0.4/99.6) | 1.18 (0.84/1.52) | Moderate | 0.000 |
WATT/6–7 vs WVT2 | 11.1 | 2.8/19.2 | Very likely equivalent (0.0/97.6/2.4) | 0.29 (0.04/0.54) | Trivial | 0.058 |