Inter-rater reliability test and test–retest results, weighted kappa
| NAT:PD-ILD item | Inter-rater reliability | Test–retest* | |||||
| Number of observations | Distribution of categories† | Weighted kappa (SE) | % Agreement | Weighted kappa (SE) | |||
| 0 | 1 | 2 | |||||
| 1. Does the patient have a carer? | 77 | NA | 14% | 86% | 0.69 (0.24) | 64.3 | 0.00 (–) |
| 2. Has the patient or carer requested a referral to a Specialist Palliative Care Service (SPCS)? | 70 | NA | 9% | 91% | 0.82 (0.44) | 72.7 | −0.14 (0.28) |
| 3. Do you require the assistance of the SPCS? | 65 | NA | 20% | 80% | 0.13 (0.30) | 72.7 | 0.23 (0.29) |
| 4. Is the patient experiencing unresolved physical symptoms? | 89 | 6% | 38% | 56% | 0.49 (0.21) | 96.3 | 0.75 (0.22) |
| 5. Is the patient experiencing unresolved psychological symptoms? | 76 | 36% | 57% | 8% | 0.32 (0.16) | 81.7 | 0.00 (0.24) |
| 6. Does the patient have problems with daily living activities? | 85 | 8% | 54% | 38% | −0.07 (0.41) | 79.0 | −0.13 (0.19) |
| 7. Does the patient have concerns about spiritual or existential issues? | 57 | 79% | 18% | 4% | 0.44 (0.30) | 85.7 | 0.59 (0.24) |
| 8. Does the patient have work, financial or legal concerns? | 75 | 83% | 13% | 4% | 0.15 (0.58) | ‡ | ‡ |
| 9. From the health delivery point of view, are there health beliefs, cultural or social factors involving the patient or family that are making care more complex? | 57 | 77% | 21% | 2% | −0.55 (0.73) | 85.9 | −0.03 (0.21) |
| 10. Is the carer or family distressed about the patient’s physical symptoms? | 85 | 32% | 47% | 21% | −0.17 (0.39) | 90.8 | 0.65 (0.22) |
| 11. Is the carer or family having difficulty providing physical care? | 79 | 61% | 34% | 5% | 0.59 (0.23) | 81.6 | 0.10 (0.23) |
| 12. Is the carer or family having difficulty coping with the patient’s psychological symptoms? | 71 | 69% | 30% | 1% | 0.36 (0.29) | 94.4 | 0.64 (0.23) |
| 13. Is the carer or family concerned about financial or legal concerns? | 69 | 93% | 7% | 0% | 0.77 (0.24) | ‡ | ‡ |
| 14. Is the carer or family experiencing problems that are interfering with their functioning or interpersonal relationships, or is there a history of such problems? | 67 | 61% | 37% | 1% | 0.21 (0.38) | 41.2 | −0.15 (0.22) |
| 15. Is the carer or family experiencing unresolved psychosocial problems or feelings that are interfering with their well-being or functioning? | 70 | 47% | 43% | 10% | 0.75 (0.12) | 94.7 | 0.78 (0.23) |
| 16. Is the carer or family experiencing grief over the future death of the patient? | 45 | 69% | 27% | 4% | 0.57 (0.27) | 75.0 | −0.03 (0.27) |
*39/53 (74%) saw only one video consultation, 13 (25%) saw two video consultations and 1 (2%) saw four video consultations, though they may have watched each video up to two times (mean (SD) 1.3 (0.6), median 1).
†Items 1–3: 1=yes, 2=no; items 4–16: 0=none, 1=some/potential, 2=significant.
‡Same category assigned by every rater on both occasions; too few categories to calculate kappa statistic.
NA, not applicable; NAT:PD-ILD, needs assessment tool: progressive disease in interstitial lung disease.