Nutritional screening tools are essential to identify patients at risk of malnutrition(1), allowing preventative action and treatment and consequently reducing morbidity and economic burden. Before tools can be recommended for widespread use, it is important to test reliability. It has to be demonstrated that all staff using the tool will obtain consistent results. The validity of the Imperial Nutritional Screening System (INSYST) has previously been demonstrated and the tool's inter-rater reliability is described here comparing use by a pre-registration dietitian with use by a nurse. INSYST is a two-tiered screening tool that does not require BMI. It is composed of a pre-screen (INSYST I; affirmation of either recent unintentional weight loss or decreased appetite) and a brief assessment carried out only if triggered by INSYST I (INSYST II;– detailing food intake, weight change and other risk factors; yielding ‘not at risk’, ‘at risk’ and ‘malnourished’ categories).
Patients were screened within 72 h by a pre-registration dietitian using INSYST I and II, including measuring weight. Once this assessment was complete the routinely-done screening data completed by the nursing staff were collected from the nursing notes. Agreement was examined using κ statistic. INSYST II at-risk and malnourished categories were combined for this analysis.
Sixty-one patients were included in the present study. The agreement between dietitian and nurse for INSYST I was κ 0.77 (95% CI 0.61, 0.93; P<0.001) and for INSYST II κ 0.39 (95% CI 0.18, 0.61; P=0.001). This result indicates a substantial and fair agreement respectively(Reference Landis and Koch2). Raw agreement data (see Table) show little disagreement for INSYST I categorisation, but nurses tended to underestimate malnutrition risk when using INSYST II.
The reliability of the INSYST I tool between dietitian and nurse in the acute setting is high and compares very favourably with data on other tools such as the malnutrition screening tool (κ 0.84)(Reference Ferguson, Capra and Bauer3). Inter-rater reliability for the malnutrition universal screening tool in the acute setting(Reference Elia4) was not assessed between dietitian and nurse, only healthcare assistant and nurse (κ 0.9), and medical student v. medical student (κ 1.0). Thus, it is difficult to compare results as a different level of reliability was being tested. The reliability of INSYST II is only fair and lower than other tools, but again many tools have not been tested comparing dietitian with nurse completion. If one assumes that the dietitian categorisation is most likely to be correct, reflecting their additional knowledge and training, this comparison is an important one to make. This outcome would suggest that nursing staff require further training to ensure that they use the screening tool correctly. It may also suggest that INSYST II requires further development to ensure that more consistent and accurate categorisation can be achieved by staff with lower levels of nutritional knowledge and skills. The present study was a relatively small pilot study, so further research is warranted. Examination of inter-rater reliability between members of the same profession would also be of value.
