The world's aging population has enormous implications for the healthcare sector. Data from Statistics Canada 2016 indicate that the population of persons aged 65 years and older (older adults) has been growing by approximately 8–14 percent per year over the past 40 years. The older adult population in Canada is expected to make up approximately 25 percent of the total population by 2036, representing an estimated population of 9.9–10.9 million people (1). Data from the 2017 revision of the World Population Prospects reported that the older person (defined here as aged 60 years and older) segment is growing at a rate of approximately 3 percent per year, which is faster than all younger age groups. The number of older persons is expected to more than double by 2050 from 962 million to 2.1 billion (2).
An aging population will challenge the health system in delivering quality health care. The current system in the industrialized world has been built around a disease-based acute care focus not well suited to meet the challenges associated with aging, including multi-morbidity and frailty (Reference Lim, Wong, Leong, Choo and Pang3). Lim et al. (Reference Lim, Wong, Leong, Choo and Pang3) state that the sustainability of the healthcare delivery system will require collaboration that extends beyond the traditional practitioner–patient relationship and includes sectors such as housing and technology.
With aging in place (4) older adults are supported in remaining in their own home and avoiding facility-based residential care (i.e., nursing homes). Evidence indicates that the health of a frail older person is negatively impacted each time they are moved (Reference Marek and Rantz5). Allowing older adults to age in place can provide a healthier approach to care with fewer moves compared with most long-term care pathways, which may require the older adult to move every time their care needs increase (Reference Marek and Rantz5). Home health services can assist older adults to remain at home longer, delaying if not avoiding admission into a nursing home (Reference Yuchi, Kalamaras, Kelly, Hornick and Yucel6). A 2014 survey conducted by the American Association of Retired Persons showed that the majority of older adults would prefer to age in place (Reference Lampkin and Barrett7).
Home monitoring technologies that measure vital signs (i.e., temperature, pulse, respiratory rate, blood pressure) and key metabolic parameters (such as blood glucose) fit well into the aging in place model and could allow older adults the ability to independently and safely manage their health concerns in a home setting. These devices have been shown to increase the perception of safety among independent older adults (Reference Peek, Aarts, Wouters, van Hoof, Demiris and Wouters8). Their use could improve patient flow, optimize clinician time, and triage patients to appropriate clinical settings (Reference Cellar and Sparks9). However, there is skepticism surrounding the integration of such monitoring technologies into the healthcare system (Reference Elenko, Underwoord and Zohar10). Much of this is due to the uncertain clinical utility of these devices caused by their inconsistent, inaccurate, and/or infrequent validation (Reference Liu, Stroulia, Nikolaidis, Miguel-Cruz and Rincon11). Patient and provider confidence in the accuracy, reliability, and precision of the measurements recorded by home monitoring devices is essential for the successful adoption of these technologies for healthcare decision making.
We conducted a narrative review of the published literature on home monitoring technologies and devices that could be used to assess vital signs and blood glucose levels for older adults in a home setting. Specifically, we assessed the presence and reporting quality in peer-reviewed publications of the accuracy, precision, and reliability of home monitoring devices for vital signs and the metabolic parameter of blood glucose.
Methods
Search Strategy
A search strategy for two medical bibliographic databases (Medline, Embase) and one engineering bibliographic database (Compendex) was developed in consultation with an academic health sciences and engineering librarian (H.G.). It was designed to identify published, peer-reviewed literature concerning devices used in the home monitoring of vital signs (specifically, temperature, pulse, respiration rate, and blood pressure) and blood glucose levels for use in an older adult population, defined as adults aged ≥ 65 years old. Database specific search strategies used keywords related to vital signs, monitoring devices and technologies, independent living, and older adults. The search strategies are described in detail in the Appendix, which can be found in the Supplementary Files. Due to the rapid turnover of home monitoring technologies, articles published before 2012 were considered not to reflect the current state of devices available in the market (Reference Cellar and Sparks9) and were not included. The initial search was restricted to publications from between the years 2012 and 2017 and was last conducted on May 31, 2018.
Search Strategy and Data Selection
The search strategy identified 1,978 articles. After eliminating duplicates across searches, 1,753 papers underwent title and abstract screening. Screening was completed in two stages. First, two members of the research team (M.C., L.B.) independently reviewed the titles and abstracts for potential relevance and any disagreements were considered by a third independent reviewer (G.F.) to resolve. Following title and abstract screening, four reviewers (G.F., J.L., M.C., L.B.) independently undertook full-text screening. The four reviewers were randomly allocated articles to review so that at least two people reviewed each of the potentially eligible literature.
Disagreements on whether an article should be retained for the qualitative synthesis were resolved through discussion and consensus among the reviewers. One of the senior investigators (J.C.) acted as a third-party independent adjudicator in the event that consensus could not be reached. Articles selected for full text review met the following criteria: (i) use of a device to measure a vital sign or blood glucose, (ii) device was intended to serve an older population (adults ≥ 65 years old), and (iii) device had an intended use in a home or independent community setting. Reviews (i.e., narrative, scoping, systematic), proof of concept papers, and conference abstracts were not included in full text screening. The title and abstract screening led to the selection of the 402 papers for full text screening for potential inclusion in the qualitative synthesis as shown in Figure 1.
Fig. 1. Flow diagram of studies included and excluded from qualitative synthesis.
A data extraction form based on an iterative piloting process that used thirty-five papers was developed. Extracted information from the selected literature included publication type, participant population, setting, which vital signs or blood glucose level was measured and how, author statement of validity, and description of the method of validation. After the initial piloting using the data extraction form, the included articles were reviewed and data extracted using Numbat (Reference Carlisle12), a meta-analysis extraction manager. Based on the full-text reviews a total of 209 articles were retained for the qualitative analysis.
Assessment of Device Quality Validation
In order to assess the presence and reporting quality of evidence on the accuracy, precision, and reliability of home monitoring devices, the research team extracted information concerning the authors’ statement of validity and/or description of the methods of validation used in the published article. The author's statement of validity was defined as claims made by the authors as to the accuracy, reliability, or precision of the device, or if the authors made an explicit claim that the device was valid or had been previously validated. Validity claims in the included articles were categorized as: (i) no validity claim made, (ii) validity claim made but not supported by a citation or a description of the methods of validation, (iii) validity claim made and method of validation described, (iv) validity claim made and a supporting citation provided, and (v) validity claim made with both methods described and a citation provided.
It was determined whether the test device was validated against a comparator device and there was a description of the methods used to verify the accuracy, reliability, and precision of the device. Two members of the research team (G.F., J.L.) assessed the method of validation described in the articles. The rating framework developed in collaboration with the teams’ medical expert (J.C.) to assess the quality of the methods used for validation is provided in Table 1. Definitions of low-, moderate-, or high-quality validation methods were based on the description of comparator devices and whether the statistical methods used are considered standard in medical practice. Two independent reviewers provided ratings for the comparator device and methodology (G.F., J.L.). Uncertain ratings and disagreements were reconciled by consensus and consultation with the team's medical expert (J.C.).
Table 1. Definitions of Low-, Moderate-, and High-Quality for Comparator Device and Method of Comparison
Results
Results of Search Strategy and Validation Claims of Medical Devices
The literature search identified 1,978 potentially relevant articles (Figure 1). After duplicate articles were removed, there were 1,753 articles that underwent title and abstract screening, which was conducted independently by two members of the research team (M.C., L.B.). Disagreements were passed along to a third independent reviewer (G.F.). Four hundred two articles were selected for full-text review, with a total of 209 included in the narrative review (182 full text articles, 9 conventional abstracts, 18 extended abstracts).
While all included articles discussed a device suitable for older adults, only 22 percent (n = 46) of the reviewed publications tested a device on a population restricted to older adults. Most tested their device on a mixed population of younger and older adults (69 percent; n = 144), younger adults only (5 percent; n = 10), or did not specify the age of the participants or test on humans 4 percent (n = 9). Sixty-eight percent (n = 142) of the reviewed publications were tested in home or independent living settings and 32 percent (n = 67) were tested in nonhome, nonindependent settings (e.g., nursing homes, laboratories, hospitals, or clinics). To illustrate the multiple items extracted from the 209 included articles, we have provided frequency tables which highlight publication type, participant population, setting, vital signs and blood glucose measured, method of vital sign and blood glucose measurement, author statement of validity, and method of validity in Supplementary Tables 1–7.
Forty-three percent (n = 89) of the reviewed publications provided no claim concerning the validity of the device. While 57 percent (n = 120) of the studies made some type of claim concerning the validity of the device, 12 percent (n = 25) of these studies did not provide support for this claim, simply stating that they used a valid device (see Figure 2). Only 45 percent (n = 95) of the studies provided a citation or some evidence to support their validation claim. Of these studies, 19 percent (n = 39) provided a citation for their validation claim, 18 percent (n = 37) tested that claim, and 9 percent (n = 19) both tested and cited that claim.
Fig. 2. Validation claims in examined literature.
Validation Assessment of Device Quality
Forty-eight of the 209 (23 percent) articles described the use of a comparator device and the method of comparison. Figure 3 shows the quality of the comparator device and method of comparison used by the included studies. Part A of Figure 3 shows the quality of the comparator device. The results show that 46 percent (n = 22) of the studies used a high-quality comparator device, 46 percent (n = 22) used one of moderate-quality, and 8 percent (n = 4) used a low-quality comparator. Part B of Figure 3 focuses on the statistical method used to compare the device of interest and its comparator. In 65 percent of the studies (n = 31), the statistical methods were minimally described and a low-quality rating was given. Twenty-three percent (n = 11) were given a moderate rating, and 13 percent (n = 6) a high one.
Fig. 3. Reporting of device validation concerning the comparator device (A) and the method of comparison (B).
Discussion
Home monitoring technologies have the potential to improve the quality of care for older adults and support aging in place by offering a safe, reliable, and cost-effective alternative to monitoring in a clinical environment (Reference Majumder, Mondal and Deen13). That being said, these technologies also bring forth concerns from older adults, caregivers, and providers around privacy, lack of standardization of device measurement, and uncertainty of the effectiveness of the device (Reference Carstemsen, Rosenberger, Smith and Modrek14–Reference Peek, Wouters, van Hoof, Luijkx, Boeije and Vrijhoef16). Our narrative review focused on addressing concerns surrounding home monitoring technologies from a healthcare provider perspective, where device uptake can be slowed due to a lack of confidence surrounding the clinical value of these home monitoring technologies as well as their ease of integration into the healthcare system (11;17).
The results observed from our assessment indicating only 45 percent of the 209 included publications provided a citation or other evidence to support their validation claim, and of the 48 articles that did describe the use of a comparator device, only 12 percent (n = 6) used high-quality statistical methods support the need for improvement of validation studies of home monitoring devices. Improvements in the conduct and reporting of validation studies have the potential to improve the adoption of validated, accurate, reliable, and value-adding devices within the healthcare system. Our observations are also relevant at a time where home monitoring technologies are attracting the attention of both academics and medical professionals due to the growing population of older adults and increasing use of these devices in medical practice (Reference Khosravi and Ghapanchi17).
The primary objective of our narrative review was to assess how the accuracy, precision, and reliability of home monitoring devices for vital signs and blood glucose levels in an older adult population was being reported based on a comprehensive review of the peer-reviewed literature within the past 6 years. We found limited reporting of high-quality evidence with respect to these devices. We found that claims of validity are often either unsubstantiated or based on limited statistical methods (i.e., percent agreement) rather than more robust methodologies. The majority of studies provided low levels of support for the validity and reliability of the devices described.
While most studies included older adults in their study population, only 22 percent tested a device on a population restricted to older adults. Additionally, a significant proportion (32 percent) of the studies did not test their device in a home setting. Testing in the intended setting by the population of interest can identify important issues with a device and would provide valuable clinical evidence on their intended use (Reference Kumar, Hennek and Smith18). The presence of peer-reviewed evidence on the use of these devices in their intended setting by their target population would allow for a balanced assessment of the claims being made.
We acknowledge the limitations of our work. We did not examine nonpeer review literature or the gray literature due to resource limitations and the limited scope of this review. We did not have access to device manufacturer data on specific medical devices. We acknowledge it is possible assessments that are more rigorous were done and are available but were not easily accessible. We also recognize that there are limitations with regard to the methods used in this review including the use of QUADAS-2 or STARD to evaluate the quality of the studies (19;20). We elected to develop a tailored evaluation framework given the diverse nature of several of the studies identified when first scoping the literature in this area (e.g., clinical trials, proof of concept studies, and case studies).
The major focus in our review was on the author's statement of validity and the description of the method of validation used in the literature regarding the reference standard and its comparison with the index test, which we considered germane to the confidence of a clinician who might be using the information for decision making. Additionally, the QUADAS-2 tool is recommended for use in systematic reviews of diagnostic accuracy but we chose to focus on a narrative review. Nonetheless, we did extract information on components of the four key domains of QUADAS-2, including patient selection, the index test, and the reference standard. The team also relied on the experience of a single medical expert in reviewing the comparator devices and statistical methods. However, the statistical methods identified as belonging to the moderate- and high-quality validation categories (e.g., Bland Altman, Cohen's Kappa, Mann-Whitney) are frequently used in clinical research and are generally considered reliable (Reference Giavarina21–Reference Vermeulen, Thas and Vansteelandt23).
The practice of making validation claims without providing evidentiary support is inappropriate. If claims continue to be based on low-quality data, clinician confidence in home monitoring technologies will likely remain limited (Reference Lampkin and Barrett7). It is our hope that there will be a renewed commitment to improving testing and reporting practices for home monitoring technologies intended for use by the older adult population.
In conclusion, this narrative review demonstrates the paucity of high-quality validation data for home monitoring technologies found in the peer-reviewed literature. Current validity claims provide limited confidence for clinical decision-making purposes. Improvements to testing and reporting practices in the validation of home monitoring devices may result in increased confidence among healthcare providers in their ability to accurately and safely monitor their patients’ health at home and improve the appropriate adoption of these devices within the healthcare system.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0266462319000527
Conflicts of interest
None of the authors report any conflicts of interest to disclose relevant to this narrative review.
Open access
