Hostname: page-component-78c5997874-j824f Total loading time: 0 Render date: 2024-11-10T15:25:36.776Z Has data issue: false hasContentIssue false
  • English
  • Français

INTERACTION BETWEEN OBJECTIVE PERFORMANCE MEASURES AND SUBJECTIVE USER PERCEPTIONS IN THE EVALUATION OF MEDICAL DEVICES: A CASE STUDY

Published online by Cambridge University Press:  08 December 2015

Matthew D. Haydock ,
Anubhav Mittal ,
Carissa F. Wilkes ,
David H. Lim ,
Elizabeth Broadbent  and
John A. Windsor
Matthew D. Haydock
Affiliation:
Departments of Surgery, Faculty of Medical and Health Sciences, University of Aucklandmdhaydock@icloud.com
Anubhav Mittal
Affiliation:
Departments of Surgery, Faculty of Medical and Health Sciences, University of Auckland
Carissa F. Wilkes
Affiliation:
Psychological Medicine, Faculty of Medical and Health Sciences, University of Auckland
David H. Lim
Affiliation:
Departments of Surgery, Faculty of Medical and Health Sciences, University of Auckland
Elizabeth Broadbent
Affiliation:
Psychological Medicine, Faculty of Medical and Health Sciences, University of Auckland
John A. Windsor
Affiliation:
Departments of Surgery, Faculty of Medical and Health Sciences, University of Auckland Medical Technology Centre of Research Excellence, Auckland Bioengineering Institute, University of Auckland
Get access Rights & Permissions [Opens in a new window]

Abstract

Objectives: Medical technology is a large and expanding industry. Introducing new medical devices is important but several challenges exist in implementing the optimal method of evaluation. Both objective and subjective measures can be used for evaluation. The former is the mainstay of evaluation, yet subjective assessment is often the basis for the introduction of new medical technology. The aim of this study was to determine the interaction and concordance between objective and subjective assessment of new medical technology.

Methods: This study used both objective performance measures and subjective user perceptions in the evaluation of a new medical device designed to improve the accuracy of gravity-assisted delivery of intravenous fluids, compared with the current, widely used “roller-clamp” device. The concordance of objective and subjective assessments was evaluated using comparative analysis.

Results: Objective assessment of the accuracy of intravenous fluid delivery revealed no difference between the two devices (p = .636). Subjective assessment revealed that the new device was perceived to be significantly more accurate (p = .001). This lack of concordance can be partially explained by both device and demand characteristics.

Conclusions: This case study reveals a significant discordance between the objective and subjective assessments. It provides some explanation for why new medical devices are adopted without objective evidence of benefit. This phenomenon has been termed “persuasive design” and its influence should be controlled for in the evaluation, purchase and introduction of new medical devices. This should help reduce the risk and associated cost of premature introduction.

Keywords

Medical technologyMedical devicesEvaluationAssessmentPersuasive designFluid therapy
Type
Assessments
Information
International Journal of Technology Assessment in Health Care , Volume 31 , Issue 5 , 2015 , pp. 297 - 303
Copyright
Copyright © Cambridge University Press 2015 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

*

An addendum notice for this article has been published (DOI: 10.1017/S0266462319000564) and appears at the end of the text as well.

The authors thank the registered and student nurses who kindly volunteered their time to participate in this study, Sir Ray Avery who provided access to materials for testing, the University of Auckland Department of Mechanical Engineering for their technical support and access to their laboratory and Auckland City Hospital for the use of procedure rooms for carrying out the clinical study. Grant support: None. This study is not based on any previous communication

References

REFERENCES

1. Edmondson, AC, Bohmer, RM, Pisano, GP. Disrupted routines: Team learning and new technology implementation in hospitals. Adm Sci Q. 2001;46:685–716.Google Scholar
2. Jennett, B. Health technology assessment. BMJ. 1992;305:67–68.Google Scholar
3. McPherson, K. International differences in medical care practices. Health Care Financ Rev. 1989;(Spec No):9–20.Google Scholar
4. Antman, EM, Lau, J, Kupelnick, B, Mosteller, F, Chalmers, TC. A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts. Treatments for myocardial infarction. JAMA. 1992;268:240–248.Google Scholar
5. Bernard, A, Vaneau, M, Fournel, I, et al. Methodological choices for the clinical development of medical devices. Med Devices (Auckl). 2014;7:325–334.Google Scholar
6. Maisel, WH. Medical device regulation: An introduction for the practicing physician. Ann Intern Med. 2004;140:296–302.Google Scholar
7. de Veer, AJE, Fleuren, MAH, Bekkema, N, Francke, AL. Successful implementation of new technologies in nursing care: A questionnaire survey of nurse-users. BMC Med Inform Decis Making. 2011;11:67.Google Scholar
8. Manu, FA, Sriram, V. Innovation, marketing strategy, environment, and performance. J Bus Res. 1996;35:79–91.Google Scholar
9. Ginsburg, G. Human factors engineering: A tool for medical device evaluation in hospital procurement decision-making. J Biomed Inform. 2005;38:213–219.Google Scholar
10. Shah, SGS, Robinson, I. Benefits of and barriers to involving users in medical device technology development and evaluation. Int J Technol Assess Health Care. 2007;23:131–137.Google Scholar
11. Hróbjartsson, A, Gøtzsche, PC. Is the Placebo Powerless? N Engl J Med. 2001;344:1594–1602.Google Scholar
12. Hróbjartsson, A, Gøtzsche, PC. Is the placebo powerless? Update of a systematic review with 52 new randomized trials comparing placebo with no treatment. J Intern Med. 2004;256:91–100.Google Scholar
13. The Center for Universal Design, North Carolina State University. The Principles of Universal Design. 1997;2013. http://www.ncsu.edu/project/design-projects/udi/center-for-universal-design/the-principles-of-universal-design/ (accessed February 18, 2013).Google Scholar
14. Bissett, IP, Brandt, TP, Windsor, JA. Survey of intravenous fluid therapies and accuracy of gravity-fed infusions in a teaching hospital. Samoa Med J. 2010;2:25–28.Google Scholar
15. Gutierrez Alejandro, A, Calvo Buey, JA, Marcos Camina, RM. [Study for the decrease of errors in the records of hydric balances of critical patients admitted to an intensive care unit]. Enferm Intensiva. 2005;16:100–109.Google Scholar
16. Han, PY, Coombes, ID, Green, B. Factors predictive of intravenous fluid administration errors in Australian surgical care wards. Qual Saf Health Care. 2005;14:179–184.Google Scholar
17. Rooker, JC, Gorard, DA. Errors of intravenous fluid infusion rates in medical inpatients. Clin Med. 2007;7:482–485.Google Scholar
18. Faasse, K, Cundy, T, Gamble, G, Petrie, KJ. The effect of an apparent change to a branded or generic medication on drug effectiveness and side effects. Psychosom Med. 2013;75:90–96.Google Scholar
19. Finniss, DG, Kaptchuk, TJ, Miller, F, Benedetti, F. Biological, clinical, and ethical advances of placebo effects. Lancet. 2010;375:686–695.Google Scholar
20. Moerman, DE. Meaningful placebos–Controlling the uncontrollable. N Engl J Med. 2011;365:171–172.Google Scholar
21. Moerman, DE, Jonas, WB. Deconstructing the placebo effect and finding the meaning response. Ann Intern Med. 2002;136:471–476.Google Scholar
22. de Craen, AJM, Roos, PJ, de Vries, AL, Kleijnen, J. Effect of colour of drugs: Systematic review of perceived effect of drugs and of their effectiveness. BMJ. 1996;313:1624–1626.Google Scholar
23. Branthwaite, A, Cooper, P. Analgesic effects of branding in treatment of headaches. Br Med J (Clin Res Ed). 1981;282:1576–1578.Google Scholar
24. Desharnais, R, Jobin, J, Côté, C, Lévesque, L, Godin, G. Aerobic exercise and the placebo effect: A controlled study. Psychosom Med. 1993;55:149–154.Google Scholar
25. Orne, MT. Demand characteristics and the concept of quasi-controls. Artifacts in behavioral research: Robert Rosenthal and Ralph L Rosnow's Classic Books. 2009:110.Google Scholar
26. Bivins, BA, Rapp, RP, Powers, P, Butler, JL, Haack, D. Electronic flow control and roller clamp control in intravenous therapy: A clinical comparison. Arch Surg. 1980;115:70–72.Google Scholar
27. Crass, RE, Vance, JR. In vivo accuracy of gravity-flow i.v. infusion systems. Am J Hosp Pharm. 1985;42:328–331.Google Scholar
28. Flack, FC, Whyte, TD. Behaviour of standard gravity-fed administration sets used for intravenous infusion. Br Med J. 1974;3:439–443.Google Scholar
29. Rithalia, SV, Rozkovec, A. Evaluation of a simple device for regulating intravenous infusions. Intensive Care Med. 1979;5:41–43.Google Scholar
30. Husch, M, Sullivan, C, Rooney, D, et al. Insights from the sharp end of intravenous medication errors: Implications for infusion pump technology. Qual Saf Health Care. 2005;14:80–86.Google Scholar
31. Himal, HS. Minimally invasive (laparoscopic) surgery. Surg Endosc. 2002;16:1647–1652.Google Scholar
32. Patel, HRH, Linares, A, Joseph, JV. Robotic and laparoscopic surgery: Cost and training. Surg Oncol. 2009;18:242–246.Google Scholar
33. Yarbrough, AK, Smith, TB. Technology acceptance among physicians: A New take on TAM. Med Care Res Rev. 2007;64:650–672.Google Scholar
34. Fogg, BJ. A behavior model for persuasive design. New York, NY: ACM; 2009 doi:10.1145/1541948.1541999Google Scholar
35. Redström, J. Persuasive design: Fringes and foundations. In: IJsselsteijn, WA, de Kort, YAW, Midden, C, Eggen, B, van den Hoven, E, eds. Persuasive technology. Berlin: Springer; 2006:112–122.Google Scholar
Supplementary material: Image

Haydock supplementary material

Supplementary Figure 1a-b

Download Haydock supplementary material(Image)
Image 16.7 KB
Supplementary material: Image

Haydock supplementary material

Supplementary Figure 2

Download Haydock supplementary material(Image)
Image 204.1 KB
Supplementary material: Image

Haydock supplementary material

Supplementary Figure 3

Download Haydock supplementary material(Image)
Image 30.7 KB
Supplementary material: File

Haydock Supplementary Material

Table 1

Download Haydock Supplementary Material(File)
File 74.1 KB
Supplementary material: File

Haydock Supplementary Material

Table 2

Download Haydock Supplementary Material(File)
File 115.7 KB