Introduction
Medication errors in the pediatric patient population are quite prevalent. In the pediatric in-patient setting, Kaushal, et al reported an error rate of approximately 55 errors for every 100 admissions, the majority (28%) being dosing errors, and a potential adverse drug event rate of 10/100 admissions. Reference Kaushal, Bates and Landrigan1 Medication errors have been shown to occur more frequently in the emergency department (ED), and error rates rise even further in the prehospital setting to just under 35% for all medications and over 60% for epinephrine. Reference Benjamin, Frush and Shaw2–Reference Hoyle, Davis, Putman, Trytko and Fales4 These errors are estimated to affect over 21,000 US children under the age of 11 each year. Reference Hoyle, Davis, Putman, Trytko and Fales4
Pediatric medication errors have been attributed to a number of causes, including weight-based dosing and the increased number of calculations required for correct dosing and administration. Reference Kaushal, Bates and Landrigan1 Many of the complications inherent to pediatric medication dosing are potentiated by the unpredictable nature of the prehospital setting. As pediatric medication dosing is predominantly weight-based, accurate assessment is essential. However, in prehospital care, obtaining reliable weight measurements is not straightforward; it is estimated that approximately 20% of out-of-hospital pediatric weight estimates are not accurate. Reference Cicero, Adelgais and Hoyle5
Many tools have been developed to assist with pediatric weight estimation, including age-based formulas, length-based tapes, paperboard dosing wheels, and electronic applications. Reference Cicero, Adelgais and Hoyle5 The Broselow method, which provides a weight estimation based on length as measured using the Broselow tape, is the most commonly used length-based tape. The Broselow method was developed in the 1980s and has been modified and studied fairly extensively in the intervening years, with varying results. Reference Hoyle, Davis, Putman, Trytko and Fales4,Reference Wells, Goldstein, Bentley, Basnett and Monteith6–Reference Sinha, Lezine, Frechette and Foster20
The Handtevy system, initially developed in 2010 as a length and age-based system, has been adapted and can now be used as an exclusively age-based tool for weight estimation. 21 There are few studies evaluating the Handtevy system, most of which were published prior to the development of the solely age-based application, however Rappaport, et al recently published a manuscript reporting almost 90% dosing accuracy using the Handtevy Field Guide (accuracy was based on correct use of the field guide/patient age, not patient weight). Reference Lowe, Campwala, Ziv and Wang12,Reference Rappaport, Brou and Givens17,Reference Rappaport, Markowitz, Hulac and Roosevelt22 There appear to be no studies in the published literature evaluating the accuracy of the Handtevy weight estimation method in the prehospital setting.
The primary objective of this study was to assess the field performance of the Broselow tape and the Handtevy method with respect to prehospital pediatric patient weight estimations. Secondary objectives included evaluating accuracy of medication dosing, the percentage of patients placed in accurate weight categories using the Broselow method, and comparing ED weight by age to predicted Handtevy weights.
Methods
This retrospective cohort study included trauma and non-trauma pediatric patients transported via Emergency Medical Services (EMS) to the ED of a Level 2 trauma center/Comprehensive Children’s Receiving Center from January 1, 2021 through June 30, 2021. Data were collected via retrospective chart review of ED records by a team of trained data abstractors using a standardized REDCap (Vanderbilt University; Nashville, Tennessee USA) data collection form. Demographics, EMS provider, prehospital weight estimation, prehospital medication/dose, ED weight, ED intubation status, and ED disposition were collected. Prehospital weight estimation method was based on EMS provider policy and statistics were calculated based on an intention to treat model. Scale-based weight from the ED was used as the standard for comparison.
All patients transported via EMS to the study institution for the duration of the study were included unless they met exclusion criteria. Patients with cerebral palsy and contractures were excluded, as were any interfacility transports (including transports from skilled nursing facilities), patients for whom ED weights were not obtained using a scale, and patients whose charts contained incomplete study data. Abstractors were trained research assistants who followed a standardized operating manual for data abstraction (including instructions regarding where each variable may be found in the chart). Abstractors completed an iterative inter-rater reliability process in which each abstractor abstracted variables for 10 randomly selected charts and any discrepancies were analyzed by TH, PKP, and CKR. Necessary changes to the operating manual identified by this process were completed between iterations. Following complete data abstraction, simple random sampling was used to select 10% of charts for repeat abstraction to assess data entry reliability. REDCap data were screened and cleaned by TH, PKP, and CKR and EMS runs with insufficient or inconsistent data were excluded.
As the intended scope of the Broselow method was predominantly directed towards children <13 years of age, and a majority of studies include subsets of children younger than this, the Handtevy and Broselow comparisons regarding weight estimation and medication dosing was limited to those under 13. Reference Wells, Goldstein, Bentley, Basnett and Monteith6,Reference Lowe, Campwala, Ziv and Wang12,Reference Rappaport, Markowitz, Hulac and Roosevelt22,Reference Rosenberg, Greenberger, Rawal, Latimer-Pierson and Thundiyil23 A secondary aim of this study was to compare the Handtevy suggested weight estimation based on age to ED weights. As the Handtevy method is designed for patients ≤13 years, children ≤13 years of age were included in this additional analysis.
In this study, a single EMS agency (Provider “Z”) used the Handtevy method. This provider transports >50% of patients transported via EMS to the study institution. Provider Z transitioned from the Broselow method to the Handtevy method midway through the study, allowing for three months of baseline and three months of comparative data to be collected. Related analyses will subsequently be described as “pre-Handtevy” Z-Provider or “post-transition” Z-Provider. All other EMS agencies employed the Broselow method and will be referred to collectively as “Y-Providers.” The Broselow tape used by EMS in this study was the Broselow 2007 edition, version B; the Handtevy method used in this study was v4.2.7.
Only weight-based EMS-administered medications were considered for this study; these included adenosine, amiodarone, atropine, dextrose, diphenhydramine, epinephrine, fentanyl, hydroxocobalamin, lidocaine, midazolam, morphine, and sodium bicarbonate. Dosing was considered accurate based on ED weight and county policy. Patients transported via out of county EMS providers were excluded from medication analysis. This study was approved by Children’s Health of Orange County (Orange, California USA) In-House Institutional Review Board (#210682).
Effect Size Calculations
A pair of effect size calculations were conducted using G*Power (version 3.1.9.3; Faul and Erdfelder; Germany), since the total sample size was bounded by the specific evaluation period (January 1, 2021 through June 30, 2021). Reference Faul, Erdfelder, Lang and Buchner24 The first effect size calculation was conducted under the family of exact tests, in which statistical power was set to 90% with two-tailed alpha at 0.05, and 54% was entered as the base accuracy rate of the Broselow method. Reference Young and Korotzer18 For the effect of the Handtevy method to be statistically significant, its accuracy rate would have to differ from the Broselow method by at least 15% (ie, Handtevy accuracy rate ≥69% or ≤39%).
The second effect size calculation was conducted under the family of t-tests, in which statistical power was set to 90% with two-tailed alpha at 0.05. For the effect of the Handtevy method to be statistically significant, its mean percent weight discrepancy would have to differ from the Broselow method by at least six percent if the accompanying standard deviation is large (eg, standard deviation of 20% for Handtevy mean percent weight discrepancy of five percent).
Statistical Analysis
Chi-squared test was used to analyze categorized weight discrepancy (over, under, or accurate estimates – defined as within 10% of ED scale weight) and the Broselow/Handtevy methods; independent samples t-test was used to analyze percent weight discrepancy and the Broselow/Handtevy methods. Percent weight discrepancy was calculated as [(EMS weight – ED weight)/ED weight]x100. Negative values signified EMS under-estimation in patient weight, while positive values signified EMS over-estimation of patient weight. Chi-squared test was used to analyze categorized weight discrepancy and Z pre-Handtevy/Z post-transition/Y; three-way between-subjects ANOVA (Handtevy/Broselow methods; non-Hispanic/Hispanic ethnicity; age groups) was used to further analyze percent weight discrepancy. Chi-squared test with Monte Carlo simulation was used to analyze medication dosing (under, over, and accurately dosed based on ED scale weight) and the Broselow/Handtevy methods. Chi-squared test with Monte Carlo simulation was also used to analyze medication type and dosing. Statistical analyses were performed using IBM SPSS Statistics, version 26 (IBM Corp.; Armonk, New York USA).
Results
Patient Characteristics
A total of 509 EMS transports were analyzed to address the primary objective of this study. Overall characteristics for this sample of patients <13 years of age were as follows: 265 (52.1%) were male; 297 (58.3%) were Hispanic; 348 (68.4%) had public insurance; five (1%) were intubated in the ED; 369 (72.5%) were discharged home; 77 (15.1%) were admitted to the general floor or to surgery; 23 (4.5%) were admitted to the pediatric intensive care unit (PICU); eight (1.6%) were transferred to an external medical facility; 31 (6.1%) were admitted to this institution’s in-patient mental health unit or transferred to an external mental health facility; and one (0.2%) died. Further data including frequencies by Provider-Z pre-Handtevy, Provider-Z post-transition, and Providers-Y may be found in Table 1.
Table 1. Patient Characteristics by EMS Provider and Weight Estimation Method
Abbreviations: ED, emergency department; PICU, pediatric intensive care unit.
Prehospital Weight Estimation (Table 2a and Table 2b)
Table 2a. Weight Discrepancy by Weight Estimation Method
Abbreviations: ED, emergency department; EMS, Emergency Medical Services.
a Calculated as [(EMS weight – ED weight)/ED weight]x100. When the value is negative, EMS under-estimated; when the value is positive, EMS over-estimated.
Table 2b. Weight Discrepancy by Agency and Weight Estimation Method
Abbreviations: ED, emergency department; EMS, Emergency Medical Services.
a Calculated as [(EMS weight – ED weight)/ED weight]x100. When the value is negative, EMS under-estimated; when the value is positive, EMS over-estimated.
The EMS providers using the Broselow method estimated patient weight within +/-10% of ED scale weight in 51.3% of patients (ie, prehospital weight was not over or under ED scale weight by more than 10%). When using the Handtevy method, EMS providers estimated patient weight to within +/-10% of ED scale weight in 43.7% of patients. When comparing the Handtevy versus Broselow method of prehospital weight estimation, there was no significant association between method and categorized weight discrepancy (over, under, or accurate estimates – defined as within 10% of ED scale weight; P = .25), as well as no significant difference in percent weight discrepancy (P = .75). On average, prehospital weight estimations were 6.7% lower than ED scale weight for the same patient.
In addition, when stratifying by Provider-Z pre-Handtevy, Provider-Z post-transition, and Y-Providers, there was no association with categorized weight discrepancy (P = .46). When percent weight discrepancy was further analyzed, significant interactions were found of the Handtevy/Broselow methods with non-Hispanic/Hispanic ethnicity (P = .005) and with age groups (P = .02). Data visualizations show substantial percent weight discrepancies from these interactive effects (Figure 1 and Figure 2). Handtevy weight estimations were noticeably less discrepant for non-Hispanic patients compared to Hispanic patients. The Broselow method tended to under-estimate weight for both non-Hispanic and Hispanic patients.
Figure 1. Mean and 95% CI of Percent Weight Discrepancy by Age Groups.
Note: Given only two Handtevy EMS runs in the 4-5.99mo age group, no confidence interval was outputted.
Abbreviation: EMS, Emergency Medical Services.
Figure 2. Mean and 95% CI of Percent Weight Discrepancy by Ethnicity.
Note: Ethnicity data missing from four EMS runs.
Abbreviation: EMS, Emergency Medical Services.
Medication Dosing
Pediatric Intensive Care Unit (PICU)—There were only 22 EMS runs in which the medications of research interest were administered (n = 24 discrete medications); 10 patients received midazolam, four received fentanyl, three patients received diphenhydramine, and three received epinephrine. Two patients received both diphenhydramine and epinephrine. There was no significant association between medication dosing (under, over, or accurate dosing) and use of either the Broselow or Handtevy weight estimation methods (P = .85; Table 3a). Additionally, there was no significant association between medication type and dosing (P = .63; Table 3b).
Table 3a. Medication Dosing Accuracy by Weight Estimation Method
Table 3b. Medication Dosing Accuracy by Medication
Broselow (Table 4 )—Among patients for whom the Broselow method of estimation was used, 42.7% had estimated weights which placed them into the correct Broselow category according to their ED weight.
Table 4. ED Scale Weights Compared to Broselow Predictions for Patients <13 Years of Age a
Abbreviations: ED, emergency department; EMS, Emergency Medical Services.
a For patients transported via providers using the Broselow method of weight estimation, if no Broselow color was included on the EMS chart, a Broselow color was assigned based on the weight estimation. Of 312 Broselow EMS runs, one had a prehospital weight of 1.4 kg (below Broselow gray), but an ED scale weight of 4.33kg. Another EMS run had a prehospital weight of 40.8kg but was documented as having a length above the Broselow black zone; ED scale weight was 77.4kg. Additionally, 22 EMS runs had a prehospital weight ≥50kg (above the Broselow black zone), but their ED scale weights ranged from 39 to 95.6kg (M=65.7kg, SD=13.5kg, Mdn=63.3kg).
Handtevy—Table 5 demonstrates the Handtevy-suggested weight estimation (based on age) compared to ED weights for patients of corresponding age included in this study; 40.7% of patients had estimated weights which placed them in the correct Handtevy “category” according to their ED weight.
Table 5. ED Scale Weights Compared to Handtevy Suggested Weights by Age a
Abbreviation: ED, emergency department.
a Calculated as [(Predicted weight – mean ED scale weight)/Predicted weight]x100 or as [(Predicted weight – median ED scale weight)/Predicted weight]x100. Handtevy under-estimation when the value is negative, Handtevy over-estimation when the value is positive.
Discussion
This study demonstrated no overall significant difference between the use of the Handtevy and Broselow methods with respect to the accuracy of prehospital weight estimation. This is in slight contrast to a systematic review by Young, et al who found that the Broselow method generally outperformed age-based formulas. Reference Young and Korotzer18 Additionally, Lowe, et al compared theoretical Broselow and Handtevy (the length/age based edition) weight estimations based on National Health and Nutrition Examination Survey data and demonstrated increased accuracy of the Broselow tape, although the Handtevy system was more accurate for taller children. Reference Lowe, Campwala, Ziv and Wang12 However, it is notable that many previous studies of weight estimation techniques have not examined their use in the prehospital care setting. Reference Wells, Goldstein, Bentley, Basnett and Monteith6,Reference Saeed, Talathi and Suneja7,Reference Wells and Goldstein9–Reference Lowe, Campwala, Ziv and Wang12,Reference Nieman, Manacci, Super, Mancuso and Fallon14,Reference Tanner, Negaard, Huang, Evans and Hennes15,Reference Young and Korotzer18,Reference Sinha, Lezine, Frechette and Foster20
Despite this, the weight estimation accuracy of the Broselow method in this study (51.3% of patient weights were within +/-10% of ED scale weights) was comparable to previous literature. Reference Wells, Goldstein, Bentley, Basnett and Monteith6,Reference Young and Korotzer18 Perhaps more importantly, EMS allocated 42.7% of patients to the appropriate Broselow category and 40.7% of patients to the appropriate Handtevy category. This is practically more relevant as medication dosing for pediatric patients is determined by weight category.
On average, the Broselow tape under-estimated patient weight by 6.9% and review of the literature demonstrates similar trends, although Both, et al examined data of over 3,000 patients and found that Broselow tended to over-estimate patient weight more frequently than under-estimate patient weight. Reference Saeed, Talathi and Suneja7,Reference Both, Schmitz, Buehler, Weiss and Schmidt10,Reference Waseem, Chen, Leber, Giambrone and Gerber11 Use of the Handtevy method was also associated with under-estimation of patient weight, on average approximately 6.3%. This is consistent with work by Lowe, et al who found that the Handtevy method tended to under-estimate patient weight. Reference Lowe, Campwala, Ziv and Wang12
Based on the current study’s findings regarding weight under-estimation and documented concerns in the literature regarding the effect of the growing obesity epidemic on prehospital weight estimation, this study sample’s scale weight was further analyzed by age compared to the Handtevy application’s weight prediction. Reference Nieman, Manacci, Super, Mancuso and Fallon14,Reference Tanner, Negaard, Huang, Evans and Hennes15,Reference Young and Korotzer18,Reference Sinha, Lezine, Frechette and Foster20 Overall, patients in this study weighed more than the Handtevy estimations, with the most profound differences occurring in Hispanic patients and those seven to eight years of age. It is not possible to suggest alterations to the Handtevy estimations or Broselow tape based on this study, as the dangers of over versus under dosing must be taken into account. However, it may be of some benefit for pediatric hospitals to study the average weights of their individual patient population and advise the relevant EMS providers accordingly.
Although the statistical analysis for this medication analysis was severely limited due to the small n, this study showed no significant difference between the use of the Handtevy or Broselow methods with respect to the medication dosing errors. The majority of medication errors in this study were due to incorrect weight estimation and a minority were due to EMS administering the incorrect dose for the estimated weight. Medication dosing error rates were quite high, but were not associated with a particular weight estimation method. Fifty percent of medications were inaccurately dosed. This is similar to the base rate found by Rappaport, et al, though well exceeds the rate of 34.7% reported by Hoyle, et al who studied records of over 5,000 children treated by paramedics. Reference Hoyle, Davis, Putman, Trytko and Fales4,Reference Rappaport, Markowitz, Hulac and Roosevelt22 The increased error rates in this study may stem from differing definitions of medication error, the much smaller sample size in the current study, or weight estimation/provider error. The small sample size of the medication analysis is limiting, yet highlights the infrequent nature of EMS pediatric medication administration. In this study, weight-based medications were only given in 4.3% of EMS runs, very similar to the rate reported by Rappaport, et al (4.9%). Reference Rappaport, Markowitz, Hulac and Roosevelt22 This rarity in itself is a risk factor for dosing errors. Reference Cicero, Adelgais and Hoyle5
Limitations
A major limitation of this study is that the determination of prehospital weight estimation method was based entirely on EMS provider policy. It is possible that EMS providers may have used alternative methods which could confound the comparisons performed in this study and affect the results in unquantifiable ways. There are some data suggesting that EMS providers in this study, at least occasionally, used alternate methods; among patients where provider protocol dictated use of the Handtevy method, only 40.3% had documented prehospital weights corresponding to selections available on the Handtevy application. However, a comparison of estimation techniques still provides summative information, despite the inability to confirm fidelity to technique. Policy adherence will inevitably be a confounding variable, and real-world implementation data should be an important consideration for EMS with regards to policy decisions.
Additionally, this analysis was conducted just as Provider-Z transitioned from the Broselow method to the Handtevy method. This may have potentially led to errors due to unfamiliarity with the system. The small numbers of medications administered in this study may have led to Type 2 error. Other limitations include that since EMS estimated weights were analyzed as officially documented in the patient care report, EMS documentation errors are possible. Of note, although weights were compared to infant, standing, or bed scales regularly used in the ED, the accuracy of these scales was not verified outside of standard maintenance.
Conclusion
This study demonstrated no statistically significant difference between the use of the Handtevy or Broselow methods with respect to either prehospital weight estimation or medication dosing. Overall, EMS tended to under-estimate patient weight, and actual weight tended to exceed Handtevy predictions. Further research is necessary in the prehospital care setting as this appears to be one of the first studies comparing these methods in practice. However, these results suggest similar field performance of the Broselow and Handtevy methods.
Conflicts of interest
Claus Hecht is the medical director for an EMS provider involved in the current study. Carl Schultz is the EMS medical director for the county in which the study took place. Neither were involved in study design, data collection, or data analysis. All other authors declare no competing interests.
Open access
