Using machine learning with passive wearable sensors to pilot the detection of eating disorder behaviors in everyday life

Abstract

Background

Eating disorders (ED) are serious psychiatric disorders, taking a life every 52 minutes, with high relapse. There are currently no support or effective intervention therapeutics for individuals with an ED in their everyday life. The aim of this study is to build idiographic machine learning (ML) models to evaluate the performance of physiological recordings to detect individual ED behaviors in naturalistic settings.

Methods

From an ongoing study (Final N = 120), we piloted the ability for ML to detect an individual's ED behavioral episodes (e.g. purging) from physiological data in six individuals diagnosed with an ED, all of whom endorsed purging. Participants wore an ambulatory monitor for 30 days and tapped a button to denote ED behavioral episodes. We built idiographic (N = 1) logistic regression classifiers (LRC) ML trained models to identify onset of episodes (~600 windows) v. baseline (~571 windows) physiology (Heart Rate, Electrodermal Activity, and Temperature).

Results

Using physiological data, ML LRC accurately classified on average 91% of cases, with 92% specificity and 90% sensitivity.

Conclusions

This evidence suggests the ability to build idiographic ML models that detect ED behaviors from physiological indices within everyday life with a high level of accuracy. The novel use of ML with wearable sensors to detect physiological patterns of ED behavior pre-onset can lead to just-in-time clinical interventions to disrupt problematic behaviors and promote ED recovery.

Introduction

Eating disorders (EDs) are serious psychiatric illnesses, taking a life every 52 minutes (Deloitte Access Economics, 2020). Only ~50% of adults with EDs respond to existing evidence-based treatments (Bulik, Berkman, Brownley, Sedway, & Lohr, 2007; Keel & Mitchell, 1997; Steinhausen, 2009; Steinhausen & Weber, 2009; van den Berg et al., 2019). Relapse is common (Walsh, Xu, Wang, Attia, & Kaplan, 2021), which contributes to the ‘revolving door phenomenon’ where individuals cyclically admit and discharge from ED treatment. Given the high mortality rate associated with EDs (Arcelus, Mitchell, Wales, & Nielsen, 2011) and the financial burden of accessing treatment (Deloitte Access Economics, 2020), improved treatment that is effective for more individuals seeking recovery from EDs is critical. Three factors contribute to the suboptimal treatment response and high relapse rates: (1) heterogeneity among those with EDs, even within the same diagnostic group (Levinson et al., 2022; Thompson, Berg, & Shatford, 1987); (2) lack of day-to-day and moment-to-moment support in an individual's naturalistic environment; and (3) the lack of knowledge of how individual heterogeneity in cognitions, emotions, and behaviors impact recovery. Thus, there is a need for passive treatment support methods specifically available for use in an individual's everyday life.

Understanding psychological phenomena in naturalistic settings represents one approach that could lead to improved treatments and reduced relapse rates. Technological advancements have facilitated new areas of clinical research that better bridge empirical research to ‘everyday life’, including the use of ambulatory assessment (e.g. ecological momentary assessment [EMA]; Stone and Shiffman, 1994) to inform psychological interventions (Wright & Zimmermann, 2019). One of the most common methods of ambulatory assessment used in the past decade is time-intensive repeated survey measurements delivered through mobile application devices (e.g., Hasselhorn, Ottenstein, and Lischetzke, 2022). EMA designs have been increasingly implemented in investigating EDs, yet most EMA research has relied solely on self-report instruments (Presseller, Patarinski, Fan, Lampe, & Juarascio, 2022; Schaefer, Engel, & Wonderlich, 2020; Smith et al., 2019). Though EMA ostensibly captures dynamic symptom relations in everyday life, intensive self-report approaches have some limitations, such as: participant burden from participants repeatedly completing surveys, subjectivity, social desirability, problems with self-reflection in EDs, and the lack of complete understanding regarding how momentary changes in other relevant process, such as neurocognition, location, and biology may impact ED symptoms (Smith et al., 2019). Collectively, EMA work suggests that ED behaviors and cognitions fluctuate as a function of time and person (Lavender et al., 2013; Levinson et al., 2022), highlighting the importance of repeated assessment and analysis at the individual level due to the high heterogeneity from person-to-person even within the same ED diagnostic category.

Along with these explicit momentary measures there are more implicit approaches, such as research examining physiological correlates of EDs, which has largely been conducted in laboratory settings (Christian, Cash, Cohen, Trombley, & Levinson, 2023; Presseller et al., 2022). However, such studies related to EDs are sparse, and laboratory-based designs lack the ecological validity needed for support and interventions in individuals' everyday lives. Wearable sensors, such as the E4 Empatica band, have demonstrated good reliability and validity to be used to passively collect common physiological data such as blood-volume pulse (heart rate [HR]), electrodermal activity ([EDA]; skin resistance and conductance variation, controlled by the sympathetic nervous system in response to arousal), and peripheral skin temperature (PST) in everyday life in a variety of populations (e.g., McCarthy, Pradhan, Redpath, and Adler, 2016; Park, Jeong, Park, and Lee, 2019; Ragot, Martin, Em, Pallamin, and Diverrez, 2018; Ravindran et al., 2022; Schuurmans et al., 2020; van Lier et al., 2020). Preliminary findings using wearable sensor devices reveal physiological recordings that correspond with self-reported ED symptoms. For example, continuous glucose monitoring demonstrates that glucose levels correspond with self-reported bulimic symptoms of fasting, binge eating, and purging (Presseller, Parker, Lin, Weimer, & Juarascio, 2020). As individuals with ED may be particularly limited in their ability to accurately know and report their emotions, cognitions, and behaviors in the moment, evidence suggests that physiological recordings may be used to accurately provide real-time assessment of loss of control eating (Ranzenhofer et al., 2016) without the burden or limitations of intensive self-report. Additionally, a new review (Ralph-Nearman et al., 2023, under review) demonstrates that physiological patterns often differ between different types of ED diagnoses and behaviors (e.g. restriction v. purging/compensation). These results specifically suggest that EDs characterized by purging behaviors (e.g., BN) have changes in HR reactivity, EDA responses, and PST associated to ED-related stimuli, which may differ from other EDs characterized by restriction or binge-eating (Krantz et al., 2020; Ortega-Roldan, Rodríguez-Ruiz, Perakakis, Fernandez-Santaella, & Vila, 2014; Papežová, Yamamotova, & Uher, 2005). Thus, physiological recordings, such as HR, EDA, and PST, obtained in individuals’ daily lives detected with enough time pre-onset of ED behaviors may preemptively indicate ED symptom engagement and, subsequently, precise time points of intervention before an individual is aware of these changes (Juarascio, Parker, Lagacey, & Godfrey, 2018; Levinson, Christian, Shankar-Ram, Brosof, & Williams, 2019; Smith & Juarascio, 2019).

More recent implicit approaches have also leveraged passive data collection (e.g. wearable sensors, smartphones) with machine learning (ML) to detect, predict, or intervene on clinical phenomena in other types of disorders outside EDs. For example, passive physiological recordings, such as those utilized in the present pilot study (HR, EDA, PST), have been shown to be able to detect or predict depressive symptoms (Zarate, Stavropoulos, Ball, de Sena Collier, & Jacobson, 2022), variability in anxiety and avoidance symptoms (Jacobson & Bhattacharya, 2022), changes in perceived safety and discomfort (Welch et al., 2022), and behavioral outbursts in real-time in samples with major depressive disorder, anxiety disorders, and autism spectrum disorder (Alban et al., 2021; Goodwin, Mazefsky, Ioannidis, Erdogmus, & Siegel, 2019; Northrup et al., 2022; Welch et al., 2023). Such research has demonstrated robust detective and predictive performance, which has led to providing preventative support and real-time intervention for unhelpful behaviors across a range of mental and physical issues (e.g. Clifton, Clifton, Pimentel, Watkinson, and Tarassenko, 2014; Regalia, Onorati, Lai, Caborni, and Picard, 2019).

Importantly, knowledge on EDs predominately rests upon scholarship that uses explanatory methods (e.g. statistical inference) (Wang, 2021). However, to advance the ED field from modeling existing data to forecasting unseen data (i.e. future behavior), ML approaches are well-suited for detecting the onset of ED behavior and, potentially, in turn, allowing for the delivery of evidence-based intervention before relapse may occur (Wang, 2021).

In this pilot we aim to use minimal physiological features, none requiring extensive data cleaning, with machine learning, in order to be easily implemented and used in the future within the real-world context. With the future aims in mind, scientifically we choose to start with these three measures (HR, EDA, and PST) which are shown to be able to detect or predict other problematic symptoms and behaviors. HR, EDA, and PST are features which are commonly detected from wearables and have been shown to have the ability to detect or predict depressive symptoms, changes in perceived safety and discomfort, and behavioral outbursts in real-time.

Thus, the current pilot study uniquely examines the detective performance of ML models using physiological recordings collected via a wearable sensor device to classify ED behaviors among six individuals diagnosed with an ED who endorsed purging behaviors. As research on physiological recordings from wearable sensors (Levinson et al., 2019; Presseller et al., 2022) and ML is limited in ED literature (Wang, 2021), the current study was an exploratory pilot. Our principal aim was to build idiographic (N = 1) detective models using passively collected physiological recordings commonly detected from wearables (HR, EDA, and PST, e.g. Christian et al., 2023; Welch et al., 2023) to evaluate ML model performance in detecting individual ED behaviors in their everyday life.

Method

Participants

A convenience sample of the first six participants, ranging from 20 to 38 years old (M_age = 29.5; s.d. = 7.3) who were diagnosed with an ED endorsing purging with adequate data, were selected from an ongoing study (N = 120) for the present pilot. This current pilot sample was recruited, and data collected representatively of the participants within the on-going study. The ongoing study recruited individuals with EDs residing in the United States to participate in a study that uses mobile and sensor technology to predict ED relapse and recovery. Participants were recruited via advertisements on social media, alumni lists from treatment centers, and research participants who have consented to be contacted about research studies. Participants were compensated with an Amazon or Target gift card valued between $25 to $110 for their participation based on the number of days the sensor band was worn and the number of EMA surveys completed.

Procedure

All study procedures were approved by the University of Louisville's Institutional Review Board (22.0503), and all participants provided verbal informed consent. Participants completed an initial phone screening to determine eligibility. To participate, individuals had to meet criteria for an active or partial-remission diagnosis of anorexia nervosa (AN), bulimia nervosa (BN), or otherwise specified ED-atypical AN (OSFED-AAN). Exclusion criteria were: active suicidality, mania, psychosis or medical instability (i.e., endorsing current chest pain, dizziness, shortness of breath, blurred vision, seeing dark spots within the past 24 hours, purging more than three times within the past 24 hours, or consuming fewer than 500 calories within the past 48 hours). Eligible participants completed an online baseline questionnaire where they reported ED and comorbid psychiatric symptoms. The research team mailed participants an Empatica E4 wristband, which participants were instructed to wear for 30 days and to use to indicate ED behaviors, charging the band at night.

Measures

Demographics and diagnoses

Participants self-reported their gender, race, age, comorbidities, and socioeconomic status, and all participants endorsed using medication (7 psychiatric and 2 non-psychiatric medications; see Tables 1 and 2). The Structured Clinical Interview for DSM-5 (SCID-5; First, Williams, Karg, and Spitzer, 2015) ED modules and the ED Diagnostic Scale (EDDS; Stice, Telch, and Rizvi, 2000, 2004) were used to determine eligibility criteria and current ED diagnosis.

Table 1. Sociodemographic characteristics of participants

Note. AN, anorexia nervosa; BN, bulimia nervosa; A, Atypical; BP, Binge-Purge; p, Purge; OSFED, other specified feeding or eating disorder; OCD, obsessive compulsive disorder; PTSD, post-traumatic stress disorder; GAD, generalized anxiety disorder. ABNP (OSFED) meets all the criteria for BN, with less frequency/duration (<1 × per week/<3 months).

Table 2. Descriptives of frequency of ED behaviors, days and time band was worn, age, and medications by participant

Physiological recordings

Over 30 days, we continuously collected the following physiological measurements passively across the waking hours of individuals using the Empatica E4 wristband, which collects continuous physiological indices, of which the present pilot will use indices common within literature to detect onset of disorders outside EDs (HR, EDA, and PST). Immediately after the screening for eligibility, participants were instructed by a researcher about how to use the wearable and were also given contact information in case they had questions or problems. At this time, participants' instructions included how to put the sensor band on in the morning when waking, to take off the sensor band when showering or swimming, how to endorse an ED behavior, how to endorse exercise and eating, and how to charge the E4 during sleeping hours. Empatica sensor-technology is used in the health sciences to identify physiological patterns associated with illness behaviors (Bidwell, Khuwatsamrit, Askew, Ehrenberg, & Helmers, 2015).

ED behaviors

ED behaviors were assessed broadly, and as such, could reflect a variety of specific behaviors, including self-induced vomiting, laxative or diuretic use, binge eating, restricting food intake, and excessive exercise. During clinical screening, and after meeting eligibility, eating disorder behavior examples and definitions were given to participants, and participants were asked if they had any questions (see online Supplementary Table S1). Participants endorsed engaging in an ED behavior by tapping twice on the Empatica E4 wristband button (88 episodes across participants across the 30 days). At the end of each day wearing the sensor band, participants were asked whether or not they tagged all ED behaviors. On this evening log, across participants, across the 30 days participants reported missing a collective of only five ED behaviors.

Data analytic plan

Preprocessing

Raw physiological data were visually checked for stability readings, and then archived into a postgreSQL relational database through Python's sqlalchemy library (Bayer, 2012). Overall, participants wore the band from 14 to all 30 full days (M = 21.83; s.d. = 6.24), on average per day for 12 hours, 29 minutes, and 7 seconds, and participants endorsed 4 to 42 ED behaviors (M = 14.67; s.d. = 13.82) (see Table 2).

We explored if we would be able to detect the pre-onset of ED behaviors - through HR, EDA, and PST physiological indices - to allow enough time for future intervention pre-onset. Therefore, we queried each participant's physiological data from 20 min prior to a behavioral episode tag and extracted them into ‘windows’ (time periods of analysis), similarly to detecting other types of behaviors (e.g. Welch et al., 2023). For each window, to best line up the different signals which were at different hertz (Hz; EDA and PST at 4 Hz, and HR at 1 Hz), we resampled each physiological signal to 4 Hz and used Python to timestamp-match all the signals. As each raw signal is exported independently from the wristband, each discrete data point was provided a Coordinated Universal Time (UTC) timestamp based on the initial UNIX timestamp provided designating the beginning of the recording. The UTC timestamps were then used to align signals to each other during resampling. Next, we derived features from each window (e.g. mean, median, minimum, maximum, standard deviation, root mean square [RMS], mean absolute deviation [MAD], mean absolute value [MAV], 25th percentile, 75th percentile) for each signal for use in our ML algorithms. Based on the three raw sensor readings (HR, EDA, and PST) and 10 derived features, we used a 30-dimensional feature vector to represent each window in ML classification analysis.

Model construction

Following synthesis of the 30 features per window for each individual, the data were separated into two classes: baseline windows (class 0) and pre-onset ED behavior windows (class 1). Once the data were separated into ~100 baseline and ~100 behavior feature vectors through randomized subsampling, we applied a logistic regression classifier (LRC) to explore which models differentiated between vectors provided for the two classes. Models were encoded and tested using Python (Van Rossum & Drake, 2009). To ensure that sufficient training data was available for each model (~90 samples), as well as to minimize overfitting 10-fold cross-validation was utilized. This approach ensures that each model used to classify a testing data sample does not use the same sample during training. Average performance measures are provided across individual cross-validation performances. We built idiographic (N = 1) LRC ML trained models to identify personalized onset of episodes (~600 windows) v. baseline (~571 windows) physiology (HR, EDA, and PST). Accuracy classified cases (True Positives (TP) + True Negatives (TN)/Total), specificity classified behavioral episodes (TN/(TN + False Positives (FP))), and sensitivity classified baseline physiological classification (TP/(TP + False Negatives (FN))). Acceptable classification performance was defined as >70% accuracy of classified cases, sensitivity to classify baseline physiological classification, and specificity to classify behavioral episodes (Swets & Picket, 1982).

Results

Table 3 provides idiographic detective performance. Using physiological data, LRC classified 91% of episodes accurately (Range = 0.84–0.99). Specificity estimates (e.g. classifying behavioral episodes) averaged 92% accuracy (Range = 0.82–1.00). Sensitivity (e.g. baseline physiology classification) averaged 90% (Range = 0.84–0.99).

Table 3. Accuracy, specificity, and sensitivity of idiographic LRC models (N = 1)

Discussion

Lack of treatment response in EDs and support in everyday life contributes to EDs being among the deadliest mental health disorders, second to opioid deaths (Berends, Boonstra, & Van Elburg, 2018). This reality indicates the urgent need for new treatment support methods available for use in everyday life. Physiological recordings from wearables have offered clinical utility in forecasting other clinical phenomena, which has led to providing preventative support for problematic behaviors. For example, via wearable or mobile phone, individuals and caregivers are notified of the risk of behavioral episodes, such as the pending onset of epileptic seizures or outbursts by persons with autism prior to onset so that steps may be taken to intervene (e.g. Clifton et al., 2014; Regalia et al., 2019). The current study expands upon previous research examining physiological correlates of ED behaviors in laboratory settings (see Presseller et al., 2022 for review) by investigating these phenomena in naturalistic settings idiographically using ML. Our pilot findings suggest the ability for ML to detect the pre-onset of ED behaviors in a personalized manner with passive wearable data collection in individuals' everyday life 20 minutes pre-onset, similar to detection of pre-onset of other mental health problems, such as depressive symptoms, anxiety and avoidance symptoms, and pre-onset of behavioral outbursts (Alban et al., 2021; Goodwin et al., 2019; Jacobson & Bhattacharya, 2022; Welch et al., 2022; Welch et al., 2023; Zarate et al., 2022). All idiographic model ML algorithm abilities (accuracy, specificity, and sensitivity) demonstrated well above the 70% acceptable performance for each individual (Swets & Picket, 1982; ranging from 84–99% accuracy, 82–100% specificity, and 84–99% sensitivity), which points to the strength of this method despite the high heterogeneity of symptoms and behavior frequencies, even within the same ED diagnosis. These results demonstrate the ML models' successful detective abilities to provide personalized ED behavior detection within individuals in everyday life despite the heterogeneity of EDs, a range of ED diagnostic types (i.e., BN, AN, AAN, ABN), variation in ED behavior frequency, and other individual differences. As such, wearable sensors may represent a method by which individuals can receive personalized support to their wearable or mobile phone within their environment to provide a warning of risk with a digital therapeutic intervention 20 min pre-onset of ED behavioral episodes, to give enough time to prevent ED behaviors, such as purging, that lead to relapse.

We had a small sample size, and as our goal was to develop idiographic detection, it does not necessitate a large sample. It has been posited that N = 5 or greater is sufficient for establishing direct replication in single-case designs (e.g. Barlow and Hersen, 1973; Hensen and Barlow, 1984; Kazdin, 2011). Although there was some diversity within our pilot regarding racial and ethnic background, gender and sexual identity, and various EDs, more diverse samples should be included in future studies for generalizability. Additionally, participants endorsing purging behaviors may have indicated other active (e.g., exercise) and passive (e.g., body checking) ED behaviors, and though the momentary endorsement of ED behaviors were cross-checked with end-of-day logs, there is no way to control for accuracy of annotations. Future research should assess the detective abilities of physiological recordings toward behaviors not explicitly examined in the present study, investigate if some physiological indicators may better classify ED behaviors than others. With a larger sample, we may explore (1) population models and (2) how we may incorporate population-level data into informing idiographic patterns. Overall, results from the current research suggest that ML algorithms using physiological recordings can detect ED behaviors. Because these methods have the potential to identify instances of risk for maladaptive onset of ED behaviors, physiological recordings from wearables may be integrated into timely digital interventions in the future, such as just-in-time adaptive interventions (JITAIs) to provide personalized interventions at the moments most needed (e.g., delivered via smartphones; Juarascio et al., 2018). For instance, a detection of the onset of risk of an ED behavior may trigger a JITAI to disrupt the behavior and intervene in a timely manner. Next steps may pinpoint the most essential indicators to best classify the pre-onset of ED behaviors, as well as other types of data which may compliment and strengthen these efforts.