Introduction
The impact of COVID-19 social restrictions on the mental health of students in education has been well documented. Reference Bu, Steptoe and Fancourt1 Prior to COVID-19, the challenges of social and academic progress associated with higher education were already linked to the development or exacerbation of mental health issues. Reference Papadatou-Pastou, Campbell-Thompson and Barley2 The effects on the 2020 university entry cohort could arguably have been more significant still, with new students deprived of on-campus teaching and the opportunity to network with peers.
Risk factors for loneliness before and during the pandemic appear unchanged, Reference Bu, Steptoe and Fancourt1 although young adults aged 18–30 reported increased levels of loneliness. Reference Bu, Steptoe and Fancourt1,Reference Pieh, Budimir and Delgadillo3 This risk is exacerbated for health professional students moving into clinical placements that may be remote to family support networks. Measurement of mental wellbeing is vital for this group, yet gathering this essential data from students at risk of depression and apathy is challenging. Recent evidence Reference Papadatou-Pastou, Campbell-Thompson and Barley2 suggested that data collection of this type should align with respondent culture, behaviours and values, particularly when loss of control over situations such as social distancing can engender non-participation.
A 2018 Ofcom report 4 reported how UK smartphone owners check their phones on average every 12 min of the waking day, and young people spend more time interacting through social media than face to face. In a UK survey of device usage during the COVID-19 pandemic, 46% of participants (n = 1012) identified that they had increased their smartphone usage during COVID-19 restrictions when electronic devices formed the mainstay of communication with peers, family and friends. 5 In this context, therefore, it was hypothesised that harvesting student wellbeing data via a short message service (SMS) ‘text’ message format would increase engagement and rapid response rates, compared to more in-depth traditional survey methods.
Emoji are visual characters commonly used in online communication to represent feelings, provide emotional tone and replace non-verbal cues. Reference Boutet, LeBlanc and Chamberland6 The use of emoji is so widespread that in 2015, the emoji ‘face with tears of joy’ was named the Oxford Dictionary ‘word’ of the year, 7 and it is estimated that around half of all messages shared on digital platforms now include an emoji. While emoji cannot totally replace textual meaning, they are often used as a snapshot of emotions, and emoji have been used successfully to convey opinion in a Likert scale format in a number of studies Reference Herring, Craven and Mughal8,Reference Bosch and Revilla9 evaluating survey responses.
This pilot project tested the feasibility and validity of gathering anonymous data from Therapeutic Radiography students concerning their mental wellbeing on clinical placement via free text emoji and SMS. At the author’s institution, link meetings are conducted between a member of university staff and groups of students on placement, usually once per clinical block of 5–6 weeks. Wellbeing data may indicate the frequency and timing with which the meetings may be beneficial.
Methods
Participants
All first-year pre-registration undergraduate radiotherapy students from the 2020 intake (n = 42) were emailed information and an invitation to participate in this predominantly quantitative survey-based study. Ethical approval was granted by the University Human Research Ethics Committee. Participants were advised that participation was voluntary and anonymous. Upon email receipt of consent, participants were emailed a mobile phone number used only for the study. They were asked to remove any identifying images from their profile and send a ‘hello’ message via WhatsApp to the mobile phone number to signal their engagement.
Data collection
Participants were asked to provide a simple response to each WhatsApp message from the mobile phone which was monitored by an independent researcher. A daily SMS message was sent around 6 pm each evening to all participants, seeking a range of responses throughout the 3 weeks of the study as follows.
A daily SMS asking ‘Please send your daily emoji’ prompted participants to reply with a single emoji summarising their mental state throughout the day. Participants were asked in the participant information to restrict their emoji choice to those featuring a face to aid interpretation. This is important, as studies have shown considerable variability in interpretation of emoji meanings with age and social media experience. Reference Boutet, LeBlanc and Chamberland6,Reference Sun, Li, Sun and Zhu10,Reference Weib, Bille, Rodrigues and Hewig11
For validation, it was also important to identify context for the emotions expressed by the students. Additional weekly prompts of ‘Please identify three factors that have improved your mental wellbeing this week and three that have reduced it’ sought a short SMS reply.
A final SMS at the end of the 3-week study asked participants for their thoughts about available support, and to provide suggestions on how to better support their mental wellbeing while on clinical placement.
Data analysis
Longitudinal emoji data were transcribed to an Excel spreadsheet for each participant. Emoji were identified by their ID number and Unicode Common Locale Data Repository (CLDR) name and grouped according to categories assigned by Unicode.Org. 12 Table 1 illustrates the most used emoji along with their CLDR names. Collation of emoji by group and date was performed to identify trends in submitted emotional responses over time.
Table 1. Most frequent emoji responses
Additional textual data were transcribed and subjected to simple thematic analysis following a three-step process adapted from Giorgi. Reference Giorgi13 All responses were read to check understanding; a second reading assigned codes to represent common words or phrases. Codes were then collated to identify broad themes. Three independent researchers performed analysis and negotiated to establish consistency.
Validation
After data analysis, participants were asked to complete a short anonymous online survey to provide validation of emoji group assignation and interpretation. They were also asked for their feedback on the data collection method via simple open questions.
Results
Quantitative ‘Emoji’ data
In total, there were 15 participants, although nine of these entered the data collection late (range 2–8 days). One participant stopped responding after 8 days; it is not known why. Overall, there were 254 emoji responses, utilising 108 different emoji. Table 1 highlights the most frequently used, appearing eight times or more over the 3-week period, along with their CLDR names and groups. Relative prevalence of CLDR groups over each block of data collection is presented in Figure 1; it can be seen that ‘happy’ faces were used most frequently.
Figure 1. Relative frequency of emoji groups per data collection block. Key: Academic campus (5 days); weekend (2 days); clinical placement (5 days).
Qualitative data
Factors impacting on mental wellbeing
Table 2 illustrates the most common factors impacting on student mental wellbeing extracted as themes from the weekly open questions. The most common theme related to this was social interaction, reflecting its significance during periods of isolated study during COVID-19. The other strong theme related to the placement experience and feedback; positive feedback and experiences were linked to positive emoji expressions. The opportunity for holidays or time at home rated highly as positive factors, and support the pattern of increasing happy emoji at weekends and towards the christmas break in the quantitative data presented in Figure 1. Tiredness and workload issues were cited as important factors impacting negatively on wellbeing and again this also triangulates well with Figure 1 data where the general reduction in frequency of tired emoji correlates with an increase in use of happy emoji.
Table 2. Themes relating to factors affecting wellbeing
Validation of method
Participants were asked to identify the positive and negative aspects of using emoji in this project as seen in Table 3.
Table 3. Themes relating to data collection method
Participants were also presented with a summary of all the emoji submitted by the cohort along with the standard pre-defined groupings and asked if any needed re-categorising. There were some emoji assigned to both ‘negative’ and ‘tired’ groups by different participants. In addition, one of the emoji (the upside-down smiling face) was felt by four participants to represent ‘negative’ rather than ‘happy’ emotions. As part of the validation exercise, participants were asked if their response would have changed if they were not anonymous; opinion was split 50%–50%. Despite this, 90% of respondents reported that they had not wanted a response to their emoji.
Discussion
Validation of method
Despite misgivings about misinterpretation of emoji, most of the respondents used emoji which were reliably attributed. The validation of emoji groupings exercise did, however, reveal a potential misinterpretation of one of the emoji in particular. This finding matches those reported in the literature Reference Sun, Li, Sun and Zhu10,Reference Weib, Bille, Rodrigues and Hewig11 with ‘senior netizens’ attributing emoji more reliably. Misinterpretation was only associated with a handful of emoji, a finding echoed by Bacon Reference Bacon, Barlas, Dowling and Thomas14 who reported misinterpretation of emoji such as the ‘grimacing’ face. Smutny Reference Smutny, Hancsics and Sulc15 reported success with restricting users to four emoji in their study on conversion of text to emoji; this may be overly restrictive for wellbeing monitoring purposes. While our restrictions eliminated potential issues with cultural interpretation (such as the ‘thumbs up’ sign), more restrictions or guidance could perhaps be of value, particularly for older or less experienced users. Reference Sun, Li, Sun and Zhu10,Reference Weib, Bille, Rodrigues and Hewig11 This must be balanced against the risk of reducing engagement by complicating the process. It could be argued that as regards interventions, it is the extremes (happy and sad) that would be of most interest and concern.
The proportion of positive emoji compared to negative or neutral ones was reflected in the quantity of text comments which described factors improving, compared to worsening participants mental wellbeing. This indicates a good representation of overall feeling from using the emoji alone. The overview of emoji over time (Figure 1) depicts patterns of increased concern at the start and relief at the end of placement blocks which is supported by the weekly text summary where students described being
‘Excited but nervous for placement’
“having unmet expectations“
‘feeling tired from a new routine’
The tired emoji was unique in capturing the emotion succinctly with the text providing little added value; however, participants did note in the validation process that tiredness can be encompassed within a negative emoji. The increase in positive emoji frequency at the start of the weekends is supported by the descriptions of satisfaction and achievement arising from:
‘good feedback from a mentor’
‘being made to feel welcome’
‘making a difference with a patient’
‘having learned a lot’.
The range of emoji expressed over the 3-week period demonstrated the ability of the method to identify trigger points for positive and negative emotion which could enable tutors to investigate more comprehensive support. What would constitute appropriate support from the emoji alone however was not always clear. The reason for an increase in negative emotion communicated with emoji at the final weekend, for example, conflicted with the reduced number of negative textual comments. This could indicate that feelings cannot always be attributed to reasons, and the emoji enabled participants to express something that there was no evidential basis for.
Benefits of using emoji
Feedback in relation to this method of data collection was positive, and it was clear that the hypothesised benefit of a fast, easy and anonymous response mechanism was also recognised by participants. This in turn led to high engagement rates across the 3 weeks when compared to standard student evaluation surveys, which report study participation rates below 47%. Reference Trespalacios and Perkins16 These reported gains were also identified in a recent systematic review Reference Trespalacios and Perkins16 of audience response systems. Although this review related to use of instant audience feedback within a teaching environment, it did focus on the student experience of providing rapid instant response. An 89% overall response rate from student participants with daily emoji (254/285) indicates a willingness to explore feelings and support mechanisms using a novel approach, with one student describing it as:
‘a new way to express yourself especially in this generation’
And another reflecting on the experience stating:
‘I found reflecting through emoji beneficial, thanks’
Many emoji responses were received within seconds or minutes of the daily reminder being sent. Thirteen out of the initial 14 participants supplemented emoji with text descriptions in the end of week one response (93%), and 13 out of 15 in both the second and third weeks (87%), comparing highly favourably to standard questionnaire response rates.
From the perspective of academic advisors and tutors, this method could provide immediate snapshot data relating to mental wellbeing that may potentially act as a triage method for tailoring support. Longitudinal data collection could equally show useful patterns of emotional change which would be beneficial in anticipating general support needs within and between the academic years.
Limitations of using emoji
The format of the study meant that no responses were provided to student emoji or comments; this was challenging for the researchers when participants shared negative feelings. A series of negative emoji sent by individual students raised concern; however, there was no recourse within the study design to follow up on specific individuals. As 90% of participants stated that they would not want a response to their messages, it is not clear at what point it would be appropriate to intervene. This may be best discussed with the cohort at the outset. This would give them some control over how they used the emoji feedback system. It is possible, however, that an app Reference Papadatou-Pastou, Campbell-Thompson and Barley2 could be developed to provide automated signposting and alerts in response to designated emoji.
The anonymity, which seems to be a key factor influencing engagement, is also the limiting factor for responding to issues. Although participants identified some benefits to this in terms of being able to offload ‘without making a big deal out of it’ and ‘open up without feeling vulnerable’, emoji have the potential to mask issues that may need addressing, and there is potential for misinterpretation and over-simplification of issues arising from these benefits.
From a tutor perspective, the move to working from home during the pandemic has blurred the boundaries of work life and home life; it is important to recognise the potential expectation of students for instant recognition and response. Phone messaging is a casual style of communication at odds with the more formal style required of most communications polices within universities. It is important therefore to set out standards and expectations before embarking on a new mode of communication so that no one feels compromised.
Empowering expression of emotion for learners
Anonymous communication through use of emoji enabled learners to feel connected to a level of support without exposing the full extent of their emotions and feeling vulnerable. Participants identified the value of providing feedback so that the programme team could recognise and acknowledge the situation affecting them, but without engaging in formalities. Feedback indicated that the method enabled users to express feelings to someone who was ‘listening’ but not responding. This is an important first step in developing emotional resilience and was viewed in a positive way in that they felt they could express themselves without triggering mechanisms and feeling that they were overreacting.
Conclusions
Overall, the findings from this pilot study have indicated that the use of emoji for rapid assessment of cohort mental wellbeing is valid and potentially useful when used alongside more formal evaluation mechanisms and individual support strategies. Capturing a simple wellbeing response from a full cohort may prevent knee-jerk reactions to the comments of a select number of more vocal individuals often dominating other methods.
Capturing longitudinal data would help to identify key points where mental wellbeing can be a challenge at cohort level to enable pro-active rather than reactive support. In addition to data collection, this method encourages students to identify that they have issues without feeling that they have to engage with formal support mechanisms or worry about impacting on their relationships with tutors. It is important however to reinforce that awareness and management of mental health are not recorded on a student’s file or normally disclosed.
Future work could address limitations by developing an app which can identify warning signs or non-compliance with reporting and then signpost students to support structures both human and self-directed. Additional work should also investigate the potential for reducing the number of emoji options to reduce the risk of misinterpretation.
Open access
