The impact of 3D stereoscopic visualisation on performance in electron skin apposition techniques using VERT

Abstract

Introduction:

The Virtual Environment for Radiotherapy Training (VERT) is a simulator used to train radiotherapy students cost-effectively with limited risk. VERT is available as a two-dimensional (2D) and a more costly three-dimensional (3D) stereoscopic resource. This study aimed to identify the specific benefits afforded by stereoscopic visualisation for student training in skin apposition techniques.

Method:

Eight participants completed six electron skin apposition setups in both 2D and 3D views of VERT using a 7 cm × 10 cm rectangular applicator setup to 100 cm focus skin distance (FSD). The standard deviation (SD) of the mean distance from each corner of the applicator to the virtual patient’s skin surface [which we define as apposition precision (AP)] was measured along with the time taken to achieve each setup. Participants then completed a four-question Likert-style questionnaire concerning their preferences and perceptions of the 2D and 3D views.

Results:

There was little difference in mean setup times with 218·43 seconds for 2D and 211·29 seconds for 3D (3·3% difference). There was a similarly small difference in AP with a mean SD of 5·61 mm for 2D and 5·79 mm for 3D (3·2% difference) between views. The questionnaire results showed no preference for the 3D view over the 2D.

Conclusion:

These findings suggest that the 2D and 3D views result in similar setup times and precision, with no user preference for the 3D view. It is recommended that the 2D version of VERT could be utilised in similar situations with a reduced logistical and financial impact.

Introduction

Therapeutic radiography students must learn both technical skills that allow the safe and efficient use of treatment equipment and a range of interpersonal skills.⁽¹⁾ Simulation is a core aspect of this training⁽²⁾ and since its introduction in 2007, the Virtual Environment for Radiotherapy Training (VERT) has facilitated this in educational facilities and clinical sites around the world.⁽³⁾ VERT offers a hybrid virtual environment both available using a two-dimensional (2D) view and three-dimensional (3D) stereoscopic visualisation using a back projector system and active stereo shutter glasses in which users can train their fine motor skills and improve their spatial awareness with reduced safety concerns and impact on busy clinical departments. Aside from the increased expense and logistical challenge, the 3D stereoscopic version has been associated with nausea in a minority of students; therefore, any advantages of the stereoscopic visualisation must be assessed to determine if it is essential to effective technical skill training.⁽⁴⁾

Since VERT’s initial development, the skin apposition technique has formed the basis for many studies due to its perceived demand for good spatial awareness and 3D visualisation using VERT.⁽⁵⁾ Indeed, Green and Appleyard determined that this technique was able to identify spatial ability in students.⁽⁶⁾ This study relied on apparently arbitrary weightings of a range of factors related to setup and also utilised learners for data collection. Participants in their study reported improved confidence but failed to determine any statistical difference between setup scores with different VERT views. This could have been attributed to lower user experience levels and variation of skills between the users, exacerbated by low participant numbers. Other available evidence is generally survey based relying on self-reported subjective measures such as confidence.⁽²⁾

Method

The lack of statistically significant data supporting the use of 3D stereoscopic visualisation led to the development of this study to compare the precision of the skin apposition and time taken for experienced VERT users performing a range of setups. The study aimed to evaluate whether 2D visualisation or 3D stereoscopic visualisation was equivalent in terms of setup times, apposition precision (AP) and preference in electron skin apposition techniques.

Ethical approval was granted by the University of Liverpool’s research ethics committee as the experiment was conducted on the University premises, undertaken by University employees, and involved human participation. The participants gave informed consent and all data were anonymised.

Aims

• To measure the precision and time taken to complete 2D and 3D simulation setups and see if these are equivalent.

• To distinguish if the precision of skin apposition remains consistent across 2D and 3D simulations.

• To evaluate if candidates have a preference for 3D simulation (questionnaire following experiment).

Null Hypothesis

The 2D view of VERT is equivalent to the 3D stereoscopic view of the simulator in terms of AP, efficiency in terms of time and preference.

Participants

Health and Care Professions Council registered therapeutic radiographers currently involved in radiotherapy education at the University of Liverpool, and partnered clinical sites in the regional area were invited to participate if they self-identified as confident in electron skin apposition technique and as expert users of VERT. This cohort was selected to eliminate issues related to inexperienced users, variable level of technical skills and to allow more repetitions due to familiarity with both the VERT system and clinical skin apposition technique. This limited the number of participants to eight due to the significant time commitment required to participate and the limited availability of the VERT facility. As nausea has previously been identified as a concern related to motion tracking, this function was not utilised, and participants were advised not to participate if they had a history of nausea.^(4,7) Participants were incentivised using departmental research support funding.

Intervention

Students in a previous VERT study took a mean time of 320·5 seconds to complete each setup.⁽⁶⁾ Although qualified radiographers would likely be faster, six setups in both 2D and 3D views were decided upon as an appropriate number of repetitions. This is because it would take a significant time to complete but would likely not dissuade participation or skew results through operator fatigue. Participants were presented with six 7 cm × 10 cm outlines on one of the patient skin renders available within VERT, each representing a different area to set up across various anatomical sites on the thorax and abdomen. The six positions were then duplicated, so there was a version of each position assigned to each view of VERT (2D and 3D). These 12 setups were randomly ordered using a random number generator programme to reduce the impact of the carryover effect due to learning on performance.⁽⁸⁾ The ‘virtual presenter’ software tool in VERT was used to present each participant with the same randomly assigned order of patients and return the simulator to the same starting position for each setup to ensure each participant started with the same setup conditions.

Data collection

Each participant attended the University VERT facility where the task was explained using standardised instructions delivered verbally from a script by the same investigator each time. They were each asked to set up the simulated patient as if it were an actual patient, controlling the gantry, couch and viewpoint controls themselves. Collisions were recorded to ensure the setups were completed in a realistic and controlled manner. The time taken for each participant to complete the setup so that the applicator was at 100 cm FSD (focus skin distance) and the field light was aligned to the rectangle on the patient was measured using a stopwatch. Timing commenced when the participant started to use a control and ended when they reported they were satisfied with the setup. In addition, participants were selected to be clinically capable so able to complete the task accurately. This is supported by participants being observed throughout by an investigator verifying 100 cm FSD being set, and the light field being aligned for each setup. The inherent VERT software function (‘accuracy tool’) was then used to determine the distance of the applicator (in mm) from the patient’s skin at all four corners of the applicator and also the standard deviation (SD) of this mean distance.⁽⁶⁾ This latter parameter was denoted as the ‘AP’ for the purpose of the rest of this work. Following practical data collection, participants were asked to complete a questionnaire that used a five-point Likert response scale concerning their preferred views (Appendix 1).⁽⁹⁾

Data analysis

Unlike data arising from conventional controlled trials where a significant difference is being sought and the null hypothesis tested, in this case the hypothesised mean measures are identical. Under these circumstances, measures such as t-tests fail to demonstrate statistical significance. This is supported in Bland and Altman plots for both the AP and time taken data (Figures 1 and 2).⁽¹⁰⁾ As the average difference in Figures 1 and 2 is close to Y = 0 and the data points are evenly distributed, this suggests there is no significant difference between the two methods investigated. For this study, the means and SDs for the timing and AP data were determined and compared for similarity. The survey data was analysed as percentages of participants to provide an overview of participants’ preferences (Table 3).

Figure 1. Bland–Altman plot of mean apposition precisions for 2D and 3D methods (excluding four setups of unpaired data).

Figure 2. Bland–Altman plot of mean time taken for 2D and 3D methods (excluding four setups of unpaired data).

Results

Overall, there were eight participants who consented to the study. As participant 7 was unable to complete all the setups, their data were removed from some of the data analysis as it is partially unpaired. Setup six of participant 3 was also removed as it was a major outlier when compared to other participant data. This was due to a large time taken to set up the 2D view which far exceeded any other setup time; it was attributed to fatigue from using the simulator as a short break was given after this setup and further results were not anomalous.

Apposition precision

Table 1 and Figure 3 show participants’ AP for each setup in both views. The mean SD for the distances from the corners of the applicator to the patient surface was used to determine this for each setup. The smaller the SD, the more precise the skin apposition, as this represents how equidistant the applicator corners are from the patient’s surface. Most of the SDs for each patient case were similar between 2D and 3D setups with 2D having less deviation than 3D in three of the six setups.

Table 1. Apposition precision (AP) for each setup in 2D and 3D views

Figure 3. Mean apposition precision for each setup in 2D and 3D views.

Time taken to complete setup

Figure 4 and Table 2 summarise the mean times taken to complete each setup for all participants.

Figure 4. Mean time taken to setup each patient case in both 2D and 3D.

Table 2. Mean time to complete each setup in 2D and 3D views

Table 3. Questionnaire responses

Questionnaire

Table 3 shows the result of a questionnaire (Appendix 1) given to each participant after completing their setups. They completed this away from observers to reduce the likelihood of bias.

Discussion

Apposition precision

The figures and tables presented illustrate how different patient setups resulted in different times and precision. For example, setup four was achieved a better AP with means of 3·44 mm for the 2D view and 2·29 mm for the 3D, compared to other setups such as setup two which had a 3D mean of 9·6 mm. This suggests that the experiment tested a sufficient variety of setups to facilitate any benefit from either view to be visible.

The minimal difference in AP between 2D and 3D (5·61 mm for 2D and 5·79 mm for 3D) suggests that 3D stereoscopic visualisation does not improve the precision of skin apposition setup. There is only a 3·2% percentage change between the means which are both within one SD of each other, although, overall there was slightly more variance in the 3D view setup and along with the increased mean, which could suggest that some participants found the 3D view more challenging to use for setup. This could be due to factors such as calibration of the 3D view or differences in perception of stereoscopic views resulting in most participants being marginally disadvantaged by this view. With this small decrease in AP, it is questionable whether the depth information offered by this 3D view is being utilised to give any advantage. This could also be due to the participants’ abilities in setting up real patients, causing them to use that experience over the simulator’s depiction of the relative positioning of objects. Therefore, students could be included in a further investigation to determine if they might have an increased reliance on the additional information available within the 3D view.

Time taken for setup

Figure 4 and Table 2 show a close similarity between time taken to complete setup using both 2D (218·43 seconds) and 3D (211·29 seconds). The percentage change for the mean was −3·28% which shows that the 3D view setups were faster by a very small margin, far below statistical significance. However, half of the setups were faster in each view with one, three and six being faster in 2D and two, four and five being faster in 3D supporting that the views are equivalent. Some participants clearly appeared to benefit from a particular view. While this could be due to chance, it could also be related to a factor specific to participant 6, such as their vision. Following up this experiment with an eye examination of participants could yield important information as this has previously been successful in VERT studies with students.⁽⁴⁾

As the participants decided when the setup was completed themselves, there is the possibility that the amount of time they dedicated to each setup impacted on AP, so it is important to consider the time and accuracy data together. Figure 5 illustrates this combined data and shows how each participant dedicated a similar amount of time to setting up in each view. There does not appear to be any correlation between AP and the time taken as for setups as shown in Figure 6. This suggests that the time dedicated to each setup did not influence the AP; therefore, any difference is likely to be a result of changing between 2D and 3D views.

Figure 5. Individual mean times and apposition precision for each participant.

Figure 6. Apposition precision and time taken for each individual setup for all paired data.

Participant feedback

Table 3 shows how most of the participants did not express preference for the 3D view of VERT, with 88% of responses being neutral to or in disagreement with the question. Most participants did not feel their setups were more accurate with the 3D view of VERT; this triangulates with the data which showed marginal improvements with the 2D view. AP is a contributing factor to accuracy as to have a perfect setup they would have the minimum deviation in standoff as well as setting 100 cm FSD and aligning the light field. This suggests that most participants can perceive their own AP within the simulator without the additional depth information provided by stereoscopy.

As our participants were all involved in education using VERT, it was important that the questionnaire also related to potential teaching in VERT. Table 3 indicates that most participants responded that they would not prefer to use the 3D view of VERT to teach with 62·5% responding from neutral to disagreement. There is, however, a discrepancy with 37% of participants who would rather teach using 3D and only 12% preferring to use it for their own setup. This could be related to other properties of the 3D simulation, such as the possibility of increased engagement of students. Following up this questionnaire with a survey designed to collect qualitative data with open answers or a focus group for participants could yield meaningful insights into the perceptions of those teaching using VERT and why they might prefer a particular view for teaching purposes.

This study aimed to identify if the use of existing 2D projections systems would achieve the same learning as the current stereoscopic visualisation format of VERT, which could be used to guide the expansion of the use of the 2D version of VERT. Previous work has suggested that 3D visualisation would be important for learning these clinical setup skills.⁽⁴⁾ The results identified a minimal difference in impact of the two VERT display options on both precision of skin apposition and the time taken to complete setups. There are some major limitations inherent in the stereoscopic version as it limits student access to the software to a single facility. If 2D and 3D views are comparable, and the same learning could be achieved in a 2D format, this could allow students to practice techniques in parallel seminar rooms or even in their own homes, enabling more flexible learning. Therefore, more investigation is required to determine the best way to make VERT more accessible and determine the viability of using desktop systems and other 2D platforms.

Another major logistical limitation of the 3D stereoscopic version of VERT arises from the limit of one student being able to operate it at once, with the possibility to get two students involved if one controls the view and the other uses the linear accelerator pendant. Previous investigation reported that 40% of students wanted more time on VERT individually.⁽¹¹⁾ With less specialist equipment such as 3D projectors that would be uneconomical to use for other purposes and no requirement for a specific room, the 2D version of VERT could decrease the cost and therefore increase the availability of VERT for cohort-wide learning.⁽¹¹⁾ This could provide students with more opportunities to develop their skills and help address capacity issues in therapeutic radiography workforce training.

Limitations

The sample in this study consisted of eight participants completing 12 setups each, with one participant only managing eight. This is a limited representation of those who use VERT to teach, although a considerable number of setups were possible due to their advanced skills improving reproducibility and reliability. The selected participants had prior experience both with VERT and completing skin apposition electron setups. They may, therefore, have had reduced reliance on the depth information offered by the 3D stereoscopic view of VERT and instead participants may have been able to draw on prior knowledge and high-level skills to set each patient up regardless of the view. As the impact of learning to complete setups was reduced for this experiment, further study is required to assess the impact of wider availability of VERT in a 2D format on students learning through simulation.

Conclusion

The 2D view of VERT appears to be equivalent to the 3D stereoscopic view in terms of precision of skin apposition setup, time efficiency and preference. Therefore, the null hypothesis cannot be rejected. These findings suggest there could be an increased application for the 2D view of VERT in clinical skills training, allowing more students greater access to technical skills training away from clinical placements which could improve development of interpersonal skills during placements.⁽¹⁾ Using existing infrastructure to enable widened access to the 2D view of VERT should be considered to provide more accessible technical skills training efficiently and cost-effectively to therapeutic radiography cohorts. Further research is needed to explore the implementation of increased application of the 2D view of VERT and whether it is best suited to groups of students using larger screens or individual student use through desktop computers.