Towards capsule endoscope locomotion in large volumes: design, fuzzy modeling, and testing

Abstract

We present the design and deployment of a capsule endoscope via external electromagnets for locomotion in large volumes alongside its digital twin implementation based on interval type-2 fuzzy logic systems (IT2-FLSs). To perform locomotion, we developed an external mechanism comprising five external electromagnets on a two-dimensional translational platform that is to be placed underneath the patients’ bed and integrated multiple Neodymium magnets into the capsule. The interaction between the central bottom external electromagnet and the internal magnet forms a fixed body frame at the capsule center, allowing rotation. The interaction between the external electromagnets and the two internal magnets results in rotation. The elevation of the capsule is accomplished due to the interaction between the upper external electromagnet and the internal magnets. Through simulations, we model the capsule rotation as a function of torque and drive voltages. We validated the proposed locomotion approach experimentally and observed that the results are highly nonlinear and uncertain. Thus, we define a regression problem in which IT2-FLSs, capable of representing nonlinearity and uncertainty, are learned. To verify the proposed locomotion approach and test the IT2-FLS, we leverage our experimental effort to a stomach phantom and finally to an ex vivo bovine stomach. The experimental results validate the locomotion capability and show that the IT2-FLS can capture uncertainties while resulting in satisfactory prediction performance. To showcase the benefit in a clinical scenario, we present a digital twin implementation of the proposed approach in a virtual environment that can link physical and virtual worlds in real time.

1. Introduction

Wireless capsule endoscopy (WCE) was developed in an effort to offer a patient-friendly experience by eliminating wire connections between the imaging unit within the body and the external environment. WCE is capable of providing images throughout the entire gastrointestinal (GI) tract, including the small intestine, which is difficult to reach via conventional endoscopy or colonoscopy tools. Moreover, the need for sedatives is eliminated in WCE [1, 2]. However, the design and development of a WCE that is capable of moving actively inside the GI tract is still a challenging task due to the locomotion problem within this unstructured environment of a stomach [3].

WCE development on the product level has focused on the imaging aspect through the use of one or multiple pixelated photodetector array (CMOS or CCD) sensor arrays along with readout and drive electronics. The acquired images are transferred to the external receiver via on-capsule antennas [4–6]. Yet, the inability of WCE to stop or rotate at the desired location for diagnostic purposes in current devices has led to a growing interest in adding a locomotion feature [3]. Different internal (capsule-embedded) locomotion approaches include inchworm-like crawlers integrated into capsules [7, 8], capsules having paddle/leg-based locomotion [9–11], capsules employing a sliding clamper [12], and swimming (i.e., hydrodynamic force-based) capsules [13, 14]. Along with locomotion, localization capability is also of great importance to monitor the spatial coordinates of the capsule [15]. Localization has been achieved based on time and direction of arrival [16], radiofrequency identification [17], and received signal strength indicator [18], through measurement of the magnetic fields of the capsule using magnetic sensors [19, 20] and video localization [21].

Table I. A categorical classification of external capsule control strategies.

Various locomotion approaches have focused on external mechanisms to control capsule movement to reduce the dedicated power consumption for locomotion within the capsule. In Table I, we summarized the research trends on external capsule control mechanisms in four Categories (Cs), which are defined as follows:

• C1: Magnet on a robotic arm

• C2: Handheld units comprising a permanent magnet

• C3: Closed-bore coil arrays placed around the patient/phantom

• C4: Open-bore coil arrays placed below and/or above the patient/phantom

The ones with robotic arm-based controllers (C1) offer a 6 Degree Of Freedom (DOF) with better control regarding accuracy, precision, and stability when compared to manually operated handheld units. Yet, the main disadvantage of robotic arms is the fact that they may potentially pose a danger to the patient unless strict safety measures are taken. The magnet-equipped manually operated handheld units (C2) offer a cost-effective control mechanism of the capsule, yet with a limited control capability. On the other hand, closed-bore control systems (C3) comprising multiple coils typically are in the form of an imaging unit with a bore (i.e., a computer tomography device or a magnetic resonance imager) and offer a high degree of freedom (5- or 6-DOF) at the expense of increased cost and system complexity. The proposed control hardware is an open-bore coil/electromagnet-based control system (C4), which stands in between the manually operated handheld units (C1) and closed-bored arrays (C3) employing a high number of external coils (>8), in terms of system complexity and cost. It is a patient-friendly experience over closed-bore control mechanisms with multiple coils, specifically for claustrophobic patients.

In Table I, we also listed the corresponding Region Of Interest (ROI) of the presented categories. As it can be seen, the robotic arm-based locomotion systems (C1) have the largest ROI value, as a consequence of the fact that the ROI is mainly determined by the reach of the robotic arm. The handheld systems are mainly operator-limited in terms of ROI. Complex coil-based external locomotion systems (C3) have a small three-dimensional ROI of (50–70 mm³) as the desired magnetic field is achieved for a limited volume. While the open-bore external magnet control systems (C4) also offer a small ROI area of ∼(15–20 mm²), our proposed system has a lateral extent that is determined by the linear tray working distance of 200 mm × 200 mm, which is advantageous in comparison to those presented within C3 and C4 devices.

In this study, we propose a new locomotion scheme comprising a capsule with multiple embedded miniature Neodymium magnets, two placed in dumbbell form and one at the capsule center, as illustrated in Fig. 1. The capsule locomotion is accomplished with five external electromagnets on a two-dimensional translational platform (for translation) that is to be placed underneath the patients’ bed to pin (via the central electromagnet) and rotate (via quadrant electromagnets placed at the periphery) the capsule within the environment. An additional electromagnet is utilized above the capsule endoscope (above the patient) to adjust the elevation of the capsule. Thus, the proposed control hardware is categorized as an open-bore coil/electromagnet-based control system (C4), which stands in between the manually operated handheld units and control mechanisms employing more than eight external coils in terms of system complexity and cost. Alongside the properties summarized in Table I, it is a patient-friendly experience over closed-bore control mechanisms with multiple coils, specifically for claustrophobic patients. In comparison to other open-bore capsule control systems, the proposed system is unique in distributing its 4 DOF capability between orthogonally placed translation trays (to be placed underneath the patient’s bed) and an electromagnet array (four electromagnets + one permanent magnet at the center) on the translation tray and an additional electromagnet to be placed above the patient. This way, the number of electromagnets (and thus the system complexity) is further reduced.

Figure 1. Illustration of the proposed locomotion scheme for a WCE in large volumes (i.e., the stomach).

We conducted a series of experiments to analyze and represent how the fabricated capsule is rotating and elevating via the proposed locomotion scheme. In this context, we first conducted a finite element analysis (FEA) of simulated electromagnetic fields and torque on the capsule to extract a linear relationship. To validate the model, we conducted proof-of-concept experiments on a board with a 15 cm × 15 cm area. We observed that the variations of the elevation and rotation angles of the capsule as a function of applied coil voltage are nonlinear and uncertain, thus concluding that a linear representation is not sufficient. To define a realistic model, we presented a data-driven regression modeling approach based on Interval Type-2 (IT2) Fuzzy Logic Systems (FLSs), which can represent nonlinearity and uncertainty. We illustrate that the data-driven models based on IT2-FLSs can successfully envelop the uncertainty while capturing the characteristics of experimental datasets. To showcase the use and benefit in a clinical scenario, we also present a digital twin implementation of the proposed locomotion scheme via the learned IT2-FLSs in a virtual stomach environment that can link physical and virtual worlds in real time. In this context, we constructed a Virtual Reality (VR) environment in which the aim is to control the capsule to obtain a panoramic image of the stomach walls for diagnostic purposes. Finally, we validate the proposed locomotion approach on 3D-printed stomach phantom and ex vivo bovine stomach tissue. The uncertainty quantification and point-wise accuracy performances of the trained IT2-FLSs are also presented and analyzed.

The main contributions of the study can be summarized as follows:

• We present the design and development of a new external locomotion scheme for large volumes (i.e., the stomach), where the capsule spends a very short time and the control is difficult to establish when compared to a guided conduit such as the bowel.

• We present a data-driven regression modeling approach based on IT2-FLSs to represent the nonlinear and uncertain elevation and rotation dynamics of the capsule. The IT2-FLSs not only result in a high point-wise accuracy performance but also capable of enveloping the uncertainty successfully, which is needed in developing a robust control scheme.

• We created a digital twin of the proposed locomotion scheme using the learned IT2-FLSs in a virtual stomach environment to demonstrate its usefulness in a clinical setting.

• We demonstrated that the fabricated capsule endoscope is capable of rotating, elevating, and translating through simulations and proof-of-concept, 3D-printed phantom, and ex vivo tissue experiments. The performance can be observed via the video file provided as Supplementary Material.

The outline of the manuscript is as follows: In Section 2, we present the proposed WCE locomotion scheme by defining its hardware and electromagnet driving scheme. Section 3 focuses on simulations of the rotation angle of the capsule as a function of applied electromagnet voltage and the torque on the capsule via FEA. In Section 4, we define and solve an interval regression problem in which IT2-FLSs are learned to capture not only the uncertainty but also to represent the nonlinear characteristics. In Section 5, we present the IT2-FLS-based digital twin implementation in a virtual stomach environment. Finally, we present the phantom and ex vivo experiments to validate in Section 6. The paper is concluded with final remarks and discussions of the outcomes of our research alongside future directions.

Figure 2. Experimental setup: (a) bottom magnet array and the capsule, (b) translation stage embedded underneath the bottom magnet array and placement of the top electromagnet, and (c) 3D-printed capsule with central and distal magnets.

2. The proposed WCE locomotion strategy

Here, we present the hardware and driving scheme of the proposed WCE locomotion strategy.

2.1. Hardware setup

In Fig. 2, we illustrate the setup with which the proposed locomotion strategy was implemented. A capsule (13 mm diameter, 25 mm length) was 3D-printed (using fused deposition modeling via Ender Creality Pro 3 Printer) with grooves for embedding three Neodymium magnets (one magnet having 6 mm diameter and 1.5 mm height at the capsule center for pinning the capsule, and two magnets placed in dumbbell fashion, having 10 mm diameter and 1.2 mm height, at 10 mm away from the center). The capsule was placed on an acrylic board with an embedded electromagnet array underneath (Flying Elephant, 40 mm diameter, 20 mm height, placed opposite each other with 7.6 cm from their center points, and Flying Elephant, 35 mm diameter, 30 mm height, placed opposite each other with 8.1 cm from their center points). Note that we used slightly different electromagnets for the bottom array and compensated for the magnetic field difference by adjusting their distance from the center using the feedback from magnetometer measurements. A central electromagnet (20 mm diameter, 15 mm height) was placed at the center of the four magnets. Finally, the largest electromagnet (8 cm diameter, 4 cm height) was placed 5 and 7 cm above the capsule to control the elevation angle. All electromagnets were driven with computer-controlled power supplies. Note that, a CMOS camera was utilized in observing the rotation and elevation angles during capsule control experiments. The placement of the camera was adjusted for each experiment to observe the angle from a perpendicular direction.

2.2. Electromagnet driving scheme

Here, we present the proposed driving scheme of the electromagnet array to rotate and elevate the capsule. The real-time performance is provided as a video file within the Supplementary Material.

To describe the capsule rotation, we present snapshots of the capsule performing rotation and the visualization of the bottom electromagnet drive scheme in Fig. 3 for clarity. As illustrated in Fig. 3b, the rotation is accomplished by driving the opposing electromagnets with out-of-phase ramp signals. Thus, we drive EM2 and EM3 for the first 90-degree rotation, EM1 and EM4 for rotating between 90 and 180°, EM3 and EM2 for rotation of the capsule from 180 to 270°, and finally EM4 and EM1 to complete the full cycle of rotation as depicted in Fig. 3c.

Figure 3. Snapshots (at every 45°) of the capsule rotation experiment and the magnet drive scheme: (a) rotation snapshots, (b) drive scheme for rotation, and (c) voltage change-flow of electromagnets.

To describe the capsule elevation, Fig. 4a lays out snapshots of the capsule performing elevation while Fig. 4b and c present the electromagnet arrangement and driving scheme, respectively. The capsule elevation is accomplished through a ramp driving the upper electromagnet up or down, while only the central bottom electromagnet is active (at 8 V fixed voltage), as depicted in Fig. 4c.

Figure 4. Snapshots of the capsule elevation experiment and the magnet drive scheme: (a) elevation snapshots, (b) magnet arrangement, and (c) drive scheme for elevation.

3. Finite element simulations: capsule – electromagnet array interaction

Figure 5 illustrates the FEA simulated electromagnetic fields and torque on the capsule (in the COMSOL software) for the process of 90° rotation. The rotation is accomplished by driving the opposing electromagnet (EM2 and EM3) voltages with out-of-phase ramp signals from −12 to 12 V (per the experimented power supply range). By doing so, the capsule rotates from the rest position (vertical orientation) by 90° (to horizontal orientation). Without loss of generality, an identical drive scheme can be consecutively applied between EM1–EM4, EM3–EM2, and EM3–EM2 to perform a complete rotation in the counterclockwise direction. Figure 5 also shows three instances of the rotation, electromagnetic field vectors on the capsule, and the corresponding current values applied to the electromagnets. Note that the electromagnet was modeled in FEM to lead up to a force of 250 N (with 12 V of applied voltage onto the electromagnet having 15,000 windings) on an adjacent magnet, per its specification. We note that the voltage vs. rotation angle behavior of the capsule was deduced based on the rotation angle of the capsule for a given applied voltage at which torque is observed to be canceled out, which we refer to as the rest position. On the other hand, the torque was calculated based on the finite simulated torque on the initial position (capsule facing EM1 and EM2, i.e., 0°). Overall, we observe a one-to-one correspondence between the torque and the applied voltage (on EM2 and EM3) and an approximately linear relationship between the voltage $v_{\text{EM}}$ and the observed torque rotational angle ( $\psi$ ), such that $\psi = k_{\psi }$ . $v_{\text{EM}}$ , where ≅ 3.75°/V.

Figure 5. Finite element simulated drive voltage increase of EM3, and decrease of EM2 and torque, vs. rotational angle behavior of the capsule. The top insets show finite element simulations of the magnetic field lines between the electromagnet array and the magnets within the capsule when the capsule is in a vertical direction, making 22.5° and making 45° with the vertical direction.

4. Data-driven modeling: capsule–electromagnet array interaction

In this subsection, we present a data-driven approach to model the capsule–electromagnet array interaction. In this context, we first performed a series of experiments that are described as follows:

• Experiment-1: demonstration of a full rotation of the capsule (whose snapshots are given in Fig. 3). The measured $\psi$ values $(\tilde{\psi })$ are presented in Fig. 7.

• Experiment-2 & 3: demonstration of upward/downward elevation of the capsule (whose snapshots are given in Fig. 4) under a low magnet array – top magnet separation of 5 cm. The measured $\theta$ values $(\tilde{\theta })$ are given in Fig. 8.

• Experiment-4 & 5: demonstration of upward/downward elevation of the capsule under a high magnet array – top magnet separation of 7 cm. The $\tilde{\theta }$ values are shown in Fig. 9.

All experiments were repeated five times.

We define a parametric regression model to map the employed voltages of the six magnets ( $v_{\text{EM}i}, i=0,..,5$ ) to the change of the rotation angle $\psi$ and the elevation angle $\theta$ of the capsule. Inspired by the quadcopter control allocation matrix and (decoupled) kinematic model [33], we define the following linear relationship:

(1) \begin{equation} \left[\begin{array}{c} \Delta \psi \\[4pt] \Delta \theta \end{array}\right]=\underset{\boldsymbol{K}}{\underbrace{\left[\begin{array}{c@{\quad}c@{\quad}c@{\quad}c@{\quad}c} k_{\psi } & k_{\psi } & k_{\psi } & k_{\psi } & 0\\[4pt] 0 & 0 & 0 & 0 & k_{\theta } \end{array}\right]}}\left[\begin{array}{c} \left| \Delta v_{\textrm{EM}1}\right| \\[4pt] \left| \Delta v_{\textrm{EM}2}\right| \\[4pt] \left| \Delta v_{\textrm{EM}3}\right| \\[4pt] \left| \Delta v_{\textrm{EM}4}\right| \\[4pt] \Delta v_{\textrm{EM}5} \end{array}\right] \end{equation}

where $k_{\psi }$ and $k_{\theta }$ are coefficients to be estimated. Then, we can also define:

(2) \begin{align} \psi \left(n+1\right) & =\psi \left(n\right)+\Delta \psi \nonumber\\[4pt] \theta \left(n+1\right) & =\theta \left(n\right)+\Delta \theta \end{align}

Note that, we have not defined $\Delta v_{\textrm{EM}0}$ in (1) as EM0 has always a constant voltage. To estimate the control allocation matrix $\boldsymbol{K}$ , we conducted a series of experiments to model the functional relationships between (i) the employed voltage to the four magnets $v_{\textrm{EM}i}(i=1,..,4)$ and $\psi$ ; (ii) the employed voltage to the vertical magnets $v_{\textrm{EM}5}$ and $\theta$ .

It is observed from the experimental results presented in Figs. 7, 8, and 9 that the mappings do not represent a linear one (like the one obtained by FEA), which we attribute to the asymmetric placement of the external electromagnets, the permanent Neodymium magnets within the capsule, and also to friction. Thus, the representation given in (1) is not suitable to model the characteristics of the real-world capsule. To overcome this issue, we redefine the mappings as follows:

(3) \begin{align} \Delta \psi & =f_{\psi }\left(u_{\psi }\right)\nonumber\\ \Delta \theta & =f_{\theta }\left(u_{\theta }\right) \end{align}

where $f_{\psi }(.)$ and $f_{\theta }(.)$ represent nonlinear functions, $u_{\psi }$ and $u_{\theta }$ are defined as follows:

(4) \begin{equation}\left[\begin{array}{c} u_{\psi }\\[4pt] u_{\theta } \end{array}\right]=\left[\begin{array}{c@{\quad}c@{\quad}c@{\quad}c@{\quad}c} 1 & 1 & 1 & 1 & 0\\[4pt] 0 & 0 & 0 & 0 & 1 \end{array}\right]\left[\begin{array}{c} \left| \Delta v_{\textrm{EM}1}\right| \\[4pt] \left| \Delta v_{\textrm{EM}2}\right| \\[4pt] \left| \Delta v_{\textrm{EM}3}\right| \\[4pt] \left| \Delta v_{\textrm{EM}4}\right| \\[4pt] \Delta v_{\textrm{EM}5} \end{array}\right]\end{equation}

In this study, we prefer to define the nonlinear functions ( $f_{\psi }(.)$ and $f_{\theta }(.)$ ) with FLSs, as they are powerful tools to represent nonlinearity [34, 35]. Moreover, we observed from the presented experimental results that there is a huge amount of uncertainty, which has even heteroscedastic characteristics. Therefore, there is not only a need to learn a model with high point-wise accuracy but also to envelope the uncertainty to define a Prediction Interval (PI), which is needed in developing a robust control scheme. Therefore, we represent the functional relationships via IT2-FLSs, which are proven to be capable of modeling uncertain data [36]. Thus, we generalize the mapping presented in (3) as follows:

(5) \begin{align} \left[\Delta \psi,\Delta \underline{\psi },\Delta \overline{\psi }\right] & =f_{\psi }\left(u_{\psi }\right)\nonumber\\[4pt] \left[\Delta \theta,\Delta \underline{\theta },\Delta \overline{\theta }\right] & =f_{\theta }\left(u_{\theta }\right) \end{align}

with $\Delta \psi \in [\Delta \underline{\psi },\Delta \overline{\psi }]$ and $\Delta \theta \in [\Delta \underline{\theta },\Delta \overline{\theta }]$ . In the latter parts, we first present brief information about the internal structure of the IT2-FLS $(f(.))$ alongside its learning approach and then the resulting identification results.

4.1. Learning strategy of IT2-FLSs

Let us first define the rule structure of an IT2-FLS with $P$ rules $(p=1,\ldots,P)$ for an input vector $\boldsymbol{x}=(x_{1}, x_{2},\ldots, x_{M})^{T}$ as follows:

(6) \begin{equation} \text{Rule}_{p}\colon \text{If } x_{1}\text{ is } \tilde{A}_{p,1}\text{ and}\ldots\, x_{M}\text{ is } \tilde{A}_{p,M}\text{ Then }y \text{ is }y_{p}=\sum _{m=1}^{M}a_{p,m}x_{m}+a_{p,0} \end{equation}

Here, $y_{p}$ define the consequent Membership Functions (MFs) defined with first-order polynomials. The antecedent MFs $(\tilde{A}_{p,m})$ are described with Gaussian IT2 fuzzy sets defined in terms of upper MF (UMF) $\overline{\mu }_{{\tilde{A}_{p,m}}}(\boldsymbol{x})$ and lower MF (LMF) $\underline{\mu }_{{\tilde{A}_{p,m}}}(\boldsymbol{x})$ as shown in Fig. 6. For an input $x_{m}$ , the UMF is defined as:

(7) \begin{equation} \overline{\mu }_{{\tilde{A}_{p,m}}}\left(x_{m}\right)=\exp \left(-\left(x_{m}-c_{p,m}\right)^{2}/2\sigma _{p,m}^{2}\right) \end{equation}

while the LMF is as follows:

(8) \begin{equation} \underline{\mu }_{{\tilde{A}_{p,m}}}\left(x_{m}\right)=h_{p,m}\exp \left(-\left(x_{m}-c_{p,m}\right)^{2}/2\sigma _{p,m}^{2}\right) \end{equation}

where $c_{p,m}$ are the centers and $\sigma _{p,m}$ are the standard deviations of LMFs and UMFs while $h_{p,m}$ are the heights of LMFs. The IT2-FLS uses the product implication and the Nie-Tan center of sets defuzzification method.

Figure 6. Illustration of an antecedent IT2-FS.

The output of the IT2-FLS ( $y$ ) for a given input vector $\boldsymbol{x}$ is defined as follows:

(9) \begin{equation} y\left(\boldsymbol{x}\right)=\frac{\sum _{p=1}^{P}\left[\underline{w}_{p}\left(\boldsymbol{x}\right)y_{p}+\overline{w}_{p}\left(\boldsymbol{x}\right)y_{p}\right]}{\sum _{p=1}^{P}\underline{w}_{p}\left(\boldsymbol{x}\right)+\sum _{p=1}^{P}\overline{w}_{p}\left(\boldsymbol{x}\right)} \end{equation}

The corresponding upper and lower bounds of (9) are defined as follows [35]:

(10) \begin{equation}\underline{y}(\boldsymbol{x})=\frac{2\sum _{p=1}^{P}\underline{w}_{p}\left(\boldsymbol{x}\right)\underline{y}_{p}}{\sum _{p=1}^{P}\underline{w}_{p}\left(\boldsymbol{x}\right)+\sum _{p=1}^{P}\overline{w}_{p}(\boldsymbol{x})}\end{equation}

(11) \begin{equation}\overline{y}(\boldsymbol{x})=\frac{2\sum _{p=1}^{P}\overline{w}_{p}(\boldsymbol{x})\overline{y}_{p}}{\sum _{p=1}^{P}\underline{w}_{p}\left(\boldsymbol{x}\right)+\sum _{p=1}^{P}\overline{w}_{p}(\boldsymbol{x})}\end{equation}

where $\underline{w}_{p}(\boldsymbol{x})$ and $\overline{w}_{p}(\boldsymbol{x})$ are the lower and upper rule firing functions of the $p\text{th}$ rule, respectively, which are defined as follows:

(12) \begin{equation}\underline{w}_{p}\left(\boldsymbol{x}\right)=\underline{\mu }_{{\tilde{A}_{p,1}}}\left(x_{1}\right)\cap \ldots \cap \underline{\mu }_{{\tilde{A}_{p,M}}}\left(x_{M}\right)\end{equation}

(13) \begin{equation}\overline{w}_{p}(\boldsymbol{x})=\overline{\mu }_{{\tilde{A}_{p,1}}}\left(x_{1}\right)\cap \ldots \cap \overline{\mu }_{{\tilde{A}_{p,M}}}(x_{M})\end{equation}

Here $\cap$ denotes the t-norm operator, which is defined with the algebraic product operator.

As we aim to learn an IT2-FLS that has a high accuracy performance and is capable of enveloping uncertainty, we define the following minimization problem as presented in [35] for N samples $\{\boldsymbol{x}_{n},y_{n}\}_{n=1}^{N}$ with $\boldsymbol{x}_{n}=(x_{n,1},\ldots,x_{n,M})^{T}$

(14)

Here defines the learnable parameter set where is the constraint set. $L_{\log -\cosh }$ is the empirical risk function for accuracy purposes and is defined as follows:

(15) \begin{equation} L_{\log -\cosh }=\frac{1}{N}\sum _{n=1}^{N}\log \left(\cosh \left(y_{n}-y\left(\boldsymbol{x}_{n}\right).\right)\right) \end{equation}

where $\epsilon _{n}=y_{n}-y(\boldsymbol{x}_{n})$ . In (15), is the tilted loss function which is used to generate an envelope to cover the expected amount of uncertainty $\varphi =[\underline{\tau },\overline{\tau }]$ and is defined as:

(16)

where

(17)

(18)

The learnable parameter set for the rule antecedents $\boldsymbol{\Omega }_{\textbf{A}}$ is $\boldsymbol{\Omega }_{\textbf{A}}=\{\boldsymbol{c},\boldsymbol{\sigma },\boldsymbol{h}\}$ where $\boldsymbol{c}=(c_{1,1},\ldots,c_{P,M})^{T}\in \mathbb{R}^{P\times M}, \boldsymbol{\sigma }=(\sigma _{1,1},\ldots,\sigma _{P,M})^{T}\in \mathbb{R}^{P\times M} (\sigma _{p,m}=\underline{\sigma }_{p,m}=\overline{\sigma }_{p,m})$ and $\boldsymbol{h}=(h_{1,1},\ldots,h_{P,M})^{T}\in \mathbb{R}^{P\times M}$ . The learnable parameter set for the rule consequents is $\boldsymbol{\Omega }_{\boldsymbol{C}}=\{\boldsymbol{a}, \boldsymbol{a}_{\textbf{0}}\}$ with $\boldsymbol{a}=(a_{1,1},\ldots,a_{P,M})^{T}\in \mathbb{R}^{P\times \textrm{M}}$ and ${\boldsymbol{a}_{\textbf{0}}}=(a_{1,0},\ldots, a_{P,0})^{T}\in \mathbb{R}^{P\times 1}$ . The learnable parameter set of the IT2-FLS is then $\boldsymbol{\Omega }=\{\boldsymbol{\Omega }_{\textbf{\textit{A}}},\boldsymbol{\Omega }_{\boldsymbol{C}}\}$ . In this paper, we have deployed the proposed deep learning-based training approach presented in [35].

4.2. Modeling performance of the IT2-FLSs

We employed the modeling in the Matlab® computing environment and evaluated the accuracy performance via Root MSE (RMSE), while the uncertainty covering performances via PI coverage probability (PICP) were measured. We set the expected coverage as 90% ( $\varphi =[0.05,0.95]$ ) and the total number of rules as $P=5$ . The number of epochs is set to 140, while the learning rate is found by performing a grid search. The resulting performance measures are tabulated in Table II. It can be concluded the performances of IT2-FLSs are highly satisfactory as the RMSE are low and the PICP values are close to the expected coverage value (90%) for all conducted experiments.

Table II. Modeling performance of the IT2-FLSs: training experiments.

In Fig. 7, we present the point-wise accuracy performance of IT2-FLS ( $\psi$ ) alongside its uncertainty representation performance ( $\underline{\psi },\overline{\psi }]$ ) and the experimental $\psi$ values $(\tilde{\psi })$ . It can be observed that the IT2-FLS is capable of capturing not only the nonlinear rotation characteristics but also the uncertainty. Moreover, when compared to the finite element simulation presented in Fig. 5, it can be observed that the variation in the real-world dataset $[u_{\psi }, \tilde{\psi }]$ has a similar characteristic, yet, especially at high magnitude $\tilde{\psi }$ values, there is a strong nonlinearity alongside uncertainty. However, the mapping of the IT2-FLS $(f_{\psi }(u_{\psi }))$ was capable of covering the uncertainty while resulting in an overall good prediction performance, as presented in Table II.

Figure 7. Training experiments: rotation angle as a function of the cumulative voltage applied to the bottom electromagnetic array. Here, the blue circles represent the experimental values $\tilde{\psi }$ , the red line represents the point-wise prediction $\psi$ , and the shaded area is the covered uncertainty $[\underline{\psi },\overline{\psi }]$ by IT2-FLS.

Figures 8 and 9 illustrate the change in the experimental elevation angle $(\tilde{\theta })$ in upward and downward directions for an electromagnet array with top electromagnet distances of 5 cm and 7 cm, respectively. We observe:

• that with the close placement of the bottom magnets and the top magnet (Fig. 8), the capsule can be controlled until it is nearly fully elevated. As the distance is increased to 7 cm, the magnetic force on the capsule becomes inadequate to fully erect it.

• a strong hysteresis when the distance between the bottom electromagnets and the top magnet is 5 cm, as the erection of the capsule causes a strong interaction between the upper magnet within the capsule and the top electromagnet. Thus, decreasing the elevation of the capsule once it is erected requires the applied voltage (to EM0) to decrease down to ∼3 V level.

• when the distance between the bottom electromagnets and the top magnet is 7 cm, as the capsule does not fully erect, the force between the top magnet within the capsule and the top electromagnet is weaker. Therefore, decreasing the elevation angle does not require the applied voltage (to EM0) to be at 3 V level.

Figure 8. Training experiments: elevation (left) and delevation (right) characteristics of the capsule with an inter-magnet distance of 5 cm. The blue circles represent the experimental values $\tilde{\theta }$ , the red line represents the point-wise prediction $\theta$ , and the shaded area is the covered uncertainty $[\underline{\theta },\overline{\theta }]$ by IT2-FLS.

Figure 9. Training experiments: elevation (left) and delevation (right) characteristics of the capsule with an inter-magnet distance of 7 cm. The blue circles represent the experimental values $\tilde{\theta }$ , the red line represents the point-wise prediction ${\theta}$ , and the shaded area is the covered uncertainty $[\underline{\theta },\overline{\theta }]$ by trained IT2-FLS.

Besides the fact that there is a dominant nonlinearity between $u_{\theta }$ and $\tilde{\theta }$ , we also note that there is a huge amount of uncertainty. Nevertheless, as shown in Figs. 8 and 9, the learned IT2-FLSs were capable of representing the nonlinear nature of the relationship while also generating envelopes that cover real-world uncertainty. This also coincides with the performance measures presented in Table II since for all experiments, the IT2-FLSs were capable of resulting in relatively low RMSE values while the PICP values are almost 90%.

In conclusion, the analysis of the IT2-FLS performance outcomes indicates that the representation of both rotation and elevation-delevation dynamics has been effectively accomplished, as evidenced by the observed low RMSE. Furthermore, the generated PIs of the IT2-FLS consistently achieved the targeted coverage value of 90% across all experimental trials.

5. Digital twin representation with VR visualization

In this section, we present the digital twin implementation to illustrate the locomotion of the capsule via the learned IT2-FLSs within a virtual environment that can link physical and virtual worlds in real time. We developed a VR environment within Unity to define a large-volume environment, namely the stomach, and deployed the learned IT2-FLSs to represent the orientation kinematics of the fabricated capsule (presented in a generic form in (5)) to define the digital twin.

In this paper, we handled a case study in which the aim was to observe a panoramic image of the stomach walls. In this context, we defined two main object models to represent the stomach and capsule models within the Unity environment. The applied voltages to the fabricated capsule within the real-world environment are transferred to the virtual capsule via the IT2-FLS in a synchronized fashion. Figure 10 illustrates the view of the stomach observed by the virtual capsule, which processes the real-world input voltages. See Supplementary Material for the conducted proof-of-concept experiments alongside its VR visualizations. It can be observed that the performance is satisfactory, although the dynamics of the capsule have not been taken into account, and thus there are slight variations between the characteristics of the digital twin and the fabricated capsule.

Figure 10. Visualization of the capsule control in the VR environment: (a) virtual stomach, (b) virtual capsule, and (c) virtual views of the capsule for different rotation angles in the real world.

6. Phantom and ex vivo experiments

Here, we conduct a series of rotation, elevation, and translation experiments on (i) a 3D-printed stomach phantom and (ii) an ex vivo bovine stomach tissue. See Supplementary Material for the conducted experiments. We also analyze the IT2-FLSs on the test results (for rotation and elevation experiments) acquired in this section.

For the stomach phantom experiments, we utilized a stomach CAD file, illustrated in Fig. 11a, which we then 3D-printed using polylactic acid via fused deposition modeling, as given in Fig. 11b. We conducted the following experiments five times:

• Experiment-6: demonstration of a full rotation of the capsule on the stomach phantom (whose snapshots are given in Fig. 12a and b)

• Experiment-7 & 8: demonstration of upward/downward elevation of the capsule on the stomach phantom (whose snapshots are given in Fig. 12c and d)

Figure 11. (a) CAD drawing of the stomach phantom, (b) 3D-printed PLA phantom, and (c) bovine stomach paved on the stomach phantom.

Figure 12. Snapshots from the stomach phantom experiments: (a–b) rotation and (c–d) elevation.

In Figs. 13 and 14, we present the experimental results of the stomach phantom alongside the modeling performance of the IT2-FLS. The performance measures obtained from these experiments are listed in Table III. A comparison with the results of the training experiments presented in Section 4.2 reveals a degradation in the RMSE and PICP performance measures. This degradation can be attributed to variations in experimental conditions as a consequence of the stomach phantom, such as friction, surface roughness, slope, and other factors. Nonetheless, the trained IT2-FLSs demonstrated their ability to represent the nonlinear characteristics of the relationship and generate uncertainty envelopes that encompass real-world uncertainty to a certain extent.

Figure 13. Stomach phantom experiments: rotation angle modeling performance of the IT2-FLS. Here, the blue circles represent the experimental values $\tilde{\psi }$ , the red line represents the point-wise prediction $\psi$ , and the shaded area is the covered uncertainty $[\underline{\psi },\overline{\psi }]$ by trained IT2-FLS.

Figure 14. Stomach phantom experiments: elevation (left) and delevation (right) modeling performance of the IT2-FLSs. Here, the blue circles represent the experimental values $\tilde{\theta }$ , the red line represents the point-wise prediction ${\theta}$ , and the shaded area is the covered uncertainty $[\underline{\theta },\overline{\theta }]$ by trained IT2-FLS.

Table III. Modeling performance of the IT2-FLSs: stomach phantom experiments.

In the next step, an ex vivo bovine stomach tissue (acquired from a local slaughterhouse) was paved on the 3D-printed phantom as shown in Fig. 11c. Another set of rotation and elevation-delevation experiments was conducted on the lubricated (to imitate the gastric mucus) ex vivo tissue, which represents the closest to a real-life scenario in the context of this study. As we have done in the phantom experiments, we evaluated the performance of the IT2-FLSs by conducting the following ex vivo experiments five times:

• Experiment-9: demonstration of a full rotation of the capsule on ex vivo tissue (whose snapshots are given in Fig. 15a and b).

• Experiment-10 & 11: demonstration of upward/downward elevation of the capsule on ex vivo tissue (whose snapshots are given in Fig. 15c and d).

Figure 15. Snapshots from the ex vivo bovine stomach experiments: (a–b) rotation and (c–d) elevation.

The modeling performance of the IT2-FLS for Experiment-9 and Experiment-10 & 11 are given in Figs. 16 and 17, respectively. Based on the distribution of the collected dataset from ex vivo experiments, we initially observed a reduction in the rotation performance of the proposed locomotion scheme when compared to the ones from the stomach phantom experiments. This can be attributed to the intricate topography of the stomach surface and the increased distance between the capsule and the electromagnet plane. However, it is worth noting that the elevation-delevation performance is improved due to the surface’s ability to consistently hold the capsule in place during each test cycle.

Figure 16. Ex vivo tissue experiments: rotation angle modeling performance of the IT2-FLS. Here, the blue circles represent the experimental values $\tilde{\psi }$ , the red line represents the point-wise prediction ${\psi}$ , and the shaded area is the covered uncertainty $[\underline{\psi },\overline{\psi }]$ by trained IT2-FLS.

Figure 17. Ex vivo tissue experiments: elevation (left) and delevation (right) modeling performance of the IT2-FLSs. Here, the blue circles represent the experimental values $\tilde{\theta }$ , the red line represents the point-wise prediction $\theta$ , and the shaded area is the covered uncertainty $[\underline{\theta },\overline{\theta }]$ by trained IT2-FLS.

Similar to the findings from the stomach phantom experiments, a degradation in the performance of the IT2-FLS is evident, as indicated by the calculated RMSE and PICP values presented in Table IV. It can be observed that the PICP value is less than the desired value of 90% and the width of the PI has been increased, especially when the input voltage is relatively large. Thus, the quality of the PI has relatively decreased. However, considering that the training dataset does not fully capture the intricacies of the rotation and elevation dynamics on ex vivo tissues, the performance of the IT2-FLS can be regarded as relatively satisfactory.

Table IV. Modeling performance of the IT2-FLSs: ex vivo tissue experiments.

In conclusion, the evaluation of the learned IT2-FLSs, both in the context of the stomach phantom and ex vivo experiments, has provided valuable insights into its performance. While there was a noticeable degradation in the RMSE and PICP measures compared to the training results, this can be attributed to the inherent variations introduced by factors such as friction, surface roughness, and ex vivo tissue. However, the trained IT2-FLS models demonstrated their capacity to capture the nonlinear characteristics of the relationship and generate uncertainty envelopes that partially accounted for real-world uncertainties. Despite the observed decrease in the quality of the PIs, it is worth underlining that the training dataset did not fully encompass the complexities of rotation and elevation dynamics on ex vivo tissues. In light of these limitations, the performance of the IT2-FLS can be deemed relatively satisfactory.

For the sake of the completeness of the study, we also conducted translation experiments on the ex vivo tissue. In these experiments, we moved the translation stage, on which the bottom magnet array is located, along the long axis of the stomach (for a total distance of 7 cm) and repeated the experiment five times in each direction.

• Experiment-12 & 13: demonstration of forward/backward translation of the capsule on the stomach phantom (whose snapshots are given in Fig. 18a and b).

Figure 18. Snapshots from the ex vivo bovine stomach translation experiments: (a–b) forward and (c–d) backward.

During the translation experiments, the central electromagnet; EM1, and EM4 were driven to ensure that the capsule stayed at a constant angle as it moved in the direction of the translation stage. Figure 19 showcases the acquired data that is defined via the stage position $(x_{\text{stage}})$ and capsule position $(x_{\text{caps}})$ . It can be observed that the characteristic is closer to a linear behavior and has minimal uncertainty when compared to the ones obtained from the rotation and elevation experiments.

Figure 19. Ex vivo tissue experiments: forward (left) and backward (right) translation characteristics.

Note that the difference between both directions (forward vs. backward) is attributed to the bumpy surface of the phantom and the tissue, as depicted in Fig. 20. We also note that the displacement extent of the translation experiment was limited to 7 cm and only one dimension due to the shape of the stomach. It is noteworthy to mention that the stages are capable of moving 20 cm in both lateral axes.

Figure 20. Perspectives to illustrate the bumpy surface of the phantom and the tissue characteristics.

7. Conclusion, discussion, and future work

We showcased an electromagnet array-based external control methodology to rotate and elevate a capsule endoscope in large volumes, such as the stomach. Our approach is classified under the open-bore coil/electromagnet-based capsule control systems category, which compares favorably over closed-bore systems with a high number of coils in terms of system complexity and cost. Such systems offer a patient-friendly experience, specifically for claustrophobic patients. In comparison to other open-bore capsule control systems, the proposed system is unique in distributing its 4 DOF capability between translation stages and electromagnets, reducing the overall number of employed coils. The rotation and elevation capabilities of the developed system were simulated and demonstrated on the capsule with proof-of-concept experiments. The locomotion was performed through the interaction of an array of external electromagnets and in-capsule magnets. Via FEA, we modeled the capsule rotation as a function of torque and applied drive voltage to the electromagnets. We also validate the proposed locomotion approach experimentally and observe deviations from the simulated behavior due to nonlinearity and uncertainty. Thus, we have solved an interval regression problem in which IT2-FLSs are defined as the prediction functions. We have evaluated the performance of the learned IT2-FLSs concerning point-wise accuracy and uncertainty representations. It has been shown that learned IT2-FLSs do not only result in an accuracy performance (i.e., low RMSE value) but also are capable of capturing the uncertainty (high PICP value) successfully. Moreover, to showcase the use and benefit in a clinical scenario, we presented an IT2-FLS-based digital twin implementation of the proposed scheme in a virtual stomach environment that is capable of linking physical and virtual worlds in real time. The proposed locomotion strategy was also utilized on a 3D-printed stomach phantom and an ex vivo bovine stomach. The results clearly validate the locomotion capability on both the phantom and ex vivo tissue and show that the data-driven model can capture uncertainties while resulting in high prediction performance.

In future studies, we note that the capsule should be equipped with magnetic field (i.e., Hall) sensors for the acquisition of the rotational and elevation angles with higher precision in a closed-loop fashion. Moreover, without loss of generality, the electromagnetic force needs to be scaled up for a real clinical scenario; the distance between the bottom magnet array and the top magnet is not adequate for a human to fit in. It is noteworthy to mention that electromagnetic forces can be strengthened through an increase in applied current and voltage, with a higher number of windings, or through the use of larger-sized electromagnets. Future research should also focus on expanding the training dataset to encompass a broader range of experimental conditions and factors to enhance the IT2-FLS’s ability to handle real-world uncertainties more effectively. Furthermore, refining the existing IT2-FLS approach may also contribute to improving the accuracy and reliability of the predictions. Overall, the findings contribute to the field of modeling rotation and elevation dynamics and provide insights for further advancements in this area. Besides, with further development, we plan to implement the proposed method in a clinical scenario after in vivo experiments.