We create a new tactile recording system with which we develop a three-axis fingernail-color sensor that can measure a three-dimensional force applied to fingertips by observing the change of the fingernail’s color. Since the color change is complicated, the relationships between images and three-dimensional forces were assessed using convolution neural network (CNN) models. The success of this method depends on the input data size because the CNN model learning requires big data. Thus, to efficiently obtain big data, we developed a novel measuring device, which was composed of an electronic scale and a load cell, to obtain fingernail images with 0$^\circ$ to 360$^\circ$ directional tangential force. We performed a series of evaluation experiments to obtain movies of the color changes caused by the three-axis forces and created a data set for the CNN models by transforming the movies to still images. Although we produced a generalized CNN model that can evaluate the images of any person’s fingernails, its root means square error (RMSE) exceeded both the whole and individual models, and the individual models showed the smallest RMSE. Therefore, we adopted the individual models, which precisely evaluated the tangential-force direction of the test data in an $F_x$-$F_y$ plane within around $\pm$2.5$^\circ$ error at the peak points of the applied force. Although the fingernail-color sensor possessed almost the same level of accuracy as previous sensors for normal-force tests, the present fingernail-color sensor acts as the best tangential sensor because the RMSE obtained from tangential-force tests was around 1/3 that of previous studies.

The enhanced dexterity and manipulability offered by master–slave teleoperated surgical systems have significantly improved the performance and safety of minimally invasive surgeries. However, effective manipulation of surgical robots is sometimes limited due to the mismatch between the slave and master robots’ kinematics and workspace. The purpose of this paper is first to formulate a quantifiable measure of the combined master–slave system manipulability. Next, we develop a null-space controller for the redundant master robot that employs the proposed manipulability index to enhance the performance of teleoperation tasks by matching the kinematics of the redundant master robot with the kinematics of the slave robot. The null-space controller modulates the redundant degrees of freedom of the master robot to reshape its manipulability ellipsoid (ME) towards the ME of the slave robot. The ME is the geometric interpretation of the kinematics of a robot. By reshaping the master robot’s manipulability, we match the master and slave robots’ kinematics. We demonstrate that by using a redundant master robot, we are able to enhance the master–slave system manipulability and more intuitively transfer the slave robot’s dexterity to the user. Simulation and experimental studies are performed to validate the performance of the proposed control strategy. Results demonstrate that by employing the proposed manipulability index, we can enhance the user’s control over the force/velocity of a surgical robot and minimize the user’s control effort for a teleoperated task.

This paper introduces a dual-user training system whose design is based on an energetic approach. This kind of system is useful for supervised hands-on training where a trainer interacts with a trainee through two haptic devices, in order to practice on a manual task performed on a virtual or teleoperated robot (e.g., for an Minimally Invasive Surgery (MIS) task in a surgical context). This paper details the proof of stability of an Energy Shared Control (ESC) architecture we previously introduced for one degree of freedom (d.o.f.) devices. An extension to multiple degrees of freedom is proposed, along with an enhanced version of the Adaptive Authority Adjustment function. Experiments are carried out with 3 d.o.f. haptic devices in free motion as well as in contact contexts in order to show the relevance of this architecture.

This paper presents a novel approach to implement bilateral control loops between local haptic devices and remote industrial manipulators using a layer of simulation and virtual reality. The remote scene of manipulation has been visualized in an open-source software environment, where forward and inverse kinematics of the manipulators can be computed. Therefore, the explicit knowledge of mathematical models of the robots is not required for the implementation of the proposed bilateral control schemes. A haptic coupling has been designed between the human operator and the task in the remote environment. Virtually introduced force feedback has contributed to the performance of the proposed bilateral loop by facilitating the adaptation of unexperienced human operators. Teleoperation of one remote manipulator has been experimentally demonstrated with the proposed controllers. Structural modularity of the bilateral haptic control schemes makes them directly extendable for the teleoperation of multiple collaborative robots. Stability and transparency of the proposed bilateral haptic controllers have been theoretically and experimentally investigated.

In this paper, we present a novel stochastic framework for network-based bilateral teleoperation systems. A Bayesian approach, which provides robust tracking performance in real-world applications, is proposed to estimate and predict the stochastic variables and compensate for the unreliable network conditions. Combining with a practical approach in transport and application layers of the Internet, this paper demonstrates a high performance and efficient prediction and estimation method for bilateral teleoperation system. Experimental results show that the proposed Bayesian approach estimates and predicts true position and force data over unreliable network conditions, and therefore, improves the performance of overall teleoperation systems.

Force-controlled series elastic actuators (SEAs) are the widely used components of novel physical human–robot interaction applications such as assistive and rehabilitation robotics. These systems are characterized by the presence of the “human in the loop” so that control response and stability depend on uncertain human dynamics. A common approach to guarantee stability is to use a passivity-based controller. Unfortunately, existing passivity-based controllers for SEAs do not define the performance of the force/torque loop. We propose a method to obtain predictable force/torque dynamics based on adaptive control and oversimplified human models. We propose a class of stable human-adaptive algorithms and experimentally show advantages of the proposed approach.

The teaching of how to exert fingertip forces and how to move the fingers is essential for transferring skill using the fingers to perform fine motor tasks. In this paper we accomplish the transfer of fingertip forces and positions in three-dimensional space by combining a multi-fingered haptic interface robot, which can measure and present the three-dimensional forces and positions at five fingertips, and an image display system that records a trainer's hand image and displays it to a trainee. Several experimental results show a high fingertip force and position transferability and the great potential of our proposed transfer system.

Surgical and search/rescue robots often work in environments with very strict spatial constraints. The tendon-sheath mechanism is a promising candidate for driving such systems, allowing power sources and actuation motors placed outside to transmit force and energy to the robot at the distal end through the constrained environment. Having both compactness and high force capability makes it very attractive for manipulation devices. On the other hand, the friction attenuation of tendon tension is nonlinear and configuration-dependent due to tendon/sheath interactions throughout the transmission path. This is a major obstacle for the tendon-sheath mechanism to be widely adopted. Here, we focus on the friction analysis for flexible and time-varying tendon-sheath configurations: the most challenging but yet commonly encountered case for real-world applications. Existing results on fixed-path configurations are reviewed, revisited, and extended to flexible and time-varying cases. The effect of tendon length to friction attenuation is modeled. While focusing on tension transmission, tendon elongation is also discussed with the length effect applied. In the end, two-dimensional results are extended to three-dimensional tendon-sheath configurations. All propositions and theorems are validated on a dedicated experimental platform.

The present paper addresses the issues that should be covered in order to develop walk-through programming techniques (i.e. a manual guidance of the robot) in an industrial scenario. First, an exact formulation of the dynamics of the tool the human should feel when interacting with the robot is presented. Then, the paper discusses a way to implement such dynamics on an industrial robot equipped with an open robot control system and a wrist force/torque sensor, as well as the safety issues related to the walk-through programming. In particular, two strategies that make use of admittance control to constrain the robot motion are presented. One slows down the robot when the velocity of the tool centre point exceeds a specified safety limit, the other one limits the robot workspace by way of virtual safety surfaces. Experimental results on a COMAU Smart Six robot are presented, showing the performance of the walk-through programming system endowed with the two proposed safety strategies.

Human reaction to external stimuli can be investigated in a comprehensive way by using a versatile virtual-reality setup involving multiple display technologies. It is apparent that versatility remains a main challenge when human reactions are examined through the use of haptic interfaces as the interfaces must be able to cope with the entire range of diverse movements and forces/torques a human subject produces. To address the versatility challenge, we have developed a large-scale reconfigurable tendon-based haptic interface which can be adapted to a large variety of task dynamics and is integrated into a Cave Automatic Virtual Environment (CAVE). To prove the versatility of the haptic interface, two tasks, incorporating once the force and once the velocity extrema of a human subject's extremities, were implemented: a simulator with 3-DOF highly dynamic force feedback and a 3-DOF setup optimized to perform dynamic movements. In addition, a 6-DOF platform capable of lifting a human subject off the ground was realized. For these three applications, a position controller was implemented, adapted to each task, and tested. In the controller tests with highly different, task-specific trajectories, the three robot configurations fulfilled the demands on the application-specific accuracy which illustrates and confirms the versatility of the developed haptic interface.

Psychophysiological measurements, which serve as objective indicators of psychological state, have recently been introduced into human–robot interaction. However, their usefulness in haptic interaction is uncertain, since they are influenced by physical workload. This study analyses psychophysiological responses to a haptic task with three different difficulty levels and two different levels of physical load. Four physiological responses were recorded: heart rate, skin conductance, respiratory rate and skin temperature. Results show that mean respiratory rate, respiratory rate variability and skin temperature show significant differences between difficulty levels regardless of physical load and can be used to estimate cognitive workload in haptic interaction.

Tendon sheath actuation is found in many applications, particularly in robotic hands and surgical robots. Due to the friction between the tendon and sheath, many undesirable characteristic such as backlash, hysteresis and non-linearity are present. It is desirable to know the end-effector force and elongation of the tendon to control the system effectively, but it is not always feasible to fix sensors at the end effector. A method to estimate the end-effector parameters using only a force and position sensor at the proximal site is given. An analytical study is presented and experiments are reported to support the result, showing a maximum full-scale error of approximately 7%. This result is achieved if the shape of the sheath remains the same and buckling is negligible. The results presented in this study could contribute towards haptic development in robotics surgery.