1. Introduction
Manipulation of soft materials by robots is a research field that has been drawing the attention of researchers and engineers for recent decades, not only for its high value in application but also for its technical difficulty. Our daily life is filled with repeating manipulation of soft materials, and enormous time and labor would be saved if robots could do these tasks for us. Representative instances include folding and arrangement of clothes in laundry, dressing, bed-making, and others.
Regarding the robot platform for soft material manipulation as a whole, numerous single-arm and dual-arm solutions have been developed and provided. While the former has lower cost and is simpler in mechatronic system, it is also limited in manipulation capability and is inefficient compared to dual-arm robots. On the other hand, dual-arm robots have the advantage of higher manipulation capability and efficiency, although it is also more complicated and the collaboration of two arms must be taken into consideration as well. Consequently, single-arm robots tend to be used mainly for the validation and verification of algorithms of perception and grasping [Reference Wu, Yan, Kurutach, Pinto and Abbeel1–Reference Hietala, Blanco-Mulero, Alcan and Kyrki4], while dual-arm robots [5–Reference Avigal, Berscheid, Asfour, Kröger and Goldberg11] concentrate more on the actual application tasks with higher requirement for hardware and technical difficulty, especially on human-like tasks that can only be done with both two arms.
Development of soft material manipulation by dual-arm robots is remarkable. For example, “Blue” in University of California, Berkeley [5] is capable of dexterous and human-like folding of a cloth, with the help of vision and force sensor feedback and AI learning. Nevertheless, the authors believe that dual-arm robots are not the only “final answer,” because two individual grasping points or areas may not be sufficient for some daily manipulation tasks of soft materials. Essentially, manipulation of soft material like a cloth is dealing with a wide area of plane, but it takes at least three points to form a plane, which cannot be completely fulfilled by dual-arm robots that only have two individual grasping points or areas. Instances of tasks that may be difficult for existing dual-arm robots include but are not limited to (1) Opening a hole at a specific point on a cloth while it is spread; (2) Cutting a cloth along a specific line on it; and (3) Wrapping of soft materials on a target that needs to be fixed. Meanwhile, when there is no sufficient space (or even no space) to place the target soft material, an extra arm would be needed for a dual-arm robot to perform manipulation tasks while the other two arms are holding the soft material in the air.
Therefore, for the exploration of better manipulation performance and flexibility, the authors developed a new robot platform “Quad-SCARA,” which is improved from the last version described in the authors’ former work [Reference Sun, Ishida, Makino, Shibayama, Terada, Niola, Gasparetto, Quaglia and Carbone12] and will be introduced later. The platform mainly consists of four SCARAs, four air grippers for grasping, and other necessary components to drive and control them. Note that instead of serial link robot arms that are commonly used in the former studies, SCARAs are used in this platform. In comparison with serial link robot arms, SCARAs have less Degree of Freedom (DoF), simpler structure, and their workspace can be considered as “2D + height” instead of 3D space. These features separate Quad-SCARA from a platform that simply integrates four serial link robot arms together. An example can be seen in ref. [13], where four serial link robot arms are used, yet the whole system is huge and complicated and is not well-suited for dexterous manipulation of soft materials.
Increased robots and individual grasping points or areas in Quad-SCARA are beneficial for manipulation performance, but on the other hand they also result in a higher risk of collision because their workspaces partially overlap with each other for the consideration of collaborative operation. Strategies against robot collision and human-robot collision mainly include (1) Impact mitigation for unavoidable collisions, with an instance that utilizes elastic deformation of passive compliant joints [Reference Seriani, Gallina, Scalera and Lughi14] and (2) Collision avoidance by area assignment or division in space and avoiding area overlap such as dynamic safety zone [Reference Scalera, Vidoni and Giusti15, Reference Scalera, Giusti, Vidoni and Gaspareto16], reachable occupancy [Reference Beckert, Pereira and Althoff17], and others. The strategy in this paper belongs to the latter, and a collision avoidance system for robot motion planning during multi-point grasping is developed and described. The core idea is based on and inspired by Buffered Voronoi Cell (BVC) [Reference Zhou, Wang, Bandyopadhyay and Schwager18], a simple but efficient collision avoidance algorithm that was originally proposed for multi-agent systems. One point worth mentioning is that BVC is mostly validated and implemented in 2D space, which also matches perfectly with SCARAs’ “2D + height” workspace.
BVC has been drawing increasing attentions among various collision avoidance algorithms, with its implementations and variations [Reference Wang and Schwager19–Reference Zhu, Brito and Alonso-Mora22] proposed in only about five years. In comparison with other known algorithms for collision avoidance, such as RVO (reciprocal velocity obstacle) [Reference van den Berg, Lin and Manocha23], ORCA (optimal reciprocal collision avoidance) [Reference van den Berg, Guy, Lin and Manocha24], and SCP (sequential convex programming), the biggest advantage of BVC is that it only requires position data (current and goal position) as input for trajectory planning with collision avoidance, while the others require both position and velocity data as input [Reference Zhou, Wang, Bandyopadhyay and Schwager18]. Moreover, the computational complexity of BVC is the least among all. Meanwhile, the requirement of input accuracy for BVC is the lowest among all as well, because the buffer of BVC that will be introduced afterward in this paper could absorb errors to a certain degree. The contents of the comparison above are summarized in Table I.
n: number of neighboring agents around an agent.
Besides the Quad-SCARA platform and collision avoidance system, the contribution of this paper also includes the system integration and optimization in (1) Hardware, (2) Software, and (3) Algorithm. For (1), the end-effector design of all robots is optimized as the result of give-and-take between space and reachable range. For (2), a simulator is developed for the examination of collision, motion planning, and calculation of other parameters. For (3), a novel approach for the implementation of BVC called “static BVC” is proposed for self-collision avoidance, with its parameters optimized as well.
This paper is organized as follows. Section 2 introduces Quad-SCARA platform and its components, including its hardware components, system configuration, and the developed simulator. Section 3 introduces BVC algorithm as the prerequisite knowledge and its mechanism for collision avoidance. Section 4 shows the new proposition about adaptation and optimizations made for the best compatibility between BVC and Quad-SCARA. Section 5 presents simulation and experiment results to evaluate the Quad-SCARA platform and the proposed collision avoidance system. Section 6 draws the conclusion of this paper and shows future works.
2. Introduction of Quad-SCARA platform
2.1. Overview
The overview of Quad-SCARA is depicted in Fig. 1. Four SCARAs (Epson G3-251S) are adopted, and an end-effector is equipped at the end of each SCARA. All robots are fixed to a thick metal table. The surface of the metal table is covered by blue papers, not only for the better look and contrast between robots and manipulation targets but also for the protection of end-effectors. The main specifications of the SCARAs used in this paper are listed in Table II, and the layout of SCARAs is shown in Fig. 2.
The total DoF of one robot arm in Quad-SCARA is four, with three revolute joints (base, joint, wrist) and a prismatic joint at its end. This number of DoF is much fewer than serial link robot arms that have six, seven, or even more DoFs. This difference in DoF becomes even bigger when it comes to a platform of four robot arms. The total number of DoF in Quad-SCARA is 12∼16 depending on whether the prismatic joints are used or not, and it would be 24∼28 if they are all serial link robot arms that are mainly adopted by former studies. This platform aims at performing manipulation tasks of soft materials that might be difficult for dual-arm platforms or tasks that would be more efficient when there are four robot arms, like spreading a cloth or a handkerchief from a state that it is randomly crumpled.
The system configuration of Quad-SCARA is illustrated in Fig. 3. The main PC plays the role of (1) Remote access to the other four PCs to control four SCARAs; (2) Control of four end-effectors; and (3) Motion planning in collision avoidance. The digital output board that is connected to the main PC makes it possible to send command signals to each robot controller simultaneously, thus guaranteeing the synchronization of all four SCARAs. At present stage, there is no sensor feedback for Quad-SCARA and the whole system is working with offline motion planning. The authors are planning to add sensors in near future for real-time control and flexibility of the whole system.
2.2. End-effector
Air grippers (Airtac HFTK 20) are chosen and connected to electromagnetic valves and a compressor in Quad-SCARA platform. Different from normal grippers that are widely used in previous studies, mechanical “fingers” are designed for this platform and extend out horizontally from the air grippers, which is a result of the optimization between space and collision avoidance, with more details explained in Section 4.1.
The close view of the end-effector (including an air gripper and two symmetrical fingers) is shown in Fig. 4. The scale of two fingers is 52 mm (L)*46 mm (W)*18 mm(H), and they are made in stainless steel (SUS440C) with the thickness of 2 mm. There is a 3 mm-long extended part at the lower end of each finger to guarantee optimal and pinpoint grasping, avoiding unexpected contact between the other parts of fingers and the target soft material.
2.3. Kinetic simulator
As explained in Section 1, one of the advantages in SCARA is that its workspace can be described as “2D + height.” Accordingly, a 2D kinematic motion simulator is developed by the authors to simulate the planned motion of each SCARA and confirm if there is any collision. Figure 5(a) presents an example view of the simulator. It is developed and programmed in Python and runs on Windows 10.
As for the judgment of collision, the contours of each SCARA and end-effector are simplified as the combination of lines and circles that are slightly bigger than the real ones to make the modeling and calculation easier. Specifically, collision judgment now becomes the intersection check between lines and/or circles. When a motion of Quad-SCARA is planned, it can be played in animation in this simulator and if there is a collision, the corresponding contours will be marked in red as the alert just like Fig. 5(b) with a warning message in command line. Meanwhile, exceptions of collision are also considered, where the contours from different SCARAs do intersect with each other, but there is no collision because of the difference in height.
3. Collision avoidance
In this section, the definition and introduction of BVC come first, followed by its mechanism of collision avoidance explained in steps.
3.1. Definition of Voronoi Cell
Assuming that there are $N$ agents (robots, cars, drones, etc.) in a 2-dimensional plane, conventional Voronoi Cell [Reference Okabe, Boots, Sugihara and Chiu26] of the $i$ th agent can be defined as
which can be rewritten as
Where $\mathcal{V}_{i}$ denotes the Voronoi Cell of the ith agent, $\textbf{p}$ denotes the position of an agent, $\mathbb{R}^{2}$ denotes the 2-dimensional Euclidean space, $\textbf{p}_{i}$ and $\textbf{p}_{j}$ denote the position of the $i$ th and $j$ th agent, respectively. $\parallel \cdot \parallel$ denotes the Euclidean distance and $i,j\in \{1,2,\cdots,N\}$ .
Geometrically, Voronoi Cells (also known as Voronoi diagram) can be visualized as in Fig. 6(a), where each agent is plotted as a point, and the borderlines between Voronoi Cells are exactly the perpendicular bisectors between each pair of points. Note that all perpendicular bisectors are trimmed properly so that there are no crossed lines.
3.2. Definition of BVC
Based on the definition of Voronoi Cell, BVC [Reference Zhou, Wang, Bandyopadhyay and Schwager18] additionally introduces the concept of safety distance $d$ that satisfies
BVC also can be simply illustrated as in Fig. 6(b), where blue lines parallel to each borderline in gray are added in comparison with Fig. 6(a), with their distance ( $d$ ) to the corresponding borderlines being all identical. The area surrounded by blue lines is called “buffer,” where all agents are not allowed to enter.
Regarding the implementation of collision avoidance, buffer in BVC is significant because while in Voronoi Cell “cores” have no size, but in reality each agent has its actual size, and there exists a certain “safety distance” between each two agent to avoid collision. Apparently, the value of $d$ can be set freely according to the size of each agent as well as other factors that may cause collision. As a result, the BVC of the $i$ th agent can thus be defined as
which is also the expansion of Eq. (2). From the definition of buffer and BVC as well as its property shown in Eq. (3), it is not difficult to find that BVC always keeps the distance between each pair of cell core longer than 2 $d$ .
3.3. Collision avoidance by BVC
This part explains how BVC works in trajectory planning to avoid collision. BVC itself is no more than an extension of Voronoi cell that can divide a space into multiple parts. However, things would be different if BVC keeps updating while the agents are moving toward their corresponding goals, and the updated BVC determines where each agent should be going at the same time. In another word, collision avoidance of BVC is just like a video, and in each frame, the expected position of each agent in the next frame is given according to the current and goal potion of each agent.
Take Fig. 7 as an example. Assume that there are four agents, with their current and goal position given as the inputs of BVC, which are shown in Fig. 7(a). Detailed definitions of marks can be seen in the title of Fig. 7. Just as what is described in Section 3.2, BVC chart can be drawn in Fig. 7(b) according to these inputs and the preset buffer size $d$ . Here comes the essential part of collision avoidance by BVC: calculation of position for each agent in the next frame.
First, similar to buffer size $d$ , the step distance $l$ that each agent moves for each frame can be set freely by the users. The next process would be divided into two cases: (1) The goal is inside the BVC of corresponding agent and (2) The goal is outside the BVC of corresponding agent. In the case of Fig. 7, the agent in green belongs to case (1) and the agents in red, yellow, and purple belong to case (2).
For case (1), the agent would go straight towards its goal for the distance $l$ . For case (2), the agent would go along the line connecting the current position and midpoint of the agent for distance $l$ instead. Here, the midpoint of an agent is defined as the point on the edges of agent’s BVC (inner buffer) that is closest to the goal. Two cases described above are also illustrated in Fig. 7(c).
Steps above explain all steps required for one frame in collision avoidance by BVC. In application, these steps will be repeated with the update of BVC until all agents reach their corresponding goals. Subsequent frames after Fig. 7 are presented in Fig. 8. Here, the frame in Fig. 7 is defined as Frame 1; thus, Fig. 8(a) is Frame 2, Fig. 8(b) is Frame 3, and Fig. 8(c) is the final frame where each agent reaches the corresponding goal. Note that the step distance $l$ that each agent moves in one frame in Figs. 7 and 8 is much longer than the real one so that it is easier to distinguish one frame from another. In this paper, $l$ is set to be 1 mm for smooth trajectory planning and better collision avoidance performance. Flowchart in Fig. 9 sums up the entire flow described above.
4. Optimizations and adaptations
Unfortunately, direct application of BVC to Quad-SCARA is not practical, because BVC was proposed for multi-agent system where each agent (robot, car or any other mobile vehicle) can move freely on a 2D space, while it is not the case for robot arms with much more constraints. Moreover, the total workspace for Quad-SCARA is also relatively small considering the limited arm length of 250 mm for each SCARA. Consequently, the robot system is modified and the implementation of BVC is also extended to adapt to Quad-SCARA platform.
4.1. Design of end-effector
As explained in Section 2.2, fingers that extrude horizontally are added to each end-effector for the grasping of soft materials instead of grasping vertically with air grippers. There are mainly two reasons:
-
(1) Taking advantage of the revolute joint at the end of each SCARA. Generally speaking, the revolute joint at the end of a SCARA is used for the twist of tools equipped on it, like drills. However, by adding horizontal fingers each SCARA obtains an additional link that can be considered as a new redundant DoF, enabling more motion options in collision avoidance and improving reachability.
-
(2) Size optimization that considers both the layout of robots and reachability. The layout of Quad-SCARA is illustrated in Fig. 2, which is relatively crowded. If the end-effector extends less, collision avoidance would be easier but its advantage in flexibility and reachability that can be gained from point (1) would be weakened and vice versa. Therefore, the end-effector is optimized to the size in Fig. 4.
4.2. Proposition of static BVC
Through simulation verification and robot experiments, an unexpected issue emerged: there is a risk of self-collision. Specifically, the air grippers and fingers may collide with the inner corners of robot bases in some cases. The normal self-collision of SCARAs between their original arm parts and their bases can be simple avoided by the default setting in terms of movable ranges for each joint and others, but it is not the same case for air grippers and fingers in this paper. This is because the air grippers and fingers are additionally equipped to the robots and work independently from SCARAs, and there is no appropriate default setting available for the robots to utilize.
To address this issue, a new approach of BVC implementation called “static BVC” is proposed. Specifically, besides treating each robot end-effector (one gripper and two fingers) as a BVC for collision avoidance, each inner corner of the robot base is additionally treated as a fixed BVC to avoid the self-collision described above. Figure 10 illustrates the image of this proposition. During the trajectory planning and collision avoidance of SCARAs, four normal cells in orange keep moving and avoiding the collision with all types of cells, including static cells in red that stay still. Simulation and experiment results prove that this proposition is effective, which will be presented in detail in Section 5.
Since static BVC is verified to be effective for static obstacles with their size known, it is expected that this idea can also be utilized in other collision avoidance situations besides the original BVC.
4.3. Optimization of buffer size
Theoretically, larger buffer size means safer collision avoidance where agents are farther from each other. However, the total workspace of Quad-SCARA is limited. This means that there may be no solution for BVC if the buffer is too large, and the collision avoidance may not be guaranteed if the buffer is too small. Empirical results and data from simulation validation reveal that a buffer of 20 mm is too small for collision avoidance, and a buffer more than 30 mm may cause no solution.
Taking the concerns explained above into consideration, the authors newly introduce the index of “shortest distance” between each pair of end-effector for the optimization of buffer size and evaluation of collision avoidance. Specifically, the simulator calculates all distances between each point and lines that form the contours of end-effectors, and the least one among them is defined as the shortest distance. With the reference of shortest distance, the buffer size is optimized to 25 mm in this paper.
5. Experiments
In this section, a grasping test is shown first to evaluate the graspability of the designed end-effector to soft materials. A handkerchief is used as the target. After that, main experiments of this paper are presented, with all four SCARAs performing collision avoidance while folding and spreading a cloth. Results are presented in snapshots of robots, simulations, and BVC charts.
5.1. Grasping of a handkerchief
Figure 11 shows the process of an end-effector grasping and moving a handkerchief with the size of 250 mm*250 mm. The height of end-effector is controlled manually. The end-effector approaches the handkerchief in the second picture, succeeds in grasping in the third picture, and lifts the handkerchief up in the fourth. Then, the end-effector moves horizontally while keeping the handkerchief in the fifth picture and releases it in the last. Similar experiments have been performed for 10 times and are all successful, proving its graspability with the condition of a correct input height.
5.2. Collision avoidance
To validate and evaluate the overall performance of collision avoidance system on Quad-SCARA, an experiment was made with the results depicted in Figs. 12, 13, and 14 and conditions listed in Table III. The experiment starts with the state that all four corners of the target cloth are grasped. The goal positions of all SCARAs for folding motion are given, and after the folding motion, all robots return to their corresponding initial positions, thus completing the spreading motion. In Fig. 13, the calculated shortest distances are displayed, with their corresponding locations marked in pink dotted lines.
5.3. Discussions
The proposed collision avoidance system for Quad-SCARA in multi-point grasping state is successful throughout the whole process of both folding and spreading motion, not only in avoiding the collision between robot end-effectors but also in avoiding self-collision with the proposition of static BVC.
Figure 15 shows the minimal value of shortest distance detected in simulation throughout the experiment described in Section 5.2, which is 1.99 mm. Note that its value in reality would be approximately 2 mm longer than that in simulation, which is about 4 mm. This is because the contours of the robots and end-effectors in simulator are slightly bigger (1 mm for each) than real ones, just as what is explained in Section 2.3. Therefore, although in simulator it seems to be very close to collision, but in reality there is a sufficient clearance to avoid collision. This result also verifies that the optimization for both end-effector size and BVC buffer size is appropriate and viable.
6. Conclusions and future works
In this paper, a collision avoidance system for Quad-SCARA is proposed, with the detail and development of robot platform introduced. Aiming at dexterous manipulation for soft materials, four SCARAs are used in Quad-SCARA so that the total DoF of all four robots is close to that of two common serial link industrial robots, which are frequently used as a dual-arm robot in the latest studies. The authors explore better performance for soft material manipulation with Quad-SCARAs in comparison with existing dual-arm robots in this study.
First, the detailed development and features of SCARA are introduced and described. Besides off-the-shelf SCARAs, an end-effector consisting of one air gripper and two mechanical fingers that stick out horizontally is developed and equipped to each SCARA for manipulation, with its size optimized for the tradeoff between available workspace and reachable range. In terms of software, a 2D kinematic simulator is developed for the collision judgment, collision avoidance planning, and evaluation. For the crucial approach for collision avoidance, BVC is utilized for its advantage of less requirement on input and good compatibility to 2D workspace that Quad-SCARA mainly deals with. The definition of BVC is given and the mechanism of collision avoidance by BVC is explained in steps. Moreover, the original BVC is adapted for Quad-SCARA especially on its implementation, with the proposition of static BVC for self-collision avoidance, which seems to be available for further expansion. The index of shortest distance is also introduced in this paper for the optimization of buffer size in BVC and evaluation of collision avoidance.
Experiments are performed on the developed robot platform for the evaluation of the proposed collision avoidance system. Results in robot experiments, simulator, and BVC charts all validate that both end-effector collision avoidance and self-collision avoidance are successful, and the data of shortest distance also prove that the optimization in this paper for end-effector size and buffer size for BVC is appropriate.
Future works for this paper include (1) Expansion of collision avoidance by BVC from 2D to 3D space and (2) Introduction of sensor feedback and AI object recognition for the realization of full-automatic soft material grasping and manipulation.
Acknowledgements
The robots along with the controllers and batteries in Quad-SCARA platform were denoted by YGK Corporation, Yamanashi, Japan. The authors would like to express the acknowledgment for their help.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0263574723001236
Financial support
This paper is partially supported by Satoshi Omura Human Resource Development Fund of Yamanashi Prefecture, Japan.
Author contributions
Xiao Sun wrote this paper and proposed the overall systems in this research. Kazuyoshi Ishida and Koji Makino provided valuable insights and comments for this research. Kotaro Shibayama assembled the Quad-SCARA platform and programmed softwares for the platform. Hidetsugu Terada managed and supervised the entire research project.
Competing interests
The authors declare no Competing interests exist.
Ethical approval
Not applicable.