A multi-stage localization framework for accurate and precise docking of autonomous mobile robots (AMRs)

Abstract

Autonomous navigation has been a long-standing research topic, and researchers have worked on many challenging problems in indoor and outdoor environments. One application area of navigation solutions is material handling in industrial environments. With Industry 4.0, the simple problem in traditional factories has evolved into the use of autonomous mobile robots within flexible production islands in a self-decision-making structure. Two main stages of such a navigation system are safe transportation of the vehicle from one point to another and reaching destinations at industrial standards. The main concern in the former is roughly determining the vehicle’s pose to follow the route, while the latter aims to reach the target with high accuracy and precision. Often, it may not be possible or require extra effort to satisfy requirements with a single localization method. Therefore, a multi-stage localization approach is proposed in this study. Particle filter-based large-scale localization approaches are utilized during the vehicle’s movement from one point to another, while scan-matching-based methods are used in the docking stage. The localization system enables the appropriate approach based on the vehicle’s status and task through a decision-making mechanism. The decision-making mechanism uses a similarity metric obtained through the correntropy criterion to decide when and how to switch from large-scale localization to precise localization. The feasibility and performance of the developed method are corroborated through field tests. These evaluations demonstrate that the proposed method accomplishes tasks with sub-centimeter and sub-degree accuracy and precision without affecting the operation of the navigation algorithms in real time.

1. Introduction

In traditional manufacturing, automated guided vehicles (AGVs) follow pre-determined paths marked on the floor using magnetic or color codes to transport goods from one location (workstation, conveyor, warehouse, storage unit, etc.) to another within a factory [1, 2]. However, with the onset of the fourth industrial revolution [3, 4], there is a shift toward creating smarter, more autonomous production islands in factories. As a result, AGVs are being replaced by autonomous mobile robots (AMRs) [5] that can provide logistics between these islands in a fully autonomous manner [6].

AMRs are similar to AGVs in terms of physical structure but differ in their software infrastructure, which provides them with unique capabilities [7, 8]. In order to function effectively in smart factories, AMRs need to have certain fundamental capabilities. For example, when picking up a load, an AMR must be able to travel safely from the loading point to the destination region while optimizing parameters such as energy consumption, travel distance, and time. Furthermore, it must adhere to industrial standards when transporting loads between production islands, conveyor lines, shelves, and other AMRs. To integrate with Automated Storage and Retrieval Systems (AS/RS), an AMR must also be able to dock with high accuracy for loading and unloading. Additionally, the AMR should be able to recharge its batteries precisely by reaching a charging station when necessary. To enable the AMRs to operate autonomously in these and similar scenarios, a precise docking infrastructure is essential.

While traveling along a route, an AMR may not need to operate within the docking zone tolerances at all stages, and this may impose additional operational load. A sonar receiver-transmitter-based positioning system, as discussed by Chen et al. [9], is particularly suitable for large areas and dynamic environments without cumulative errors. However, the performance of such systems can degrade in environments with highly reflective surfaces, such as factories. While such approaches generally have low computational overhead, they do not allow for orientation estimation, which is indispensable for maneuvering processes. However, there are many localization techniques available in the literature that can be used to estimate the pose of the AMR when operating outside the docking zone. The Kalman filter, one of the fundamental approaches used in literature for localization, is widely employed, especially in sensor fusion, to enhance odometer sources. Building on this foundation, one study developed a location estimation method combining ensemble Kalman filtering and compensation approaches [10]. The results demonstrated improved performance compared to the traditional Extended Kalman Filter. Similarly, many more techniques are available in the literature, including rigid body localization using wireless sensor networks [11, 12], WiFi-based localization architecture [13, 14], vision-based approaches using different types of cameras [15–17], and particle filter-based approaches such as Adaptive Monte Carlo Localization (MCL) [18], Self-Adaptive MCL (SA-MCL) [19], and particle flow filtering architecture [20]. It is also possible to improve their performance with suggested arrangements [21]. These methods are well-suited for the delivery stage, where the AMR operates outside the docking zone since they have the advantage of working with similar performance across the entire work area of the factory on a large scale, and their computational load is usually not too high. However, when trying to reach a target with sub-centimeter accuracy [22], these large-scale localization methods may not be sufficient to complete the task at the docking phase with the required precision.

The real-time kinematic technique available for GNSS systems can produce centimeter-level pose estimation accuracy [23, 24], but such architectures are not suitable for indoor environments like a factory. Many localization methods are available in the literature, claiming that they can perform localization in an indoor environment with high accuracy [25, 26]. In some studies, for example, known objects with specific shapes are placed in the docking area to enable relative positioning. Using the contour detection technique, positioning is performed based on these objects. For instance, an approach involving vision-based object recognition developed using neural network architectures, as demonstrated by Nowak et al. [27], has made it possible for autonomous buses to approach charging stations. In other studies, a similar structure has been constructed using geometric markers such as V-shaped [28, 29], L-shaped, VL-shaped, and bar-shaped markers [30]. Some research projects have also explored positioning in docking areas by utilizing special reflector tapes that change intensity values when laser beams are reflected [31].

Similarly, QR code and Apriltags-based positioning approaches are particularly preferred in docking applications where a QR code can be closely positioned with a camera [32–34]. However, the localization performance can rapidly deteriorate, especially depending on the distance and heading between the camera and the QR code, as well as the lighting conditions in the environment. In another study, a QR code-based visual feedback system was designed to correct odometry data through feedback mechanisms [35]. While the achieved localization performance may seem adequate for docking, such localization methods do not provide continuous positioning along the entire motion trajectory. Therefore, it is not feasible for practical applications to outfit an entire factory environment with QR codes or equip cameras to detect QR codes for large-scale localization.

Another precise indoor localization approach is visible light positioning, as described by Guan et al. [36]. However, its applicability in complex and dynamic environments like factories is limited due to its reliance on line-of-sight communication, susceptibility to interference from ambient light, and the requirement for relatively unobstructed sightlines between devices and light sources.

Low-cost millimeter-wave radar is another technique that can be employed for localization purposes in a factory environment, as discussed by Almalioglu et al. [37]. In this method, radar point clouds are matched using scan-matching approaches, enabling localization even under different weather conditions (such as smoke, humidity, dust, etc.). However, its suitability for factory environments may be limited due to its low spatial resolution, performance degradation in the presence of highly reflective surfaces, and its calibration complexity.

It is claimed in some research studies that sub-centimeter precision can be achieved for pose estimation in factory-like environments. Some of these studies suggest hybrid approaches to increase the performance of coarse localization approaches [22, 38, 39], while others develop a localization algorithm for docking purposes only [40–42]. However, deploying high-accuracy positioning approaches at the exact moment of docking may not be sufficient to compensate for errors caused by large-scale localization approaches, as most AMRs have physical movement limitations and may require extra maneuvering. Therefore, instead of using them at the docking moment, it is necessary to continue the movement with precise navigation for a while. A critical question arises here, when and how to switch from large-scale navigation to precise navigation for docking. As a result, it is essential to determine the switching procedure from large-scale localization to precise localization or whether it will be necessary to switch back to large-scale localization after switching to precise localization.

Multi-stage localization methods are available for various applications. For instance, in one study, such a framework was proposed using 5G communication systems [43]. In another study, a wireless sensor network was established using a multi-stage localization method for underwater robotic systems [44]. These studies have enabled localization over vast areas. However, applications like the one in our research, which involve docking, require much higher pose estimation accuracy.

In another study that employs a coarse-to-fine localization approach, precise positioning with sub-centimeter accuracy is achieved by refining it with Trimmed ICP, which includes an outlier elimination step based on the entropy of the rough position generated by AMCL, as presented in Sun et al. [45]. Researchers have developed a multi-precision localization framework that combines a virtual reality headset with AMCL, as presented in Hsiao et al. [46]. While these strategies appear suitable for the problem defined, a specifically developed switching decision mechanism regarding when to activate precise localization during the process for localization precision is absent in these studies.

In this study, as a result, a multi-stage localization framework, including a decision mechanism for switching between large-scale coarse navigation, or the delivery stage, and fine navigation, or the docking stage, is proposed. We note that, although different sensor configurations are employed in studies related to high-accuracy pose estimation, LiDAR sensors, and scan-matching-based methods are commonly used [47–49]. Therefore, our proposed localization framework is designed for systems using scan-matching-based localization approaches in the docking stage. To develop this framework, we conducted various experiments that considered essential parameters, such as distance to the target position, the difference between target and current orientations, and initial pose accuracy, to analyze the pose estimation accuracy profile of the localization approaches. Our findings suggest that the similarity between the model (reference) and data (measured) point sets is the determining factor for the switching point since these parameters impact the similarity between the point sets, and the registration performance of scan-matching approaches is highly correlated with how much the point sets overlap [50, 51]. The presented similarity rate-based probabilistic switching mechanism is validated via field tests using a differential-drive, robot operating system (ROS)-enabled mobile platform. The decision-making process in switching is based on the correlation entropy (correntropy) criterion. The results demonstrate that our proposed mechanism can safely move AMRs and dock them to a target with sub-centimeter precision while imposing a negligible computational load.

While some previous studies have presented methods for coarse-to-fine localization [52, 53], where the initial pose estimation is done coarsely and then refined, resulting in a single final pose, this study proposes a two-stage process. In this new structure, the AMR first moves using coarse localization for a period of time, and then switches to fine localization to dock with the target. This approach differs from previous studies in which the entire process is done in a single stage. In another study by Garrote et al. [54], a similar approach to our work involves providing odometry and scan input to AMCL. Subsequently, pose updates are made by maximizing the overlap between the map and the scan based on the output from AMCL. However, a traditional approach, namely the least-square criterion, is used in estimating the overlap rate, which can reduce localization performance, especially in dynamic and noisy environments. In this regard, it can be asserted that our study offers superior performance. As a result, significant contributions of our study to the field can be listed as follows:

1. 1. Development of a Multi-Stage Localization Framework: A novel multi-stage localization framework tailored for AMRs is introduced, addressing the challenge of transitioning from large-scale navigation to precise docking. Unlike previous studies focusing on a single-stage process or merging multi-process, our framework incorporates a two-stage approach, where the AMR initially uses coarse localization and subsequently switches to fine localization for docking. This innovative design allows for a more nuanced and accurate localization process.

2. 2. Introduction of a Similarity Rate-Based Probabilistic Switching Mechanism: A cutting-edge switching decision mechanism based on the correlation entropy (correntropy) criterion is introduced. This mechanism, utilizing a similarity rate-based approach, facilitates the seamless transition between large-scale and precise navigation. By employing the correntropy criterion, our method enhances the registration performance of scan-matching approaches, ensuring accurate docking even in dynamic and noisy environments. This mechanism represents a significant advancement in the field of AMR localization.

3. 3. Validation of the Proposed Framework through Field Tests: A comprehensive validation process involving field tests conducted on a ROS-enabled mobile platform is performed. The results from these tests demonstrate the effectiveness of our proposed framework and the similarity rate-based probabilistic switching mechanism. The ability to safely move and dock AMRs with sub-centimeter precision, coupled with a minimal computational load, showcases the practical viability and superiority of our approach.

The remainder of the study is organized as follows: Section 2 provides an overview of the necessity and idea behind the proposed multi-stage localization framework. The proposed decision-making process in the localization framework for switching between coarse and fine localization is described in Section 3. The experimental setup, environment, test scenarios, results, and comparisons are presented in Section 4. Concluding comments and future work suggestions are highlighted in the last section, Section 5.

2. Preliminary information on multi-stage localization strategy and necessity

AMRs are expected to perform various tasks in smart factories such as carrying goods from one point to another or autonomous navigation to a charging station. Nevertheless, to carry out such tasks, different operation modes may be required. During the initial part of the task, the primary focus is on energy efficiency and quick navigation from the current position to the destination. Achieving high accuracy in path tracking is not crucial at this stage. Conversely, in the vicinity of the target, the robot’s ability to reach the destination with the desired accuracy and precision becomes imperative. Therefore, the algorithms and their requirements for these segments will differ accordingly.

Figure 1. Demonstration of the multi-stage localization framework and the need to switch between large-scale coarse and precise localization approaches.

In the Industry 4.0 paradigm, sub-centimeter accuracy in docking to a target has become an industrial standard [22], despite higher error rates being acceptable in regions far from the target. In the regions close to the target, two primary challenges include preparing a feasible maneuvering area that accounts for the physical constraints of AMRs and tracking the path with high accuracy and precision. For safe arrival at a destination, the localization algorithm’s accuracy in the docking region must be adequate; otherwise, even with proper path planning and designed tracking, the target may not be reached with the desired precision. Although achieving high-accuracy pose estimation for the entire trajectory may seem desirable, such an approach often requires high computational resources, necessitating the use of different methods outside the docking region. Therefore, developing a multi-stage localization algorithm that can determine the optimal time for switching from a large-scale positioning approach to a highly accurate pose estimation method is crucial for the successful operation of AMRs in smart factories.

Based on the aforementioned explanations, this study divides the task into two distinct stages: the delivery stage and the docking stage. As illustrated in Fig. 1(a), the delivery stage aims to efficiently carry the AMR from its current position to the target vicinity. This stage emphasizes the applicability of the approach in large operating areas, where a rough localization suffices. Conversely, the docking stage is characterized by cautious navigation and aims to attain sub-centimeter precision when docking with the target. Therefore, the localization process in this stage should ensure high accuracy and precision, complying with industrial standards. To this end, a transition decision mechanism, shown in Fig. 1(b), is also proposed to facilitate a smooth switch from the delivery stage to the docking stage. The proposed switching mechanism determines the appropriate localization approach to use based on a combination of factors, including the measurements taken by the AMR, the target pose, and the current pose of the AMR. By analyzing these factors, the mechanism is able to determine which stage of the task the AMR is currently in and select the suitable localization approach accordingly.

3. Developed multi-stage localization framework and probabilistic switching mechanism

The proposed multi-stage localization framework consists of three main components, as illustrated in Fig. 1(b): a large-scale localization approach for the delivery stage, a precise localization method for the docking stage, and a decision mechanism that selects the appropriate technique depending on the AMR’s status.

While approaches used for large-scale localization generally have similar positioning performance over the entire map (operating field), the performance of precise localization approaches and how they vary within the docking region may differ. Therefore, the transition from large-scale to precise navigation, i.e., from the delivery stage to the docking stage, depends on the positioning performance of the precise localization approach. In this study, we developed a probabilistic switching decision mechanism based on scan-matching-based precise positioning approaches, including our previous works such as [26, 42]. In addition, the SA-MCL algorithm presented in our previous study [19] is employed as a coarse localization method.

3.1. Transition zone phenomenon instead of switching point

Relying solely on a single threshold level while transitioning from one algorithm to another may lead to fluctuations or instability around the switching point due to the probabilistic nature of the pose estimation process and the presence of measurement and computation errors. Another challenge is posed by the physical limitations of AMRs, which require precise maneuvering to successfully dock at the desired location. Even when an AMR has entered the docking area, it may briefly exit this region during the maneuvering process. Therefore, choosing a fixed switching point may be inadequate for such scenarios.

To address this issue, a switching region can be employed instead of a single point, and the transition can be implemented with hysteresis behavior within this region. Suppose that the switching mechanism is based on the distance of the AMR to the target, whereby a large-scale localization approach is used until the distance to the target pose is greater than or equal to $r_1$ meters. Subsequently, the algorithm switches to a precise localization approach. However, to ensure stability, a hysteresis behavior is implemented, requiring the AMR to be more than $r_2$ meters from the target to switch back to the large-scale localization approach where $r_2 \gt r_1$ . However, heuristically using the distance between the AMR and the target as the sole parameter for the switching decision may not be sufficient since the difference in orientation between the target and current positions, as well as the initial pose accuracy parameters, are also crucial in achieving accurate pose estimation. These factors all contribute to the determination of the corresponding point for scan-matching-based precise localization approaches and ultimately affect the similarity between the model and data point sets. This similarity is a crucial factor in determining the accuracy of pose estimation, as it impacts the registration of point sets.

3.2. Similarity based on overlap ratio computation

When the similarity between the model and data point sets falls below a certain threshold level, the point sets may not be registered correctly, resulting in a reduction in pose estimation accuracy. Therefore, real-time computation of the similarity between the model and data sets enables the determination of the switching point for the transition from large-scale localization to precise localization or vice versa.

In the literature, the term overlap ratio is commonly used to express similarity between point sets. Performance evaluation of registration methods is carried out by adding disruptive components such as noises, outliers, and missing data to the point sets at known rates [55, 56], and initial overlap ratios between the point sets are given to make performance comparisons between algorithms [57]. Therefore, the overlap ratio is easily quantifiable in such articles, given that the amount of disruptive components in the point sets is known or adjusted. However, estimating the degree of similarity between two given point sets becomes more challenging when there is no prior knowledge about the level of disruptive components.

In the trimmed ICP paper [58], which is one of the milestones in the point set registration field, a method for calculating the overlap ratio ( $\eta$ ) between a model and data point sets was proposed. Let two point sets denoted as $\mathcal{X}\triangleq \{\vec{x}_i\}_{i=1}^{\mathit{N_x}}$ and $\mathcal{Y}\triangleq \{\vec{y}_j\}_{j=1}^{\mathit{N_y}}$ represent the model and the data sets. Similarly, $\vec{y}_{j(i)}$ denotes the correspondence of $\vec{x}_i$ , the ith element of the data set, in the model set. The objective function to compute the optimal overlap ratio between point sets was defined as

(1) \begin{equation} \gamma (\eta ) = \dfrac{1}{N_\eta } \left (\dfrac{\sum _{k = 1}^{N_\eta } d_k^2}{\eta ^{1+\lambda }}\right ) \end{equation}

where $\lambda$ is a preset parameter to eliminate undesired cases such as the matching of symmetric parts of the point sets and $d_k$ represents the distance between the $k$ th corresponding point pair. The number of overlapping points can be calculated as $N_\eta = \lfloor \eta \cdot N_x \rceil$ for a certain overlap ratio $\eta$ . Here, the overlap ratio is a measure that ranges from 0 to 1 and indicates the degree of similarity between point sets. The least-squares criterion is the basis for finding the overlap ratio. However, least-squares approaches are not robust enough to process point sets containing noise and/or outlier components [59]. Since disturbances such as noise, outliers, and missing parts are inevitable in real-world applications [60], a new metric based on the correntropy criterion has been defined to be used in the decision mechanism developed to switch from large-scale localization to precise localization.

3.3. Using correntropy criterion for similarity rate computation

The correntropy criterion, which is a combination of the terms correlation and entropy, addresses a generalized measure of similarity between two random variables [61]. The least-squares criterion may be used to get information about the difference between point sets, whereas the similarity between point sets can be inferred with the correntropy criterion [62]. Let $\rho$ be the similarity rate between the point sets to be calculated using the correntropy criterion, in this case, the objective function to be optimized to get the similarity rate can be constructed as

(2) \begin{equation} \phi (\rho ) = \dfrac{\partial e_{corr}(\rho )}{\partial \rho } \end{equation}

where $e_{corr}$ represents

(3) \begin{equation} e_{corr}(\rho ) = \dfrac{1}{N_\rho }\sum _{k = 1}^{N_\rho } \exp \left ( -\dfrac{d_k^2}{2\sigma ^2}\right ) \end{equation}

summation where $\sigma$ is the kernel width for the correntropy criterion. As a result, the value at which the change of the $e_{corr}$ with respect to $\rho$ is minimum gives the similarity rate between the point sets. For a particular similarity rate $\rho$ , the number of similar points between the model and data sets can be executed via $N_\rho = \lfloor \rho \cdot N_x \rceil$ relation. The optimal value of the similarity rate can be computed by following the procedure below:

1. 1. Find corresponding points between the model and data sets using the initial transformation $(\textbf{R},\vec{t})$ kd-trees nearest neighbor searching method [63] is employed to determine corresponding point pairs.

2. 2. Sort the point pairs in ascending order considering the distance between the corresponding point pairs $d_k = \lVert (\textbf{R}\vec{x}_k+\vec{t})-\vec{y}_{j(k)}\rVert$ relation is used to calculate this distance for $(\vec{x}_k,\vec{y}_{j(k)})$ point pair.

3. 3. Determine $\sigma$ parameter for existing model and data sets according to the distribution of distance values between the corresponding point pairs. In this work, the standard deviation calculated by ignoring the distance values greater than twice the median value of the distribution has been selected as the $\sigma$ value.

4. 4. Apply Golden Section Search [64] or similar algorithm to find the optimal $\rho$ that makes the $\phi (\rho )$ in (2) minimum.

It is common to use the correntropy criterion to obtain the transformation that maximizes the similarity between point sets. However, the use of the correntropy criterion in evaluating how similar two point sets are may have been put forward in this study. For this reason, it would be appropriate to examine the performance of the derived similarity rate definition in finding the similarity rate between point sets.

3.4. Verification of similarity rate definition

First, synthetic point sets have been generated to investigate and demonstrate the feasibility of similarity rate definition in expressing similarity between point sets. Imitated laser scan measurement sets are utilized for this purpose.1 The model and data sets, point sets to be employed for similarity rate and overlap ratio analysis, are generated as in Fig. 2. A laser scanner with 30 m range and 0.25 $^\circ$ resolution is placed on the blue point on the occupancy grid map, and the original point set is obtained. Then uniformly distributed noise is inserted into each point in the set, and noisy points with Gaussian distribution are added to the point set to construct the data set. The original occupancy grid map is slightly modified for the model set production by adding a simple obstacle to the environment, and a fake laser scanner with the same specifications is located at the red point to get the modified point set. Then, since it is aimed to calculate the similarity rate in different cases, the modified point set is cropped at different rates.

Figure 2. Synthetic laser scan data generation for similarity rate definition performance tests.

The real similarity of two point clouds can not be computed since this is a probabilistic definition. That is why the similarity rate computed according to the aforementioned definition is compared with the real (ground truth reference) overlap ratio between the point sets in terms of whether they have similar increase/decrease profiles.

Figure 3. Performance of overlap ratio and similarity rate approaches in estimating two laser scan sets’ relation through ten separate cases for different initial transformation states: initial transformation between two laser scan sets is (a) Zero, (b) 1% for translation and $\pm 7^\circ$ for orientation, (c) 2% for translation and $\pm 14^\circ$ for orientation, and (d) 3% for translation and $\pm 21^\circ$ for orientation. The graphs show the characteristic of similarity estimation results for each initial transformation pair.

The similarity rate computation results for the model and data sets described in Fig. 2 are given in Fig. 3 over four experiments. In the first experiment seen in Fig. 3(a), there is no initial transformation between the model and data sets. For this scenario, the ground truth (GT) reference and the results computed by the trimmed ICP method almost completely overlap as expected. The similarity rate calculations developed within the scope of this study based on the correntropy criterion are parallel with the results of GT.

In other experiments in Fig. 3(b-d), the data set is transformed with a certain initial transformation involving a rotation and a translation. Although the overlap ratio definition in trimmed ICP seems to be sufficient in calculating the similarity between point sets in low-rate transformations, its performance rapidly decreases when the amount of rotation and translation between point sets increases. However, when the similarity rate definition is utilized to calculate the similarity between the point sets, it is seen that similar results are produced for different initial transformations. The results reveal that consistent outputs are obtained when the similarity between point sets is calculated with the similarity rate definition based on the correntropy criterion.

3.5. Probabilistic switching mechanism using similarity rate

As explained by the transition zone phenomenon in Section 3.1, designing a transition technique using a single threshold level may cause instability at the time of transition due to calculation and measurement errors. In the developed probabilistic approach, the similarity rate between the point sets will determine the boundaries of this transition region, as shown in Fig. 4(b). However, when the transition is made according to the similarity rate, these boundaries become floating rather than a fixed distance, as demonstrated in Fig. 4(a). In this case, the switching region will take shape depending on which side the AMR reaches the target and how the obstacles are located around the docking region. This arrangement will support the provision of docking at industrial standards.

Figure 4. Probabilistic switching decision mechanism between coarse and fine localization approaches based on similarity rate parameter.

An example describing that the similarity rate may differ for the same distance to the target is given in Fig. 5. It can be deduced that the similarity rate may be lower for a point closer to the target, depending on the environmental conditions. This result supports the necessity of the presented probabilistic switching idea.

Figure 5. Demonstration of floating boundaries over a scenario: (a) Map of the environment, (b) Four different paths followed to reach target pose, and (c) Similarity profiles of the reference and measured point sets for entire paths.

Figure 6. Flowchart for probabilistic switching algorithm between large-scale coarse localization and precise localization.

The preceding section elaborated on the proposed switching mechanism and its operation principle. Fig. 6 illustrates the algorithmic flowchart for implementing the probabilistic switching approach. The flowchart highlights that coarse localization involves the global localization phase as well. Upon completion of the global localization stage, a decision is made regarding whether the AMR is in the delivery or docking stage to execute the corresponding localization approach. The operational logic of the multi-stage localization approach is highlighted in Algorithm 1.

Algorithm 1 Multi-stage localization framework pseudocode

4. Experimental studies

To provide context for the experimental validation of the proposed multi-stage localization framework, it is necessary to first introduce the test unit and the field test environment where the hypotheses are examined through various scenarios. Therefore, the test unit will be introduced first. Secondly, the field test environment where the hypotheses are examined through some scenarios and application principles of the experiments will be identified. Then, the ground truth reference data extraction process related to the experimental setup and methodology will be described. Finally, the experimental results will be demonstrated, and comparisons will be made according to the results achieved.

4.1. Experimental setup

Within the scope of the experimental setup sub-section, the vehicle employed in the field tests and the test environment are introduced.

4.1.1. Test unit: ITU-AGV

The field tests employed the ITU-AGV, as depicted in Fig. 7, which is a laboratory-type AMR equipped with state-of-the-art sensors, including front and rear 2D laser scanners (SICK LMS200) and an inertial measurement unit (IMU, Xsens MTi-300). The dimensions of the differential-drive ITU-AGV unit are 49 cm in width, 82 cm in length, and 22 cm in height (sensorless). The sensorless ITU-AGV weighs approximately 70 kg and can carry loads of up to 100 kg. It is essential to note that the ITU-AGV can benefit from all the facilities of the ROS.

Figure 7. ITU-AGV: a ROS-enabled and differential-drive laboratory-type AMR unit.

Figure 8. ITU robotics laboratory, the field test environment: (a) The occupancy grid map of the environment, the real scene from (b) the delivery, and (c) Docking zones of the lab.

4.1.2. Test environment

The ITU Robotics Laboratory was the site where field tests were conducted. Fig. 8(a) shows the occupancy grid map of the laboratory, which was generated by Hector SLAM [65]. The size of each grid, which corresponds to the resolution of the occupancy grid map, was chosen as 10 cm to suit large-scale working areas. Real images of the test environment are presented in Fig. 8(b-c). The region in Fig. 8(b) is where the delivery stage of the field tests was conducted, while the region in Fig. 8(c) mainly includes the docking zone. Some parts of the images are highlighted to establish a connection between the occupancy grid map and the environment images. The multi-stage localization strategy proposed in this study was tested by conducting real-world experiments in this field.

4.2. Test scenarios

The prepared scenarios to validate the developed approach with field tests are demonstrated in Fig. 9. In the first two scenarios, the AMR started to move from an initial point and reached the target pose by following the waypoints. In the third and last scenario, the AMR moved from a starting point to a region close to the target and docked to the target by moving backward direction. The point that distinguishes the first scenario from the others is that, in the first scenario, the AMR started to move from a region relatively close to the target in terms of distance. However, it is desired to be shown that the similarity rate between the point sets will remain low since the field of view of the AMR is different for initial and target poses, even though the distance to the target is relatively short.

Figure 9. The scenarios performed for performance evaluations: (a) case 1, (b) case 2, and (c) case 3.

The scenarios are operated as follows:

1. 1. AMR follows the pre-determined waypoints using ground truth reference pose data and reaches the target.

2. 2. During the motion, the switching mechanism determines which localization method will be active.

3. 3. With the active localization approach, the current pose of the AMR is estimated in real time.

4. 4. The localization results are compared with the ground truth to evaluate the performance.

Before providing the implementation procedure of the localization framework, it would be appropriate to clarify how the ground truth reference data were derived for field tests.

4.3. Ground truth reference system

Ground truth reference data are required for comparison to analyze or assess the performance of any novel method or strategy. Producing ground truth data is challenging because it requires a reference source that yields more accurate results than the proposed approach. Since the architecture proposed in this study is aimed at docking to a target with sub-centimeter accuracy, ground truth reference data should include fewer error rates in the docking region.

As shown in Fig. 8(a) and Fig. 8(c), the Optitrack motion capture (MoCap) system is utilized to generate ground truth reference data. According to the datasheet, this optical system is able to produce 3D position and 3D orientation information with error levels less than 0.3 mm and 0.05 $^\circ$ , respectively. In the calibration stage, the average error is achieved as 0.551 mm before the experiments.

In the delivery stage, however, the pose estimation accuracy is lower. Therefore, the accuracy and precision of the fluorescent reflectors-based ground truth data are sufficient and these data are employed in the delivery zone of the field test environment. The layout of the fluorescent reflectors for Case 2 is shown in Fig. 8(b).

4.4. Implementation procedure of the multi-stage localization algorithm

The working principle of the switching decision mechanism is explained with the flowcharts in Fig. 6. As demonstrated there, for the probabilistic approach, the switching parameter is similarity rate definition $(\rho )$ . The upper and lower bounds for the switching parameter can be chosen differently according to the application. The literature suggests that scan-matching-based approaches yield satisfactory results when the overlap ratio between point sets is above 50% [58]. When examining the relationship between overlap ratio and similarity rate definitions over synthetic laser scan sets, as shown in Fig. 3, it was found that the similarity rate value is generally higher than the overlap ratio. Therefore, the center of the transition region from large-scale to precise localization is determined as 0.7 for the probabilistic switching decision mechanism. By adding a $\mp 5\%$ deviation, $\rho _1 = 0.75$ and $\rho _2 = 0.65$ can be assigned as the upper and lower threshold values, respectively, for the probabilistic switching mechanism.

Since the ITU-AGV mobile platform, depicted in Fig. 7, operates in the ROS environment, the developed switching decision approach was adapted to this platform. To achieve this, the switching mechanism was initially implemented in MATLAB-SIMULINK and subsequently transferred to the ROS environment. Further details of this process are provided in the Appendix section.

The authors utilized the scan-matching-based method from their previous publication [42] as a precise localization method in the docking zone of the experiments conducted in this study. Details on transferring the method to the ROS environment can be found in the relevant publication. In the delivery zone of field tests, an MCL variant called SA-MCL, which was introduced in ref. [19], was implemented as a large-scale coarse localization approach since it has demonstrated similar localization performance throughout the entire operating field.

4.5. Results and comparisons

Within the scope of this study, it is stated that the autonomous logistics operation can be divided into two parts, delivery and docking, and the regions of these operations may vary according to the environmental conditions. While the important thing in the delivery region is to carry out the logistics operation without interruption, it is to reach the target, which gains more importance in the docking region, with high accuracy. As a result, a multi-stage localization strategy that can be suitable for the delivery and docking zones and a decision mechanism that can be used to move from the delivery zone to the docking zone or vice versa is presented. The validation results of the proposed mechanism by field tests are discussed in this section in terms of showing why the similarity rate concept is used in the decision mechanism, assessing the performance of the multi-stage localization framework, and comparing it with the state-of-the-art methods.

4.5.1. Insufficiency of overlap ratio for switching

This study employs the similarity rate as a correntropy criterion-based metric, which serves as the decision variable in the switching mechanism for transitioning between large-scale coarse and precise localization. While the overlap ratio is another measure of similarity between point sets that could also be used as a decision variable, field tests have shown that it only produces reliable results when the transformation between point sets is precisely known. For instance, in Fig. 10(a), the overlap ratio outputs are reasonable when the transformation between the point sets is known exactly, which corresponds to the precisely known pose in field tests. However, when using the pose estimation calculated by SA-MCL, which contains some error, for the transformation, the overlap ratio outputs become distorted, as shown in Fig. 10(b). On the other hand, the similarity rate can still predict the similarity between point sets with significant distinctiveness and effectiveness, as demonstrated in Fig. 10(c). Therefore, the similarity rate is utilized as the decision variable in the switching mechanism.

Figure 10. The overlap ratio/similarity rate estimations of trimmed ICP and correntropy for the followed trajectory of case 1: overlap ratio estimates of the trimmed ICP (a) Under the assumption that the initial pose is exactly known, (b) Using initial pose information calculated by SA-MCL, and (c) Similarity rate estimates of the correntropy with the aid of initial pose data computed by SA-MCL.

4.5.2. Performance evaluations of the developed switching strategy

The results of the experiments conducted to highlight the importance and advantages of using a coarse localization approach in the delivery zone and a precise positioning technique in the docking zone are presented in Fig. 11. The results are analyzed over three different cases, demonstrating the issues that may arise when using large-scale coarse localization throughout the entire path or relying solely on precise localization to reach the target. Fig. 11(b) shows that if coarse localization is employed continuously from the initial pose to the target pose, the mean error will remain at a certain level throughout the entire route, but it may not meet industrial standards to reach the target. Conversely, using precise localization from start to end may result in accurate and precise docking to the target, but the AMR may get lost in areas far from the target due to the decrease in pose estimation performance, as shown in Fig. 11(c).

Figure 11. The localization performance of two-stage approach demonstrated through three cases: (a) Waypoints followed, localization by (b) SA-MCL, (c) SM, and (d) SA-MCL + SM vs GT data.

The distance to the target at the switching point in Fig. 11(d) is different for each case. This is because the switching point is determined based on the similarity rate parameter, which is affected by the environmental conditions and the specific path followed by the AMR. In Case 1, the switching point is closer to the target since the path is shorter and more straightforward, while in Case 3, the switching point is farther away since the path is longer and has more turns. However, in all three cases, the multi-stage localization approach leads to a more accurate and reliable docking operation. This demonstrates the necessity and benefit of using a combination of coarse and precise localization techniques in autonomous logistics operations, depending on the specific environmental conditions and operational requirements.

If the switching steps are examined through Case 2, it can be deduced that the switching point determined by similarity rate is quite efficient. As seen in Fig. 12, the pose estimation performance of the SM is low in the region where coarse localization, that is, SA-MCL, is active. As of the switching point, coarse localization is deactivated, and fine localization becomes active. It is seen that the pose estimation performance of SM is satisfactory in the docking zone where the SM is active. This allows for reaching the target with the desired accuracy.

Figure 12. Two-stage localization approach application for case 2 - the SA-MCL is active during the delivery stage, and the SM becomes active (docking stage) when the switching point is reached, according to the similarity rate computed employing the measurements: (a) Similarity rate profile, (b) and (c) Pose estimation performance of the algorithms for x and y axes, respectively, and (d) the trajectory followed in case 2 and estimated poses by the algorithms.

At the switching point, there can be a significant difference between the last calculated pose of coarse localization and the first pose computed by fine localization, depending on the pose estimation accuracy rate of coarse localization. In such cases, jumps may occur in the position estimation, as shown in Fig. 12(b). These jumps may cause fluctuations in the controller output used for path following. To avoid such unwelcome situations, one of the following two remedies can be implemented: (1) When a transition from coarse to precise localization or vice versa is decided, the AMR should first stop and resume movement after the transition is completed. (2) In the transition zone, a weighted average of the estimated poses can be used to ensure a smooth switching.

Figure 13. The pose estimation error rates of the approaches (SA-MCL (dashed-blue), SM (solid-orange), SA-MCL + SM(dotted-yellow)): average, minimum, and maximum (a) Position and (b) Orientation estimation error rates for entire trajectories, overall (c) Position and (d) Orientation estimation error rates of reaching target pose for 100 separate offline trials.

To test the algorithms in terms of docking accuracy, 100 offline trials were performed on the paths of case 1, case 2, and case 3. Fig. 13(a) and Fig. 13(b) indicate minimum, maximum, and average error rates for entire trajectories. As seen from them, multi-stage localization has lower average errors in both position and orientation. On the other hand, Fig. 13(c) and Fig. 13(d) show the mean, minimum, and maximum values of docking errors in terms of position and orientation, respectively. Although overall performance is sufficient, it is seen that docking with high accuracy to a target may not be possible if a large-scale coarse localization approach is used along the entire route. However, with the aid of the switching mechanism, the use of large-scale localization in the delivery zone and precise positioning in the docking zone makes both having a sufficient localization performance overall and reaching a target at industrial standards possible.

4.5.3. Performance comparisons with the state-of-the-art

The results of field tests conducted to demonstrate the superiority of the developed two-stage localization approach over the single-level approach have been provided up to this point. In this section, on the other hand, the developed multi-stage localization method will be compared with some of the state-of-the-art methods in the literature. The proposed approach in this study has been compared with multi-stage localization approaches that combine MCL and entropy-based TrimICP proposed by Sun et al. [45] and the scan-matching-based refinement performed after particle filter proposed by Garrote et al. [54].

The results evaluating the overall localization performance of the methods are presented in Fig. 14, utilizing three cases previously used in the tests. The results in Fig. 14(b-d) clearly indicate that state-of-the-art multi-stage methods outperform in terms of localization when the coarse localization is active. However, as highlighted throughout the article, for the delivery stage, what matters significantly is the rough estimation of the pose and the secure transportation of the AMR to the docking area. In our approach developed, although the localization performance in the delivery region is relatively lower, it is sufficient to reach the docking region. Moreover, considering the computational costs shown in Table I, it can be observed that the localization process is three times faster in our developed method for the delivery stage.

Figure 14. The localization performance comparisons through three cases: (a) Cases (b) Case 1, (c) Case 2, (d) Case 3 results, and (e) Close-up view of Case 3 results around target pose.

Table I. Performance comparison with state-of-the-art methods in terms of fastness and computational load (PF: particle filter, SM: Scan matching, SW: Switching).

When specifically discussing the docking area, although the superior performance of our developed method in terms of localization might not be distinctly discernible from Fig. 14(e), it is clearly evident in Table II that our approach can estimate the position of the AMR with sub-centimeter accuracy and precision level. Furthermore, in the docking region, pose estimation frequency is higher in our method (see Table I). The primary reason for this lies in the fact that other methods require sequential computations of both particle filters and scan-matching approaches for pose estimation in the docking region. This brings additional computational load and allows our algorithm to achieve higher frequency in localization around the vicinity of the target pose since the computational burden of the switching decision process is negligible, whose operating frequency is around 300 Hz.

Table II. Performance comparison with state-of-the-art methods in terms of localization accuracy (PF: particle filter, SM: scan matching, SW: switching).

In summary, the proposed multi-stage localization framework in this study dynamically switches between particle filter-based localization and scan-matching-based localization approaches based on the prevailing conditions. A continuously operating switching mechanism in the background determines which localization system will be active. The computational load of the decision mechanism is relatively low, enabling faster localization predictions compared to other state-of-the-art methods. When considering all these results together, it can be inferred that the multi-stage localization method developed, which includes the correntropy-based switching mechanism, can be confidently utilized for the localization problem of AMRs used for logistics purposes in factory-like environments.

4.6. Brief discussion on achievements

In the preceding sections, we have presented a comprehensive multi-stage localization framework, addressing the challenges of large-scale navigation and precise docking for AMRs in smart factory environments. Our focus has been on achieving sub-centimeter precision during the docking phase, a critical requirement for the seamless integration of AMRs into dynamic production landscapes.

Now, turning our attention to potential avenues for further enhancement and exploration, we consider opportunities for refining the precise localization component. For the precise localization component, the implementation of the ICP algorithm with improved speed and the use of the SLAMICP library [66] for updating the map in response to changing environmental obstacles could be explored. This approach may enable an increase in the current localization frequency, which is approximately 6 Hz, to around 20 Hz.

Moreover, while this study concentrated on achieving accurate position data for the docking process, it is imperative to delve into the realm of precise path tracking within the docking zone. Particularly in dynamic environments, where docking maneuvers may vary, numerous studies have addressed the high-accuracy tracking of dynamically changing routes [67]. Considering these factors during the implementation of the methods derived from this study is crucial for optimizing overall performance.

5. Conclusions

Achieving high-accuracy docking is crucial for vehicles involved in logistics processes such as loading-unloading, reaching the charging station, and task sharing in factories. With the advancement of the fourth industrial revolution, there is a growing demand for achieving sub-centimeter docking accuracy autonomously. In this study, we introduced a multi-stage localization framework that uses particle filter-based localization in the delivery phase and scan-matching-based localization in the docking phase. The switching decision between stages is made upon a correntropy-based similarity metric since the similarity of the paired point sets directly affects the registration performance for scan-matching-based techniques. Owing to the developed method, only one localization approach is activated at any given moment, and due to the low computational load of the switching strategy, faster pose estimations are achievable compared to other methods in the literature. Performing localization based solely on the particle filter method in the delivery region leads to a higher localization error than other multi-stage methods in the literature. However, with the developed method, switching to the scan-matching approach is carried out in the regions where high accuracy is required, mitigating potential drawbacks. Field tests over three dedicated scenarios demonstrated that the proposed method allows safe movement toward the target and docking to it with sub-centimeter and sub-degree accuracy and precision. Thus, our study substantiates the effectiveness of the proposed method, representing a significant advancement in achieving high-accuracy autonomous docking in factory environments.

As autonomous vehicle technologies continue to develop, it is expected that 3D LiDAR technology will become more widespread in industrial settings. Although safety-critical certifications for the use of 3D LiDARs in factories are not yet available, it is anticipated that these sensors will be increasingly used during production processes in the near future. Therefore, promising avenues for future research could be extending the decision-making mechanism proposed in this study to systems equipped with 3D LiDARs and evaluating it in various challenging industrial settings and scenarios to assess their robustness, efficiency, and adaptability.