This study focuses on the kinematic and dynamic modeling of a wheeled-legged robot (WLR), taking into account kinematic and dynamic slippage. In this regard, the Gibbs–Appell formulation was utilized to derive dynamic equations. Determining the slippage in the wheels for movement equations is a challenging task due to its dependency on factors such as the robot’s postures, velocities, and surface characteristics. To address this challenge, machine vision was used to quantify the slippage of the wheels on the body based on the pose estimation method. This data served as input for movement equations to analyze the robot’s deviation from its path and posture. In the following, the robot’s movement was simulated using Webots and MATLAB, followed by various experimental tests involving acceleration and changes in leg angles on the WLR. The results were then compared to the simulations to demonstrate the accuracy of the developed system modeling. Additionally, an IMU sensor was utilized to measure the robot’s motion and validate against machine vision data. The findings revealed that neglecting the slippage of the wheels in the robot’s motion modeling resulted in errors ranging from 5% to 11.5%. Furthermore, lateral slippage ranging from 1.1 to 5.2 cm was observed in the robot’s accelerated movement. This highlights the importance of including lateral slippage in the equations for a more precise modeling of the robot’s behavior.

As a kind of lower-limb motor assistance device, the intelligent walking aid robot plays an essential role in helping people with lower-limb diseases to carry out rehabilitation walking training. In order to enhance the safety performance of the lower-limb walking aid robot, this study proposes a deep vision-based abnormal lower-limb gait prediction model construction method for the problem of abnormal gait prediction of patients’ lower limbs. The point cloud depth vision technique is used to acquire lower-limb motion data, and a multi-posture angular prediction model is trained using long and short-term memory networks to build a model of the user’s lower-limb posture characteristics during normal walking as well as a real-time lower-limb motion prediction model. The experimental results indicate that the proposed lower-limb abnormal behavior prediction model is able to achieve a 97.4% prediction rate of abnormal lower-limb movements within 150 ms. Additionally, the model demonstrates strong generalization ability in practical applications. This paper proposes further ideas to enhance the safety performance of lower-limb rehabilitation robot use for patients with lower-limb disabilities.

Machine vision has been extensively researched in the field of unmanned aerial vehicles (UAV) recently. However, the ability of Sense and Avoid (SAA) largely limited by environmental visibility, which brings hazards to flight safety in low illumination or nighttime conditions. In order to solve this critical problem, an approach of image enhancement is proposed in this paper to improve image qualities in low illumination conditions. Considering the complementarity of visible and infrared images, a visible and infrared image fusion method based on convolutional sparse representation (CSR) is a promising solution to improve the SAA ability of UAVs. Firstly, the source image is decomposed into a texture layer and structure layer since infrared images are good at characterising structural information, and visible images have richer texture information. Both the structure and the texture layers are transformed into the sparse convolutional domain through the CSR mechanism, and then CSR coefficient mapping are fused via activity level assessment. Finally, the image is synthesised through the reconstruction results of the fusion texture and structure layers. In the experimental simulation section, a series of visible and infrared registered images including aerial targets are adopted to evaluate the proposed algorithm. Experimental results demonstrates that the proposed method increases image qualities in low illumination conditions effectively and can enhance the object details, which has better performance than traditional methods.

Welding is a complex manufacturing process. Its quality depends on the welder skills, especially in welding complex paths. For consistency in modern industries, the arm robot is used to accomplish this task. However, its programming and reprogramming are time consuming and costly and need an expert programmer. These limit the use of robots in medium and small industries. This paper introduces a new supervised learning technique for programming a 4-degree of freedom (DOF) welding arm robot with an automatic feeding electrode. This technique is based on grasping the welding path control points and motion behavior of an expert welder. This is achieved by letting the welder move the robot end effector, which represents the welding torch, through the welding path. At the path control points, the position and speed are recorded using a vision system. Later, these data are retrieved by the robot to replicate the welding path. Several 2D paths are tested to assess the proposed approach accuracy and programming time and easiness in comparison with the common one. The results prove that the proposed approach includes fewer steps and consumes less programming time. Moreover, programming can be accomplished by the welder and no need for an expert programmer. These enhancements will improve the share of robots in welding and similar industries.

Spot spraying POST herbicides is an effective approach to reduce herbicide input and weed control cost. Machine vision detection of grass or grass-like weeds in turfgrass systems is a challenging task due to the similarity in plant morphology. In this work, we explored the feasibility of using image classification with deep convolutional neural networks (DCNN), including AlexNet, GoogLeNet, and VGGNet, for detection of crabgrass species (Digitaria spp.), doveweed [Murdannia nudiflora (L.) Brenan], dallisgrass (Paspalum dilatatum Poir.), and tropical signalgrass [Urochloa distachya (L.) T.Q. Nguyen] in bermudagrass [Cynodon dactylon (L.) Pers.]. VGGNet generally outperformed AlexNet and GoogLeNet in detecting selected grassy weeds. For detection of P. dilatatum, VGGNet achieved high F1 scores (≥0.97) and recall values (≥0.99). A single VGGNet model exhibited high F1 scores (≥0.93) and recall values (1.00) that reliably detected Digitaria spp., M. nudiflora, P. dilatatum, and U. distachya. Low weed density reduced the recall values of AlexNet at detecting all weed species and GoogLeNet at detecting Digitaria spp. In comparison, VGGNet achieved excellent performances (overall accuracy = 1.00) at detecting all weed species in both high and low weed-density scenarios. These results demonstrate the feasibility of using DCNN for detection of grass or grass-like weeds in turfgrass systems.

Military applications of technologies for enhancing or producing vision play a key role in composing contemporary security, as such technologies are deployed to make security sense of everyday sociality, of battlefields, and of much in between these extremes. In this article, I set out to recompose militarised techno-vision through the public detritus left by its heterogenous development, use, and appropriation. I argue that as an heterogenous and amalgamated object, military techno-vision can be composed by speaking the stories of its leftovers, and that this composition is characterised by and in turn characterises a longstanding dilemma between fact and vision – between the ambiguity that is constitutive of the human practices of visual perception and image-making, and the desire for machines that can produce visual ‘actionable intelligence’ that can underpin security decisions. Discourses, practices, and regimes of visibility are deployed alongside technologies to occlude the ambiguity of technological vision and sustain the imaginary of technologically altered vision as neutral production of military or security facts.

Weed interference during crop establishment is a serious concern for Florida strawberry [Fragaria×ananassa (Weston) Duchesne ex Rozier (pro sp.) [chiloensis×virginiana]] producers. In situ remote detection for precision herbicide application reduces both the risk of crop injury and herbicide inputs. Carolina geranium (Geranium carolinianum L.) is a widespread broadleaf weed within Florida strawberry production with sensitivity to clopyralid, the only available POST broadleaf herbicide. Geranium carolinianum leaf structure is distinct from that of the strawberry plant, which makes it an ideal candidate for pattern recognition in digital images via convolutional neural networks (CNNs). The study objective was to assess the precision of three CNNs in detecting G. carolinianum. Images of G. carolinianum growing in competition with strawberry were gathered at four sites in Hillsborough County, FL. Three CNNs were compared, including object detection–based DetectNet, image classification–based VGGNet, and GoogLeNet. Two DetectNet networks were trained to detect either leaves or canopies of G. carolinianum. Image classification using GoogLeNet and VGGNet was largely unsuccessful during validation with whole images (Fscore<0.02). CNN training using cropped images increased G. carolinianum detection during validation for VGGNet (Fscore=0.77) and GoogLeNet (Fscore=0.62). The G. carolinianum leaf–trained DetectNet achieved the highest Fscore (0.94) for plant detection during validation. Leaf-based detection led to more consistent detection of G. carolinianum within the strawberry canopy and reduced recall-related errors encountered in canopy-based training. The smaller target of leaf-based DetectNet did increase false positives, but such errors can be overcome with additional training images for network desensitization training. DetectNet was the most viable CNN tested for image-based remote sensing of G. carolinianum in competition with strawberry. Future research will identify the optimal approach for in situ detection and integrate the detection technology with a precision sprayer.

The practical application of yield loss prediction models using relative leaf area of weeds is limited due to the lack of a quick and accurate method of leaf area estimation. Leaf cover (the vertical projection of plant canopy on the ground) can be used to approximate leaf area at early stages of plant development. An automated digital image analysis system for measuring leaf cover has been developed. The system has an operator-assisted module aimed at validating the automated functions. The objective of this research was to demonstrate the accuracy of the operator-assisted module under different weed–crop conditions. A laboratory experiment was conducted using simulated weed–crop populations. Two additional field experiments were conducted using corn in competition with: (1) common lambsquarters, barnyardgrass, or a mixture of both species, and (2) a natural weed community. In the laboratory experiment, a narrow linear relation was observed between leaf cover estimated with the operator-assisted module and leaf area measured with an optical area meter (r 2 > 0.98). In field experiments, the regression between corn leaf cover estimated by the operator-assisted module and corn leaf area measured with the optical area meter was not as good (r 2 < 0.55). The poor performance of the module was probably due to the overlapping and the architecture of corn leaves (especially unexpanded leaves). Nevertheless, the system showed high precision in estimating leaf area of both grassy weeds and broadleaf weeds (r 2 > 0.89). Generally, the accuracy of the estimates decreased as the growth stage became more advanced. Apart from its initial purpose as a calibration tool for the automated system, the operator-assisted module can have several potential research applications. It can be used: (1) as an alternative to destructive leaf area measurement at early stages of plant development, (2) as a tool in the study of plant competitive ability, and (3) as an objective and quantitative support to visual observations.

Multispectral images of leaf reflectance in the visible and near infrared region from 384 to 810 nm were used to establish the feasibility of developing a site-specific classifier to distinguish lettuce plants from weeds in California direct-seeded lettuce fields. An average crop vs. weed classification accuracy of 90.3% was obtained in a study of over 7,000 individual spectra representing 150 plants. The classifier utilized reflectance values from a small spatial area (3 mm diameter) of the leaf in order to allow the method to be robust to occlusion and to eliminate the need to identify leaf boundaries for shape-based machine vision recognition. Reflectance spectra were collected in the field using equipment suitable for real-time operation as a weed sensor in an autonomous system for automated weed control.

Automated detection and identification of weeds in crop fields is the greatest obstacle to development of practical site-specific weed management systems. Research progress is summarized for two different approaches to the problem, remote sensing weed mapping and ground-based detection using digital cameras or nonimaging sensors. The general spectral and spatial limitations reported for each type of weed identification system are reviewed. Airborne remote sensing has been successful for detection of distinct weed patches when the patches are dense and uniform and have unique spectral characteristics. Identification of weeds is hampered by spectral mixing in the relatively large pixels (typically larger than 1 by 1 m) and will not be possible from imagery where weed seedlings are sparsely distributed among crop plants. The use of multispectral imaging sensors such as color digital cameras on a ground-based mobile platform shows more promise for weed identification in field crops. Spectral features plus spatial features such as leaf shape and texture and plant organization may be extracted from these images. However, there is a need for research in areas such as artificial lighting, spectral band requirements, image processing, multiple spatial resolution systems, and multiperspective images.

Prawns are a significant marine resource in Australia and elsewhere, but their processing after the harvest is a very labour-intensive operation. Every prawn is presently hand packed. The possibility of utilising machine vision and robots to automate this operation has been investigated. Experiments indicate that machine vision can classify prawns into mass-related categories better than people now do, and that prawn orientation can be determined by this means. Although at the present state of technology it appears infeasible for robots to make up the multi-layer fixed-weight packs of prawns as is now done manually, a change to single-layer variable-weight packs should allow this process to be automated and would offer significant advantages to the Australian industry.

A prototype robot for picking citrus is described which utilized real-time, color machine vision to vision-servo the robot on a targeted fruit. A programming technique is presented which simplified development of the task-level, robot control program. An economic evaluation of robotic harvesting in Florida determined that robotic harvesting would be approximately 50 percent more expensive than conventional hand harvesting. Harvest inefficiency was identified as the most influential factor affecting robotic harvesting economics. Field trial performance is summarized and the potential of commercial robotic harvesting discussed.

This R&D project implements a new way of perception of the three-dimensional surrounding space, based exclusively in sounds and thus especially useful for the blind. The innate capability of locating sounds, the externalization of sounds played with headphones and the machine capture of the 3D environment are the technological pillars used for this purpose. They are analysed and a summary of their main requirements are presented. A number of laboratory facilities and portable prototypes are described, together with their main characteristics.

Vision-aided flexible link robot positoning using the Camera Space Manipulation (CSM) method is developed. The primary motivation for this work is to use an autonomous vision-aided robotic system to pick-up and accurately move a flexible object that it encounters. The work consists of analytical and experimental investigation of the performance of CSM for a kinematic model of the PUMA manipulator with a flexible structure at the wrist which accounts for the gravitation. Trade-offs between camera view parameters and axial deflection model parameters were investigated. View parameter reestimation and maneuvering resulted a very accurate placement of the end-effector at the target.

Machine vision-based sensing enables automatic hover stabilization of helicopters. The evaluation of image data, which is produced by a camera looking straight to the ground, results in a drift free autonomous on-board position measurement system. No additional information about the appearance of the scenery seen by the camera (e.g. landmarks) is needed. The technique being applied is a combination of the 4D-approach with two dimensional template tracking of a priori unknown features.

This paper reports a visual tracking system that can track moving objects in real-time with a modest workstation equipped with a pan-tilt device. The algorithm essentially has three parts: (1) feature detection, (2) tracking and (3) control of the robot head. Corners are viewpoint invariant, hence being utilised as the beacon for tracking. Tracking is performed in two stages of Kalman filtering and affine transformation. A technique of reducing greatly the computational time for the correlaton is also described. The Kalman filter predicts intelligently the fovea window and reduced computation dramatically. The affine transformation deals with the unexpected events when there is partial occlusion.

In machine vision, objects are observed subject to an unknown projective transformation, and it is usual to use projective invariants for either testing for a false alarm or for classifying an object. For four collinear points, the cross-ratio is the simplest statistic which is invariant under projective transformations. We obtain the distribution of the cross-ratio under the Gaussian error model with different means. The case of identical means, which has appeared previously in the literature, is derived as a particular case. Various alternative forms of the cross-ratio density are obtained, e.g. under the Casey arccos transformation, and under an arctan transformation from the real projective line of cross-ratios to the unit circle. The cross-ratio distributions are novel to the probability literature; surprisingly various types of Cauchy distribution appear. To gain some analytical insight into the distribution, a simple linear-ratio is also introduced. We also give some results for the projective invariants of five coplanar points. We discuss the general moment properties of the cross-ratio, and consider some inference problems, including maximum likelihood estimation of the parameters.