Language identification (LID) is a crucial preliminary process in the field of Automatic Speech Recognition (ASR) that involves the identification of a spoken language from audio samples. Contemporary systems that can process speech in multiple languages require users to expressly designate one or more languages prior to utilisation. The LID task assumes a significant role in scenarios where ASR systems are unable to comprehend the spoken language in multilingual settings, leading to unsuccessful speech recognition outcomes. The present study introduces convolutional recurrent neural network (CRNN)-based LID, designed to operate on the mel-frequency cepstral coefficient (MFCC) characteristics of audio samples. Furthermore, we replicate certain state-of-the-art methodologies, specifically the convolutional neural network (CNN) and Attention-based convolutional recurrent neural network (CRNN with Attention), and conduct a comparative analysis with our CRNN-based approach. We conducted comprehensive evaluations on thirteen distinct Indian languages, and our model resulted in over 98 per cent classification accuracy. The LID model exhibits high-performance levels ranging from 97 per cent to 100 per cent for languages that are linguistically similar. The proposed LID model exhibits a high degree of extensibility to additional languages and demonstrates a strong resistance to noise, achieving 91.2 per cent accuracy in a noisy setting when applied to a European Language (EU) dataset.

Deep learning (DL) has been widely used in bearing fault diagnosis. In particular, convolutional neural networks (CNNs) improve diagnosis accuracy by extracting excellent fault features. However, CNN lacks an explicit learning mechanism to distinguish between different fault characteristics in the input signal to the diagnosis results. This article presents a new end-to-end depth framework called multi-head self-attention convolution neural network (MSA-CNN) for bearing fault diagnosis. Firstly, we adopt a data pre-processing method that directly converts one-dimensional (1D) original signals into two-dimensional (2D) grayscale images, which is simple to implement and preserves the complete information of the original signal. Secondly, multi-head self-attention (MSA) is first constructed to aggregate the global information and adaptively assign weights to the input signal's features. Thirdly, the CNN with small-scale kernels extracted detailed local features. Finally, the learned high-level representations are fed into the full connect (FC) layer for fault diagnosis. The performance of the MSA-CNN is validated on different datasets. The results show that the proposed MSA-CNN can significantly improve fault diagnosis accuracy compared with the other state-of-the-art methods and has excellent noise immunity performance.

We present a data-driven emulator, a stochastic weather generator (SWG), suitable for estimating probabilities of prolonged heat waves in France and Scandinavia. This emulator is based on the method of analogs of circulation to which we add temperature and soil moisture as predictor fields. We train the emulator on an intermediate complexity climate model run and show that it is capable of predicting conditional probabilities (forecasting) of heat waves out of sample. Special attention is payed that this prediction is evaluated using a proper score appropriate for rare events. To accelerate the computation of analogs, dimensionality reduction techniques are applied and the performance is evaluated. The probabilistic prediction achieved with SWG is compared with the one achieved with a convolutional neural network (CNN). With the availability of hundreds of years of training data, CNNs perform better at the task of probabilistic prediction. In addition, we show that the SWG emulator trained on 80 years of data is capable of estimating extreme return times of order of thousands of years for heat waves longer than several days more precisely than the fit based on generalized extreme value distribution. Finally, the quality of its synthetic extreme teleconnection patterns obtained with SWG is studied. We showcase two examples of such synthetic teleconnection patterns for heat waves in France and Scandinavia that compare favorably to the very long climate model control run.

The seismic response of a wide variety of structures, from small but irreplaceable museum exhibits to large bridge systems, is characterized by rocking. In addition, rocking motion is increasingly being used as a seismic protective strategy to limit the amount of seismic actions (moments) developed at the base of structures. However, rocking is a highly nonlinear phenomenon governed by non-smooth dynamic phases that make its prediction difficult. This study presents an alternative approach to rocking estimation based on a physics-informed convolutional neural network (PICNN). By training a group of PICNNs using limited datasets obtained from numerical simulations and encoding the known physics into the PICNNs, important predictive benefits are obtained relieving difficulties associated with over-fitting and minimizing the requirement for a large training database. Two models are created depending on the validation of the deep PICNN: the first model assumes that state variables including rotations and angular velocities are available, while the second model is useful when only acceleration measurements are known. The analysis is initiated by implementing K-means clustering. This is followed by a detailed statistical assessment and a comparative analysis of the response-histories of a rocking block. It is observed that the deep PICNN is capable of effectively estimating the seismic rocking response history when the rigid block does not overturn.

Very metal-poor (VMP, [Fe/H]<-2.0) stars serve as invaluable repositories of insights into the nature and evolution of the first-generation stars formed in the early galaxy. The upcoming China Space Station Telescope (CSST) will provide us with a large amount of spectral data that may contain plenty of VMP stars, and thus it is crucial to determine the stellar atmospheric parameters ($T_{\textrm{eff}}$, $\log$ g, and [Fe/H]) for low-resolution spectra similar to the CSST spectra ($R\sim 200$). This study introduces a novel two-dimensional Convolutional Neural Network (CNN) model, comprised of three convolutional layers and two fully connected layers. The model’s proficiency is assessed in estimating stellar parameters, particularly metallicity, from low-resolution spectra ($R \sim 200$), with a specific focus on enhancing the search for VMP stars within the CSST spectral data. We mainly use 10 008 spectra of VMP stars from LAMOST DR3, and 16 638 spectra of non-VMP stars ([Fe/H]>-2.0) from LAMOST DR8 for the experiments and apply random forest and support vector machine methods to make comparisons. The resolution of all spectra is reduced to $R\sim200$ to match the resolution of the CSST, followed by pre-processing and transformation into two-dimensional spectra for input into the CNN model. The validation and practicality of this model are also tested on the MARCS synthetic spectra. The results show that using the CNN model constructed in this paper, we obtain Mean Absolute Error (MAE) values of 99.40 K for $T_{\textrm{eff}}$, 0.22 dex for $\log$ g, 0.14 dex for [Fe/H], and 0.26 dex for [C/Fe] on the test set. Besides, the CNN model can efficiently identify VMP stars with a precision rate of 94.77%, a recall rate of 93.73%, and an accuracy of 95.70%. This paper powerfully demonstrates the effectiveness of the proposed CNN model in estimating stellar parameters for low-resolution spectra ($R\sim200$) and recognizing VMP stars that are of interest for stellar population and galactic evolution work.

Human activity recognition (HAR) is an emerging challenge among researchers. HAR has many possible uses in various fields, including healthcare, sports, and security. Furthermore, there are only a few publicly accessible datasets for classifying and recognizing physical activity in the literature, and these datasets comprise fewer activities. We created and compared our dataset with available datasets, that is, NTU-RGBD, UP-FALL, UR-Fall, WISDM, and UCI HAR. The proposed dataset consists of seven activities: eating, exercise, handshake, situps, vomiting, headache, and walking. The activities were collected from 20 people between the ages of 25 and 40 years using Kinect V2 sensor at 30 FPS. For classification, we use deep learning architectures based on convolutional neural network (CNN) and long short-term memory (LSTM). Additionally, we developed a novel hybrid deep learning model by combining a CNN, a bidirectional LSTM unit, and a fully connected layer for activity identification. The suggested model builds unique guided features using the preprocessed skeleton coordinates and their distinctive geometrical and kinematic aspects. Results from the experiment are contrasted with the performance of stand-alone CNNs, LSTMs, and ConvLSTM. The proposed model’s accuracy of 99.5% surpasses that of CNN, LSTM, and ConvLSTM, which have accuracy rates of 95.76%, 97%, and 98.89%, respectively. Our proposed technique is invariant of stance, speed, individual, clothes, etc. The proposed dataset sample is accessible to the general public.

Radar-based hand gesture recognition is a potential noncontact human–machine interaction technique. To enhance the recognition performance of hand gesture, a multidomain fusion-based recognition method using frequency-modulated continuous wave radar is proposed in this article. The received raw echo data of gestures is preprocessed to obtain the range–time matrix, Doppler–time matrix, and range–Doppler–frame tensor. The obtained three-domain radar data corresponding to each gesture are input into the three-channel convolutional neural network for feature extraction. In particular, the extracted features from three-domain data are fused with learnable weight matrices to obtain the final gesture classification results. The experimental results have shown that the classification accuracy of the proposed multidomain fusion network based on learning weight matrix-based fusion is 98.45%, which improves the classification performance compared with the classic average-based fusion and concatenation fusion.

Weeds contribute to biodiversity and a wide range of ecosystem functions. It is crucial to map different weed species and analyze their physiological activities. Remote sensing techniques for plant identification, especially hyperspectral imaging, are being developed using spectral response patterns to vegetation for detection and species identification. A library of hyperspectral images of 40 urban weed species in northeast China was established in this study. A terrestrial hyperspectral camera was used to acquire 435 hyperspectral images. The hyperspectral information for each weed species was extracted and analyzed. The spectral characteristics and vegetation indices of different weeds revealed the differences between weed species in the cities of northeast China and indirectly characterized the growth and physiological activity levels of different species, but could not effectively distinguish different species. Five methods—first derivative spectrum (FDS), second derivative spectrum (SDS), standard normal variate (SNV), moving averages (MA), and Savitzky-Golay (SG) smoothing—were used to pretreat the spectral curves to maximize the retention of spectral characteristics while removing the influence of noise. We investigated the application of a convolutional neural network (CNN) with terrestrial hyperspectral remote sensing to identify urban weeds in northeast China. A CNN classification model was established to distinguish weeds from the hyperspectral images and demonstrated a test accuracy of 95.32% to 98.15%. The accuracy of the original spectrum was 97.45%; SNV had the best accuracy (98.15%) and SG was the least accurate (95.32%). This provides a baseline for understanding the hyperspectral characteristics of urban weed species and monitoring their growth. It also contributes to the development of a hyperspectral imaging database with global applicability.

The goal of few-shot semantic segmentation is to learn a segmentation model that can segment novel classes in queries when only a few annotated support examples are available. Due to large intra-class variations, the building of accurate semantic correlation remains a challenging job. Current methods typically use 4D kernels to learn the semantic correlation of feature maps. However, they still face the challenge of reducing the consumption of computation and memory while keeping the availability of correlations mined by their methods. In this paper, we propose the adaptively mining correlation network (AMCNet) to alleviate the aforementioned issues. The key points of AMCNet are the proposed adaptive separable 4D kernel and the learnable pyramid correlation module, which form the basic block for correlation encoder and provide a learnable concatenation operation over pyramid correlation tensors, respectively. Experiments on the PASCAL VOC 2012 dataset show that our AMCNet surpasses the state-of-the-art method by $0.7\%$ and $2.2\%$ on 1-shot and 5-shot segmentation scenarios, respectively.

In laparoscopic surgery, surgeons should develop several manual laparoscopic skills before carrying out real operative procedures using a low-cost box trainer. The Fundamentals of Laparoscopic Surgery (FLS) program was developed as a program to assess fundamental knowledge and surgical skills, required for basic laparoscopic surgery. The peg transfer task is a hands-on exam in the FLS program that assists a trainee to understand the relative minimum amount of grasping force necessary to move the pegs from one place to another place without dropping them. In this paper, an autonomous, sequential assessment algorithm based on deep learning, a multi-object detection method, and, several sequential If-Then conditional statements have been developed to monitor each step of a surgeon’s performance. Images from three different cameras are used to assess whether the surgeon executes the peg transfer task correctly and to display a notification on any errors on the monitor immediately. This algorithm improves the performance of a laparoscopic box-trainer system using top, side, and front cameras and removes the need for any human monitoring during a peg transfer task. The developed algorithm can detect each object and its status during a peg transfer task and notifies the resident about the correct or failed outcome. In addition, this system can correctly determine the peg transfer execution time, and the move, carry, and dropped states for each object by the top, side, and front-mounted cameras. Based on the experimental results, the proposed surgical skill assessment system can identify each object at a high score of fidelity, and the train-validation total loss for the single-shot detector (SSD) ResNet50 v1 was about 0.05. Also, the mean average precision (mAP) and Intersection over Union (IoU) of this detection system were 0.741, and 0.75, respectively. This project is a collaborative research effort between the Department of Electrical and Computer Engineering and the Department of Surgery, at Western Michigan University.

Twelve climate models and observations are used to attribute the global mean surface temperature (GMST) changes from 1900 to 2014 to external climate forcings. The external forcings are decomposed into the effects of the well-mixed greenhouse gas concentration variation, the effects of anthropogenic aerosol concentration changes, and the effects of natural forcings. First, a convolutional neural network (CNN) is trained to estimate the simulated historical GMST from single-forcing experiments using outputs from the multi-model ensemble. We then use this CNN to solve the attribution problem using an original variational inversion approach. The variational inversion is first validated using historical climate simulations as pseudo-observations. Then we perform an inversion from observations. This provides a distribution of the GMST resulting from the three forcings. For 2014, inversions estimate that the greenhouse gases changes are responsible for a GMST anomaly within [0.8 $ {}^{\circ } $ C,1.9 $ {}^{\circ } $ C], while anthropogenic aerosols and natural forcings anomalies are within [−0.7 $ {}^{\circ } $ C,−0.1 $ {}^{\circ } $ C] and [−0.1 $ {}^{\circ } $ C,0.3 $ {}^{\circ } $ C], respectively. The method designed here can be adapted and extended to attribute the changes of other variables or to focus on the regional scale.

Robotic assistive or rehabilitative devices are promising aids for people with neurological disorders as they help regain normative functions for both upper and lower limbs. However, it remains challenging to accurately estimate human intent or residual efforts non-invasively when using these robotic devices. In this article, we propose a deep learning approach that uses a brightness mode, that is, B-mode, of ultrasound (US) imaging from skeletal muscles to predict the ankle joint net plantarflexion moment while walking. The designed structure of customized deep convolutional neural networks (CNNs) guarantees the convergence and robustness of the deep learning approach. We investigated the influence of the US imaging’s region of interest (ROI) on the net plantarflexion moment prediction performance. We also compared the CNN-based moment prediction performance utilizing B-mode US and sEMG spectrum imaging with the same ROI size. Experimental results from eight young participants walking on a treadmill at multiple speeds verified an improved accuracy by using the proposed US imaging + deep learning approach for net joint moment prediction. With the same CNN structure, compared to the prediction performance by using sEMG spectrum imaging, US imaging significantly reduced the normalized prediction root mean square error by 37.55% ($ p $ < .001) and increased the prediction coefficient of determination by 20.13% ($ p $ < .001). The findings show that the US imaging + deep learning approach personalizes the assessment of human joint voluntary effort, which can be incorporated with assistive or rehabilitative devices to improve clinical performance based on the assist-as-needed control strategy.

Reproducibility of a deep-learning fully convolutional neural network is evaluated by training several times the same network on identical conditions (database, hyperparameters, and hardware) with nondeterministic graphics processing unit operations. The network is trained to model three typical time–space-evolving physical systems in two dimensions: heat, Burgers’, and wave equations. The behavior of the networks is evaluated on both recursive and nonrecursive tasks. Significant changes in models’ properties (weights and feature fields) are observed. When tested on various benchmarks, these models systematically return estimations with a high level of deviation, especially for the recurrent analysis which strongly amplifies variability due to the nondeterminism. Trainings performed with double floating-point precision provide slightly better estimations and a significant reduction of the variability of both the network parameters and its testing error range.

The capabilities of teleoperated robots can be enhanced with the ability to recognise and reproduce human-like behaviour. The proposed framework presents motion recognition for a Kinect-based NAO teleoperation. It allows the NAO robot to recognise the human motions and act as a human motion imitator. A stable whole-body imitation is still a challenging issue because of the difficulty in dynamic balancing of centre of mass (CoM). In this paper, a novel adaptive balancing technique for NAO (ABTN) is proposed to control the whole body in single as well as double supporting phases. It targets dynamic balancing of the humanoid robot by solving forward kinematics and applying a weighted average of mass with the CoMs of individual links with respect to the previous joint frames, which provides us with the dynamic CoM of the whole body. Our novel approach uses this dynamic CoM and calculates joint angles using proposed pitch and roll control algorithm to keep the dynamic CoM inside the stable region. Additionally, the NAO robot is capable of recognising human motions using the proposed 7-layer one-dimensional convolutional neural network (1D-CNN). To solve the problem of variable length of time sequences, Zero padding is introduced with 1D-CNN. It attains a recognition accuracy of 95% as compared to the hidden Markov model and neural network. The experimental results demonstrate that the developed teleoperation framework is robust and serves as potential support for the development and application of teleoperated robots.

Magnesium ions (Mg2+) are vital for RNA structure and cellular functions. Present efforts in RNA structure determination and understanding of RNA functions are hampered by the inability to accurately locate Mg2+ ions in an RNA. Here we present a machine-learning method, originally developed for computer visual recognition, to predict Mg2+ binding sites in RNA molecules. By incorporating geometrical and electrostatic features of RNA, we capture the key ingredients of Mg2+-RNA interactions, and from deep learning, predict the Mg2+ density distribution. Five-fold cross-validation on a dataset of 177 selected Mg2+-containing structures and comparisons with different methods validate the approach. This new approach predicts Mg2+ binding sites with notably higher accuracy and efficiency. More importantly, saliency analysis for eight different Mg2+ binding motifs indicates that the model can reveal critical coordinating atoms for Mg2+ ions and ion-RNA inner/outer-sphere coordination. Furthermore, implementation of the model uncovers new Mg2+ binding motifs. This new approach may be combined with X-ray crystallography structure determination to pinpoint the metal ion binding sites.

A deep convolutional neural network has been developed to denoise atomic-resolution transmission electron microscope image datasets of nanoparticles acquired using direct electron counting detectors, for applications where the image signal is severely limited by shot noise. The network was applied to a model system of CeO2-supported Pt nanoparticles. We leverage multislice image simulations to generate a large and flexible dataset for training the network. The proposed network outperforms state-of-the-art denoising methods on both simulated and experimental test data. Factors contributing to the performance are identified, including (a) the geometry of the images used during training and (b) the size of the network's receptive field. Through a gradient-based analysis, we investigate the mechanisms learned by the network to denoise experimental images. This shows that the network exploits both extended and local information in the noisy measurements, for example, by adapting its filtering approach when it encounters atomic-level defects at the nanoparticle surface. Extensive analysis has been done to characterize the network's ability to correctly predict the exact atomic structure at the nanoparticle surface. Finally, we develop an approach based on the log-likelihood ratio test that provides a quantitative measure of the agreement between the noisy observation and the atomic-level structure in the network-denoised image.

Achieving robust and fast two-dimensional adaptive beamforming of phased array antennas is a challenging problem due to its high-computational complexity. To address this problem, a deep-learning-based beamforming method is presented in this paper. In particular, the optimum weight vector is computed by modeling the problem as a convolutional neural network (CNN), which is trained with I/O pairs obtained from the optimum Wiener solution. In order to exhibit the robustness of the new technique, it is applied on an 8 × 8 phased array antenna and compared with a shallow (non-deep) neural network namely, radial basis function neural network. The results reveal that the CNN leads to nearly optimal Wiener weights even in the presence of array imperfections.