Supervised deep learning approaches can artificially increase the resolution of microscopy images by learning a mapping between two image resolutions or modalities. However, such methods often require a large set of hard-to-get low-res/high-res image pairs and produce synthetic images with a moderate increase in resolution. Conversely, recent methods based on generative adversarial network (GAN) latent search offered a drastic increase in resolution without the need of paired images. However, they offer limited reconstruction of the high-resolution (HR) image interpretable features. Here, we propose a robust super-resolution (SR) method based on regularized latent search (RLS) that offers an actionable balance between fidelity to the ground truth (GT) and realism of the recovered image given a distribution prior. The latter allows to split the analysis of a low-resolution (LR) image into a computational SR task performed by deep learning followed by a quantification task performed by a handcrafted algorithm based on interpretable biological features. This two-step process holds potential for various applications such as diagnostics on mobile devices, where the main aim is not to recover the HR details of a specific sample but rather to obtain HR images that preserve explainable and quantifiable differences between conditions.

Coarse spatial resolution in gridded precipitation datasets, reanalysis, and climate model outputs restricts their ability to characterize the localized extreme rain events and limits the use of the coarse resolution information for local to regional scale climate management strategies. Deep learning models have recently been developed to rapidly downscale the coarse resolution precipitation to the high local scale resolution at a much lower cost than dynamic downscaling. However, these existing super-resolution deep learning modeling studies have not rigorously evaluated the model’s skill in producing fine-scale spatial variability, particularly over topographic features. These current deep-learning models also have difficulty predicting the complex spatial structure of extreme events. Here, we develop a model based on super-resolution deconvolution neural network (SRDN) to downscale the hourly precipitation and evaluate the predictions. We apply three versions of the SRDN model: (a) SRDN (no orography), (b) SRDN-O (orography only at final resolution enhancement), and (c) SRDN-SO (orography at each step of resolution enhancement). We assess the ability of SRDN-based models to reproduce the fine-scale spatial variability and compare it with the previously used deep learning model (DeepSD). All the models are trained and tested using the Conformal Cubic Atmospheric Model (CCAM) data to perform a 100 to 12.5 km of hourly precipitation downscaling over the Australian region. We found that SRDN-based models, including orography, deliver better fine-scale spatial structures of both climatology and extremes, and significantly improved the deep-learning downscaling. The SRDN-SO model performs well both qualitatively and quantitatively in reconstructing the fine-scale spatial variability of climatology and rainfall extremes over complex orographic regions.

Representation learning has proven to be a powerful methodology in a wide variety of machine-learning applications. For atmospheric dynamics, however, it has so far not been considered, arguably due to the lack of large-scale, labeled datasets that could be used for training. In this work, we show how to sidestep the difficulty and introduce a self-supervised learning task that is applicable to a wide variety of unlabeled atmospheric datasets. Specifically, we train a neural network on the simple yet intricate task of predicting the temporal distance between atmospheric fields from distinct but nearby times. We demonstrate that training with this task on the ERA5 reanalysis dataset leads to internal representations that capture intrinsic aspects of atmospheric dynamics. For example, when employed as a loss function in other machine-learning applications, the derived AtmoDist distance leads to improved results compared to the $ {\mathrm{\ell}}_2 $-loss. For downscaling one obtains higher resolution fields that match the true statistics more closely than previous approaches and for the interpolation of missing or occluded data the AtmoDist distance leads to results that contain more realistic fine-scale features. Since it is obtained from observational data, AtmoDist also provides a novel perspective on atmospheric predictability.

A common task in Earth Sciences is to infer climate information at local and regional scales from global climate models. Dynamical downscaling requires running expensive numerical models at high resolution, which can be prohibitive due to long model runtimes. On the other hand, statistical downscaling techniques present an alternative approach for learning links between the large- and local-scale climate in a more efficient way. A large number of deep neural network-based approaches for statistical downscaling have been proposed in recent years, mostly based on convolutional architectures developed for computer vision and super-resolution tasks. This paper presents deep learning for empirical downscaling (DL4DS), a python library that implements a wide variety of state-of-the-art and novel algorithms for downscaling gridded Earth Science data with deep neural networks. DL4DS has been designed with the goal of providing a general framework for training convolutional neural networks with configurable architectures and learning strategies to facilitate the conduction of comparative and ablation studies in a robust way. We showcase the capabilities of DL4DS on air quality Copernicus Atmosphere Monitoring Service (CAMS) data over the western Mediterranean area. The DL4DS library can be found in this repository: https://github.com/carlos-gg/dl4ds

The oceans have a very important role in climate regulation due to their massive heat storage capacity. Thus, for the past decades, oceans have been observed by satellites to better understand their dynamics. Satellites retrieve several data with various spatial resolutions. For instance, sea surface height (SSH) is a low-resolution data field where sea surface temperature (SST) can be retrieved in a much higher one. These two physical parameters are linked by a physical link that can be learned by a super-resolution machine-learning algorithm. In this work, we present a subpixel convolutional deep learning model that takes advantage of the higher resolution SST field to guide the downscaling of the SSH one. The data fields that we use are simulated by a physic-based ocean model at a higher sampling rate than the satellites provide. We compared our approach with a convolutional neural network model. Our architecture generalized well with validation performances of 3.94 cm root mean squared error (RMSE) and training performances of 2.65 cm RMSE.

In the twenty-first century, machine learning and deep learning have been successfully used to find hidden information from coarse-grained data in various domains. In Computer Vision, scientists have used neural networks to identify hidden pixel-level information from low-resolution (LR) image data. This approach of estimating high-resolution (HR) information from LR data is called the super-resolution (SR) approach. This approach has been borrowed by climate scientists to downscale coarse-level measurements of climate variables to obtain their local-scale projections. Climate variables are spatial in nature and can be represented as images where each pixel denotes a grid point where the variables can be measured. We can apply the deep learning-based SR techniques on such “images” for statistical downscaling of such variables. This approach of downscaling can be termed as deep downscaling. In this work, we have tried to make HR projection of the Indian summer monsoon rainfall by using a novel deep residual network called ResDeepD. The aim is to downscale the 10 × 10 low LR precipitation data to get the values at 0.250 × 0.250 resolution. The proposed model uses a series of skip connections across residual blocks to give better results as compared to the existing models like super-resolution convolutional neural network, DeepSD, and Nest-UNet that have been used previously for this task. We have also examined the model’s performance for downscaling rainfall during some extreme climatic events like cyclonic storms and deep depression and found that the model performs better than the existing models.

To overcome the physical barriers caused by light diffraction, super-resolution techniques are often applied in fluorescence microscopy. State-of-the-art approaches require specific and often demanding acquisition conditions to achieve adequate levels of both spatial and temporal resolution. Analyzing the stochastic fluctuations of the fluorescent molecules provides a solution to the aforementioned limitations, as sufficiently high spatio-temporal resolution for live-cell imaging can be achieved using common microscopes and conventional fluorescent dyes. Based on this idea, we present COL0RME, a method for covariance-based $ {\mathrm{\ell}}_0 $ super-resolution microscopy with intensity estimation, which achieves good spatio-temporal resolution by solving a sparse optimization problem in the covariance domain and discuss automatic parameter selection strategies. The method is composed of two steps: the former where both the emitters’ independence and the sparse distribution of the fluorescent molecules are exploited to provide an accurate localization; the latter where real intensity values are estimated given the computed support. The paper is furnished with several numerical results both on synthetic and real fluorescence microscopy images and several comparisons with state-of-the art approaches are provided. Our results show that COL0RME outperforms competing methods exploiting analogously temporal fluctuations; in particular, it achieves better localization, reduces background artifacts, and avoids fine parameter tuning.

Single-molecule localization microscopy (SMLM) is a family of super-resolution microscopy techniques based on localizing clusters of detected photons that are emitted by single molecules. The localization procedure is based on careful statistical analysis of long image sequences to derive the nanometer positions of the molecules. By introducing additional optics, such as cylindrical lenses in the optical system, SMLM techniques have been extended to 3D super-resolution imaging. This adds a calibration step, thereby further complicating the data analysis. Here we present Huygens Localizer, a well-supported user-friendly package that carries out these tasks quickly by offloading carefully designed 2D and 3D analysis and visualization procedures to massively parallel graphical processors (GPUs).

To use VRI systems, a field is divided into irrigation management zones (IMZs). While IMZs are dynamic in nature, most of IMZs prescription maps are static. High-resolution thermal images (TI) coupled with measured atmospheric conditions have been utilized to map the within-field water status variability and to delineate in-season IMZs. Unfortunately, spaceborne TIs have coarse spatial resolution and aerial platforms require substantial financial investments, which may inhibit their large-scale adoption. Three approaches are proposed to facilitate large-scale adoption of TI-based IMZs: 1) increase of the capacity of aerial TI by enhancing their spatial resolution; 2) sharpening the spatial resolution of satellite TI by fusing satellite multi-spectral images in the visible-near-infrared (VIS-NIR) range; 3) increase the capacity of aerial TI by fusing satellite multi-spectral images in the VIS-NIR range. The scientific and engineering basis of each of the approaches is described together with initial results.

Aberrant aggregation of improperly folded proteins is the hallmark of several human neurodegenerative disorders, including Huntington's Disease (HD) with autosomal-dominant inheritance. In HD, expansion of the CAG-repeat-encoded polyglutamine (polyQ) stretch beyond ~40 glutamines in huntingtin (Htt) and its N-terminal fragments leads to the formation of large (up to several μm) globular neuronal inclusion bodies (IBs) over time. We report direct observations of aggregating Htt exon 1 in living and fixed cells at enhanced spatial resolution by stimulated emission depletion (STED) microscopy and single-molecule super-resolution optical imaging. Fibrils of Htt exon 1 arise abundantly across the cytosolic compartment and also in neuritic processes only after nucleation and aggregation into a fairly advanced stage of growth of the prominent IB have taken place. Structural characterizations of fibrils by STED show a distinct length cutoff at ~1·5 µm and reveal subsequent coalescence (bundling/piling). Cytosolic fibrils are observed even at late stages in the process, side-by-side with the mature IB. Htt sequestration into the IB, which in neurons has been argued to be a cell-protective phenomenon, thus appears to saturate and over-power the cellular degradation systems and leaves cells vulnerable to further aggregation producing much smaller, potentially toxic, conformational protein species of which the fibrils may be comprised. We further found that exogenous delivery of the apical domain of the chaperonin subunit CCT1 to the cells via the cell medium reduced the aggregation propensity of mutant Htt exon 1 in general, and strongly reduced the occurrence of such late-stage fibrils in particular.

For over a decade fluorescence microscopy has demonstrated the capacity to achieve single-molecule localization accuracies of a few nanometers, well below the ∼200 nm lateral and ∼500 nm axial resolution limit of conventional microscopy. Yet, only the recent development of new fluorescence labeling modalities, the increase in sensitivity of imaging hardware, and the creation of novel image analysis tools allow for the emergence of single-molecule-based super-resolution imaging techniques. Novel methods such as photoactivated localization microscopy and stochastic optical reconstruction microscopy can typically reach a tenfold increase in resolution compared to standard microscopy methods. Their implementation is relatively easy only requiring minimal changes to a conventional wide-field or total internal reflection fluorescence microscope. The recent translation of these two methods into commercial imaging systems has made them further accessible to researchers in biology. However, these methods are still evolving rapidly toward imaging live samples with high temporal resolution and depth. In this review, we recall the roots of single-molecule localization microscopy, summarize major recent developments, and offer perspective on potential applications.

Structured illumination fluorescence microscopy is a powerful super-resolution method that is capable of achieving a resolution below 100 nm. Each super-resolution image is computationally constructed from a set of differentially illuminated images. However, real-time application of structured illumination microscopy (SIM) has generally been limited due to the computational overhead needed to generate super-resolution images. Here, we have developed a real-time SIM system that incorporates graphic processing unit (GPU) based in-line parallel processing of raw/differentially illuminated images. By using GPU processing, the system has achieved a 90-fold increase in processing speed compared to performing equivalent operations on a multiprocessor computer—the total throughput of the system is limited by data acquisition speed, but not by image processing. Overall, more than 350 raw images (16-bit depth, 512 × 512 pixels) can be processed per second, resulting in a maximum frame rate of 39 super-resolution images per second. This ultrafast processing capability is used to provide immediate feedback of super-resolution images for real-time display. These developments are increasing the potential for sophisticated super-resolution imaging applications.

Localization microscopy techniques based on localizing single fluorophore molecules now routinely achieve accuracies better than 30 nm. Unlike conventional optical microscopies, localization microscopy experiments do not generate an image but a list of discrete coordinates of estimated fluorophore positions. Data display and analysis therefore generally require visualization methods that translate the position data into conventional images. Here we investigate the properties of several widely used visualization techniques and show that a commonly used algorithm based on rendering Gaussians may lead to a 1.44-fold loss of resolution. Existing methods typically do not explicitly take sampling considerations into account and thus may produce spurious structures. We present two additional visualization algorithms, an adaptive histogram method based on quad-trees and a Delaunay triangulation based visualization of point data that address some of these deficiencies. The new visualization methods are designed to suppress erroneous detail in poorly sampled image areas but avoid loss of resolution in well-sampled regions. A number of criteria for scoring visualization methods are developed as a guide for choosing among visualization methods and are used to qualitatively compare various algorithms.