Introduction
In this paper, morphological and kinematical analysis of sea ice are divided into local and global structures. The rotational and translational motion of a local structure is analyzed via a feature selection and registration process. The motion of a global structure (an entire image covering 100 km by 100 km) is analyzed via an automatic template process guided by a predicted motion model. Currently, most motion analysis techniques are based on tie points selected either manually or automatically. The selected tie points imply geometrical mapping between two images and the motion is analyzed via the displacement of each tie-point pair under the assumption that the two images represent the same region.
In the case of manual selection (1), the global motion is analyzed by selecting tie points over an entire image area. After the tie points have been selected, each displacement vector is computed by connecting the tie-point pair’s locations. The magnitude of the displacement vector indicates the amount of displacement of a given feature between two time frames.
where
Vi is displacement vector of ith tie-point pair
(Xi, Yi) is the location of the ith tie-point in a reference image
(xi, Yi') is the location of the ith tie-point in a compared image
When the two images do not overlap exactly, the actual displacement can be updated by subtracting the regional displacement vector from each displacement vector. Also, relative motion with respect to a selected land mark can be analyzed in a similar manner by replacing the regional displacement vector with the displacement vector of the land mark. Equation 2 shows the displacement vector update where T is a common displacement between two images.
where
(x0, y0) is the origin location of a reference image
(x0' y0') is the matching location in a compared image
An image pair covering 100 km by 100 km requires several hundreds of tie-point pairs, which takes more than a day, even for an experienced scientist. The manual approach is not only time — consuming but also very strenuous to human eyes. Besides, it does not provide rotational information of specific sea-ice structures since the structural connectivity is not present in tie points.
In order to resolve the strenuous tie-point selecting process of the manual approach, various tie-point selection techniques have been researched. Major problems which exist in developing automatic tie-point selection techniques on SEASAT SAR images are: first, the sea-ice images lack distinctive features; secondly, the boundary features may vary due to seasonal formation and/or deformation; thirdly, the range of sea-ice motion is enormous; fourthly, there is no solid motion unit.
In the case of automatic selection (2), template matching is used to automate tie-point detection, in order to eliminate the laborious tie-point selection process. A template is an image segment inside an arbitrarily chosen window, where a window is usually defined as a regular grid. A selected template from the reference image, is passed over a search area in the compared image until the best match is found. The center of the reference image template and the center of the best matching template are selected as a tie-point pair. The template matching process is then repeated over the entire area for global motion analysis. The template matching process can be expressed as
where
| Cn | is a correlation score array of the nth template |
| Tn | is nth template |
| Sn | is nth template search area |
| (nx,ny) | is the size of the template Tn |
| (sx,sy) | is the size of the search area Sn |
The location (k,1) that yields the maximum matching score will be the tie-point location for the template Tn.
The number of multiplication required for each template matching is the number of pixels in a template multiplied by the number of pixels in a search area. In order to reduce the computational burden, the template matching is usually performed in a hierarchical manner using a low-resolution image first to find an approximate match and then increase the resolution step by step to narrow down the search area. However, due to the similarity between sea-ice image pairs, using a low-resolution image can be dangerous. The computation time of this approach fluctuates greatly depending on the motion characteristics of the sea-ice images. Also, this approach cannot be used for a local ice patch motion analysis since regular templates do not represent sea-ice structures.
This paper presents an alternative for sea-ice structure and motion analysis with a hybrid approach at local and global scale. Major objectives of this approach are to reduce the laborious tie-point selecting process of the manual approach and to reduce the computation time in the automatic template matching approach. The first objective is achieved by replacing the tie-point selecting process with a computer-assisted feature selection and registration process. The second objective is achieved by improving the blind search of automatic template matching with a guided search from a motion prediction model. The procedure flow of this approach İs illustrated in the flow chart of Fig.1. First, a subset of primary features are selected from the reference “rigid body”. Thus, movement of a local ice structure can be realized by the translational as well as rotational deviations of a feature. The movement vector field of the local structures can then be utilized to evaluate the motion of the global ice field.
Fig. 1 Procedure flow chart of sea-ice motion anaylsis.
A set of computer-aided tools was developed to assist the feature selection and registration process. The tools for the feature selection process consist of three commands that can be activated by simply pressing a button on the trackball or on the mouse of a frame buffer. The commands are “move cursor”, “draw line”, and “delete previous line”. With these commands, a user can extract features of interest from a given area displayed on the frame buffer. The feature lines are drawn on the overlay plane of the frame buffer. For the local motion analysis of the selected features, the compared image is displayed on the frame buffer and the selected features are registered individually to the compared image. The tools for the feature registration include four commands: “select a feature”, “translate (up, down, left, right)”, “rotate (clockwise, counter-clockwise)”, and “delete” (the selected feature). The “feature delete” command is essential when the feature can not be found from, or registered to, the compared image. As a feature is registered to the compared image, the translation and rotation information is updated and it is displayed on the terminal screen.
The SEASAT SAR images taken over Banks of Island image and they are registered to the compared image. The local motion analysis of specific features can be performed in this procedure. Secondly, a predicted global motion model is generated using the result of the first step. Thirdly, a guided template matching is applied and tie-points are selected. Fourthly, the accuracy of the tie-points is verified. Global ice-floe vector fields are computed when the accuracy test is passed; otherwise, the third step is repeated with larger search areas until the accuracy test is passed. Each step is discussed in detail at the following sections.
Local Motion Analysis Via Feature Selection and Registration
Simply stated, motion analysis is performed by evaluation of the “rigid body”‘s morphical deviations from an earlier image to a later image (in SEASAT’s case, three days apart). The earlier image is used as a reference image and the later image is used as a compared image. This evaluation is performed in part by selecting a set of recognizable features from the reference image and then registering them to the compared image, A feature can be represented by a set of independent points, as in the tie- point methods, or, alternatively, by a set of connected line segments belonging to a single rigid structure. The line representation reduces the redundant tie-point pair selection process since all of the points within a “rigid body” must maintain their geometrical relationships throughout the movements. The relationship of the line segments to a single structure also incorporates the rotational attributes of a were used to illustrate the local motion analysis. Fig.2a shows selected features from the reference image where five easily recognizable features are drawn with polygons and polylines. Each feature is assigned an identification number so that it can be controlled individually. Fig.2b shows the result of a feature registration process, where the five features are registered on the second image using the translation and rotation commands. Fig.2c shows the local motion vectors of each feature, where the translation of each feature is computed by connecting the rotation center of a feature from the reference image to the compared image. The rotation of each feature is expressed as a set of displacement vectors from every vertex of a feature relative to its rotation center. The rotation center of a feature is defined to be the center of an imaginary box that surrounds the feature. The rotation center can be arbitrarily chosen if desired. This process provides tools to analyze the detailed motion of any local structure very easily without intensive computation or laborious tie-point selection.
Fig. 2a Reference image features (“fast ice” in Banks of Island area).
Fig. 2b Result of feature registration on the compared image (three days later).
Fig. 2c Local motion analysis of the selected features.
Motion Prediction Model Generation
The feature selection and registration process is also used to assist the global motion analysis so that the automatic template matching technique can be applied without intensive computation on a blind search. The global motion prediction is based on the assumption that sea ice cannot move randomly but in a somewhat correlated manner. Therefore, the correlated motion can be predicted based on the information acquired from the selected features using proper interpolation and extrapolation techniques. The predicted motion model is used to determine the search area for the template matching instead of the hierarchical correlation technique discussed earlier. The generation of a prediction model takes insignificant computation time compared to the correlation process. The model is more accurate than other types of prediction model since it is based on manually selected and registered features (i.e. the surface fitting uses the tie points extracted from the vertices of the selected features and the registered features).
Two types of surface fitting methods were considered, the triangulation method for discontinuous surface fit and the least-squares surface fit method for continuous surface fit. The triangulation method connects all the tie points to form the shortest distance triangles and the points inside the triangle are interpolated based on the relation between matching triangles. The mapping of the triangulation method is shown in Equation (4), where three vertices of a pair of mapping triangles are used to compute the coefficients of a bi-linear equation.
where
where
(xi y1) is a vertex of a triangle in the reference image
(xi'yi') is a vertex of a triangle in the compared image
After the Equation (4) is solved for every triangle in the image plane, a motion prediction model is constructed by computing predicted locations of regular grid points. Each grid point location is analyzed to find the triangle in which the grid point resides; then, the proper set of coefficients to the matching triangle is applied to determine the predicted location of every grid point. This method cannot be used for extrapolation directly. It has to employ the least-squares surface fit method to produce boundary points; then new triangles are formed to the boundary points for extrapolation.
The least-squares surface fit method solves a polynomial equation to find a smooth function that describes the relation between two image surfaces. The least-squares surface fit method is described in Equation (5), where any subset of the polynomial equation can be used to estimate the surface fitting. The first-order surface fit requires up to (a2, b2), the second-order surface fit requires up to (a5, b5), the third-order surface fit requires up to (a9, b9), and etc. The order of surface fit may be determined by the characteristics of motion.
where
(x, y) is a location in a reference image
(x', y') is a matching location in a compared image
The Equation (5) can be written as
where
The matrix A can be solved by
After the least-squares surface fit equation is solved, the predicted motion model is generated by computing the predicted location of the regular grid points that will be used for template matching. Fig.3 shows a predicted motion model using a regular grid of 16 pixels by 16 pixels over a 512 pixel by 512 pixel image area. The sheared grid illustrates a predicted shearing of the compared image. Each grid point is a predicted location of the corresponding regular grid point, which also implies that it is a predicted tie-point.
Global Motion Analysis
After a set of predicted tie points is obtained, the final step is to apply automatic template matching to correct the error of the predicted tie points. The prediction error can be corrected by applying automatic template matching described earlier around the predicted area. The global motion result displayed in Fig.4 was obtained using a 16 pixel by 16 pixel template to search over a 48 pixel by 48 pixel area, where each pixel represents a 200 m by 200 m area. The search area allows up to a 16 pixel prediction error in four directions (left, right, up, down). In order to analyze motion of the images shown in Fig.2 without motion prediction, the search area should be greater than 200 pixels by 200 pixels, which implies at least 16 times more multiplication for each template and correspondingly more CPU time. For a image of 512 by 512 pixels, the proposed approach takes about 1.5 VAX 11/780 CPU minutes for template matching.
The areas with no motion vector in Fig.4 indicate that the template matching process has failed to find matching templates. The reasons for failure are twofold: (1) features in the area were distorted beyond recognition, and (2) the search area was not large enough to correct the prediction error. The first case cannot be resolved by template matching and can only be interpolated from the surrounding motion. The second case can easily be resolved by iterating the template matching process to a larger search area as shown in Fig.1, The iteration may continue as more templates find their matching templates; otherwise it terminates.
An image pair, collected over the Beaufort Sea three days apart on orbits 1439 and 1482, is analyzed for comparison with existing methods since it is frequently analyzed by other techniques. Fig.5a displays five selected features, Fig.5b shows the result of feature registration, Fig.5c shows the prediction model using the third-order, least-squares surface fit, Fig.5d shows the results of automatic template matching which shows displacement vectors of regularly distributed tie-points and Fig.5e shows the distortion surface characteristics of 1482 due to sea-ice motion. The manual approach took approximately 8 hours to detect 800 tie-points in the central area and 5 hours to detect 200 tie-points in the marginal area. The proposed approach took 80 seconds of VAX 11/780 CPU time and a few minutes of user time to select and register the features. As for the existing automatic approach, timing was not published by the authors. This particular image pair is an easier case to solve since ice pack does not have the large motion range of fast ice as in the case shown in Fig.2. However, the search area for the template matching process has to be larger without a prediction model, which implies longer processing time.
Fig. 5a Reference image features (“ice pack” in Beaufort Sea).
Fig. 5b Result of feature registration on the compared image (three days later).
Fig. 5c Predicted motion model.
Fig. 5d Global motion analysis.
Fig. 5e Surface deformation.
Summary and Conclusions
The entire procedures, starting from the feature selection and registration to the display of final motion vectors, has been implemented using VAX 11/780 and RASTER TECHNOLOGY frame buffer model 125. Without intensive computation or laborious user interaction, this approach not only allows a detailed motion analysis of an individual sea-ice morphism but also analyzes structure and motion of global ice fields with high accuracy. Major future research areas include calculation of stress and speed, motion map generation using a motion mosaic of multiple image pairs, multi-factor prediction model generation which utilizes climatological information as well as motion history of a given area, intelligent template matching where template matching is regressed from a highly confident area to a less confident area based on neighboring template matching results, and, finally, feature-based motion analysis instead of a template-based motion analysis via automatic feature selection and registration.
Acknowledgements
The work described in this paper was carried out by the Jet Propulsion Laboratory, California Institute of Technology, under contract with the National Aeronautics and Space Administration.
