Probabilistic segmentation and fuzzy classification of natural vegetation in hyperspectral imagery

Thesis for the degree of M.Sc. in Geodetic Engineering Delft University of Technology May 2003 Jochem Lesparre

Complete version in Dutch (3,3 MB pdf file)

Summary
 

Introduction

The Survey Department (Meetkundige Dienst) investigates semi-automated interpretation of remote sensing images for Rijkswaterstaat to improve the efficiency and especially the objectivity of its vegetation mapping. This shows two problems. First, there seem to be differences when comparing two successive classifications that are caused by chance, not by real changes. The second problem is that gradual changes in mixing proportions of two classes are detected not at all or too late, because they often give no cause for classification in another class. The solution to these to problems is expected to be found in fuzzy classification methods.
Another problem is the poor spectral distinction between classes of natural vegetation, which complicates automated classification. Therefore one desires the use of hyperspectral scanners and researches the use of new smart classification methods that for instance make use of expert knowledge or spatial structures.
 

Objective

The objective of this master thesis is to investigate to what extend the use of fuzzy classification by means of probabilistic segmentation [Gorte 1998] gives an improvement compared to crisp pixel-by-pixel classification for monitoring of natural vegetation. An improvement by probabilistic segmentation is expected because this method takes into account spatial characteristics of the field objects. Probabilistic segmentation is developed on multispectral data of agricultural areas. Natural vegetation does not grow that structured as cultivated crops, but it is not completely random either. Hence, an improvement of the classification by segmentation is expected for natural vegetation too.
 

Methodology

The used method consisted of the application of the probabilistic segmentation method on hyperspectral images of natural vegetation. Additionally, a brief literature research was executed. As the development and modification of software take a lot of time, this research is focused on the extension of probabilistic segmentation for application on hyperspectral images of natural vegetation, and programming this.
During the research the idea arose for a new method of fuzzy training for maximum likelihood classification. As it was decided to use k-nearest neighbours classification instead of maximum likelihood classification in the end, the fuzzy training method is only described, and has not been applied.
 

Data

The used images are acquired by the HyMap (Hyperspectral Mapping) system. This is a hyperspectral airborne scanner that records 128 bands. This scanner is used to acquire the salt marsh on the Dutch Friesian Island Schiermonnikoog with a spatial resolution of 3.5 meters.
For training and validation of the classification 384 field plots are used which were acquired during fieldwork. These have been split about fifty fifty in a set of training samples and validation samples. In this study four different divisions in classes are used. The appointment of these divisions is made manually, which makes them quite subjective. This results in classes with rather large spectral overlaps.
 

Fuzzy training for maximum likelihood classification

Most existing methods need pure pixels for training, which complicates training for natural vegetation. Because of the occurrence of many mixed pixels it is difficult to obtain a sufficiently large number of pure training samples. The use of almost pure samples is not advisable. These cause the estimated variance to be composed partly by the degree of occurrence of other classes, instead of the natural variation in the spectrum of the class. The solution for the lack of pure training samples is to be found in the use of mixed pixels to estimate the spectra of pure classes. F. Wang [1990] suggests a method to calculate a weighted average and (co)variance by using the fraction of a class as the weight. This method does not give an unbiased estimation of the spectra of the pure classes.
It is possible to execute an unbiased maximum likelihood training that estimates pure spectra out of mixed pixels by using the adjustment theory and probability model estimation. The advantage of fuzzy training is that more pixels in the image can be used for training, which enables the use of heterogeneous areas for training or random sampling. There are two conditions on this fuzzy training method. First, one needs to have estimations of the fractions of the classes for the mixed training samples. Secondly, the spectra of the mixed pixels should be a linear mixture of the composing classes. In order to test the utility of fuzzy training for maximum likelihood classification, it is recommended to apply the method to an image for which class fractions of the training samples have been determined during fieldwork.
 

Probabilistic segmentation

Probabilistic segmentation is an integration of image segmentation and classification. The classification is carried out according to Bayes formula. The prior probabilities of a class in this formula are normally kept equal for all classes, so the classification is not favoured too much towards the prevailing classes. Probabilistic segmentation, however, does do this. The method estimates this probability locally on the basis of the image using the result of a probabilistic fuzzy classification. By estimating this probability locally it's possible to improve the classification. A yellow pixel in a yellow field for instance is more likely to be wheat than grass, while the same yellow pixel in a green field is more likely to be (parched) grass than wheat. To estimate the local prior probabilities a subdivision of the image in segments is needed, in which segments coincide with field objects as much as possible. So only one or a few classes are present per segment.
The applied segmentation method merges adjacent pixels to segments on the basis of the spectral characteristics of three bands. Pixels are merged if the Euclidean distance in the feature space between the pixels is smaller than a threshold, and the (co)variances of the created segment is smaller that another threshold. It's not clear, however, which threshold gives the best segmentation. Moreover, the best threshold seems to vary for different areas in the image. The solution to this is building a segmentation pyramid by segmenting the image with increasing thresholds. Next, segments are selected from different levels of the pyramid, on the basis of the class proportions in the segments. To do this, class proportions per segment are needed for the entire pyramid, which are acquired by the estimation method for the prior probabilities. The segments with as few as possible classes are selected. If there is more than one candidate, the largest segment is selected, because the estimates of the prior probabilities are more accurate for large segments.
 

Conclusions

For the utilised image and accompanying field data the k-nearest neighbours classification gives the best estimated probabilities as input for probabilistic segmentation. Two modifications were carried out for applying probabilistic segmentation on hyperspectral images of natural vegetation with a limited number of training samples. First, an implementation of k-nearest neighbours classification that's able to deal with hyperspectral data is adopted. Secondly, the segment selection rules are adapted in a way that no longer only pure segments are preferred, but segments with as least classes as possible are aimed for.
Probabilistic segmentation gives a slight improvement in the overall accuracy of the classification. The rate of improvement depends on the used division in classes. The maximum improvement occurred with a division in 9 classes (level 1). The overall accuracy increased from 61.93% to 67.43%. The division with the minimum improvement has 21 classes (level 3). Here the overall accuracy increased from 40.83% to 41.28%. It is unclear why the improvements are quite small. The data or the probabilistic segmentation method could be the cause. The data can cause this by a too large spectral overlap of the classes, but also by a too low spatial resolution or by too few training samples. A too large preference for pure segments (even when these are really small) in the segment selection of the probabilistic segmentation method could also cause this.
 

Recommendations

In the first place the cause of the minor improvement by probabilistic segmentation should be examined, as well as the reason for the selection of many small segments. By applying probabilistic segmentation on other data it could be verified whether the data is the problem. By using a manually defined segmentation it could be checked whether the segment selection method is the cause.
Furthermore, the design of a method to validate a fuzzy classification could be strongly recommended.