Advanced Remote Sensing: Lab 6 Classification Accuracy Assessment

Goals and Background

The main goal of this lab is to gain knowledge on evaluating the accuracy of classification results as accuracy assessment is a mandatory exercise following image classification. It is a vital part of the post-processing stage of remotely sensed data. In order to learn the accuracy assessment process there are two main objectives for this lab:
1) collect ground reference testing samples for accuracy assessment

2) use ground reference testing samples to perform accuracy assessment

Methods

The accuracy assessment in this lab was done using ERDAS Imagine 2015. 

Part 1: Generating ground reference testing samples for accuracy assessment 

The first step in the process of accuracy assessment is to create ground reference testing samples. These ground samples can be collected in the field before classification but if that is not an option they can also be created using a high resolution image as we are doing in this lab.

The first part of this lab is about creating those ground sample points using high resolution aerial imagery of our study area. The image that was assessed for accuracy is the coded unsupervised classification image created in Lab 4. First I opened this image in a ERDAS viewer and then brought in an high resolution aerial image of  the same area into another viewer. This image from 2005 and will serve as the reference image in the accuracy assessment. This image is also where the reference samples will be created. Once they are both open (Figure 1) then the accuracy assessment dialogue is opened. Select the first viewer with the unsupervised classification image and click on the raster tab > supervised > accuracy assessment. This will open the accuracy assessment window (Figure 2) in which you want to open the classified image. Next clicking anywhere in viewer two containing the 2005 imagery will select that image as the reference image for the assessment. Next random points need to be generated. This is done by going to edit > create/add random and this will open the add random points window. In this window some presets need to be changed. For this lab we did 125 in the number of points, set the distribution parameter to stratified random, the minimum number of point to 15 and selected the 4 classes from the unsupervised classification image (Figure 3). Click OK and the reference image has 125 points that appear on it.     

Figure 1 These are the two images used for the accuracy assessment. The unsupervised classification on the left and 2005 reference image in the right. 

Figure 2 This is the accuracy assessment dialogue where the classification and reference images are selected. 

Figure 3 This is the add random points dialogue where the 125 points are added to the 2005 reference image to conduct the assessment. 

Part 2: Performing accuracy assessment 

Section 1: Evaluation of reference points 

Now that the sample points are generated the accuracy assessment can begin. In the accuracy assessment window the first 10 random points are selected. Click show current selection from the view menu and these points will change appear on the reference image as white. Using the same numbering scheme for the classes from Lab 5 I went through and identified the LULC class for each of the 125 random points in the reference image. This is done by finding the point on the reference map and then looking at the unsupervised classification image from lab 4 for the LULC class and recording that in the accuracy assessment table. After each sample point is classified it will change from white to yellow.  This process can be seen in Figure 5. Figure 6 is the table is the table with the reference points in it seen in Figure 5. This is where the classification number is entered.  

Figure 5 This is what the reference image will look like after all of the random sample points have been classified. They will turn from white to yellow.

Figure 6 The are the random generated points in the table with the classification number in the left hand reference column. 

Section 2: Generating accuracy assessment report 

Once each of the 125 points is classified in the accuracy assessment window the next step is to generate the accuracy report. This is done by selecting accuracy report from the report drop down. Figure 8 is what that report will look like. To make the report easier to understand I created an Excel table (Figure 9).

Figure 8 This is the raw accuracy assessment report created in ERDAS Imagine 2015. 

Figure 9 This is the cleaned up easier to understand accuracy report I created in Excel.

Part 3: Accuracy assessment of supervised classification 

The same process was repeated from parts 1 and 2 to conduct an accuracy assessment on the supervised classification image from Lab 5 (Figure 10). 125 points were created to do the assessment and run the accuracy report. There was however an error when the report was created. Labels were not accurate and the report did not produce any Kappa Statistics (Figure 11). This malfunction may be due to an error between the algorithm used and the newest version of the ERDAS software. For his reason an accuracy assessment of the classified image has not been completed and the accuracy of the classified and unclassified images can not be compared. 

Figure 10 The supervised classification image on the left and the reference image with the 125 random points on the left.

Figure 11 This is the accuracy report containing the error for the supervised image. 

Sources

The Landsat satellite image is from Earth Resources Observation and Science Center, United States Geological Survey. 

The high resolution image is from United States Department of Agriculture (USDA) National Agriculture Imagery Program.