dl_tools

DL tools

A collection of tools for image classification and recognition using deep transfer learning

Written by Dr Daniel Buscombe Northern Arizona University daniel.buscombe@nau.edu

Credits

Imagery in demo_data collected by Jon Warrick, USGS Santa Cruz

This toolbox was prepared for the “MAPPING LAND-USE, HAZARD VULNERABILITY AND HABITAT SUITABILITY USING DEEP NEURAL NETWORKS” project, funded by the U.S. Geological Survey Community for Data Integration, 2018

Thanks: Jenna Brown, Paul Grams, Leslie Hsu, Andy Ritchie, Chris Sherwood, Rich Signell, Jon Warrick, and all of those who attended two CDI Deep Learning workshops in summer 2018

Please cite

If you find these tools useful for your work, please cite the following paper:

Buscombe, D., and Ritchie, A.C. 2018. Landscape Classification with Deep Neural Networks. Geosciences 8 (7) https://doi.org/10.3390/geosciences8070244

in which you’ll also find more details about the algorithms and workflows used in this toolbox


Installation

Open an anaconda terminal and clone this repository from github

git clone --depth 1 https://github.com/dbuscombe-usgs/dl_tools.git

picture1

conda env create -f tf_env.yml 

picture2

You can see that this has been created in your conda env folder, e.g.

picture3

conda activate dl_tools

picture4


Data

The provided example dataset is organized as follows

dl_tools_data



METHOD 1

1) Create test an training data sets

This function is designed to take a folder of images and create a new set of testing and training images based on a specified proportional split. Training images will be used to train a model. Testing images will be used to test that model.

python create_library\images_split_train_test.py -p 0.5

picture9

dl_tools_data_split


2) Create groundtruth (label) image using the CRF approach outlined by Buscombe & Ritchie (2018)

python create_groundtruth\label_1image_crf.py -w 500 -s 0.2

which says I will be using a 500x500 pixel window of the image at any one time, and the image will be rescaled to 20% of its original size to do the CRF processing (for speed)

pick_image

pick_labels

image

tile

cmd_label

active_anno

generating

CRF inference time depends primarily on image complexity and size, but is also secondarily affected by the number and spatial heterogeneity of the class labels.

python create_groundtruth\label_1geotiff_crf.py -w 500 -s 1.0

There are other python libraries you’ll need to use it:

conda install rasterio gdal


[Optional]

To help speed your test up, twelve ground truth label files are provided in demo_data\gt. Assuming you have executed Step 1 above, you may copy those files into the correct folder (train or test) using

cd create_library
python cp_mat.py
cd ..

Now if you look inside test and train you will see a few *.mat files per folder


3) Create a library of image tiles for retraining a DCNN

python create_library\retile.py -t 96 -a 0.9 -b 0.5

picture10

picture11

picture12


4) Retrain a deep convolutional neural network

Example: 0.01

python train_dcnn_tfhub\retrain.py --image_dir demo_data\train\tile_96 --tfhub_module https://tfhub.dev/google/imagenet/mobilenet_v2_100_96/classification/1 --how_many_training_steps 1000 --learning_rate 0.01 --output_labels labels.txt --output_graph monterey_demo_mobilenetv2_96_1000_001.pb --bottleneck_dir bottlenecks --summaries_dir summaries

picture25

tensorboard --logdir=summaries

Then you can view training and validation data in your browser (localhost) using the url that is specified.


5) Evaluate image tile classification accuracy

This script evaluates how well the retrained DCNN performs at an ‘image recognition’ task (could be whole images, or tiles like this example)

python eval_imrecog\test_class_tiles.py -n 100

picture26

picture27

test_cm


6) Perform semantic segmentation of an image using the “hybrid” approach outlined by Buscombe & Ritchie (2018)

python semseg_crf\semseg_cnn_crf.py demo_data\test\D800_20160308_221825-0.jpg monterey_demo_mobilenetv2_96_1000_001.pb labels.txt demo_data\labels\monterey_label_colors.txt 96 0.5 0.5 8 0.25

picture28


7) Evaluate the accuracy of the semantic segmentation

python eval_semseg\test_pixels.py

d800_20160308_221825-0_ares_96cm



METHOD 2

This method is designed to predict pixelwise labels without training on tiles (i.e. it uses pixelwise label images directly to train the model and infer classes on unseen imagery)

8) Fully convolutional semantic segmentation, implementing the method of Long et al 2015

python semseg_fullyconv\make_labels.py demo_data\data\labels\gtFine\train\data
python semseg_fullyconv\make_labels.py demo_data\data\labels\gtFine\val\data
python semseg_fullyconv\train.py --name data_test10 --data-source data --data-dir demo_data\data --epochs 10
python semseg_fullyconv\infer.py --name data_test10 --samples-dir demo_data\data\samples\RGB\val\data --output-dir test_output --data-source data
python semseg_fullyconv\infer_crf.py --name data_test10 --samples-dir demo_data\data\samples\RGB\val\data --output-dir test_output --data-source data