What makes a "class"?

August 4, 2020

Doodler segments images into "classes" that are discrete labels that you define to represent and reduce the dimensionality of features and objects in your imagery.

Each class really represents a spectrum of textures, colors, and spatial extents. It can be difficult to define a good set, but the golden rule (in my humble opinion) is that each class in the set of classes should represent a spectrum of image features that collectively have much greater inter-class variability than intra-class variability.

That can be difficult to define in practice with generality, but here's a tip: imagine a really low- (coarse-) resolution version of your image (or make one!) and ask yourself, "are there are two sets of features that could be easily misconstrued as being in the same class, but are in fact two separate classes?" If the answer in yes, ideally you should lump those classes into a merged class. If your intended outcome dictates that's not possible to do, ask yourself if the two classes could be broken up further, say into 3 or 4 classes, to change your answer to the above question?

In general, doodler is optimized for classes that are relatively large (in terms of contiguous spatial extent) and distinct from one another. See examples in the gallery and elsewhere on this website. However, you can change that behavior by modifying the values of the hyperparameters theta_col and compat_col in your config file. Also, note that the CRF inferences are averaged using weights based on the relative frequency of the classes in your contributed doodles.

You can have any number of classes (well, 2 or more, with the two classes case styled as something and other) but note that increasing the number of classes also increases the amount of time it takes to interact with the program, since each class must be either labelled or actively skipped.

Be prepared to experiment with classes - how well do they work on a small subset of your imagery? Look at the unusual images in your set and and decide if there is a class there you hadn't previously thought of. If you can't decide whether to lump or split certain classes or sets of classes, make separate config files with the different class lists, doodle using the first config file, then run doodle again by passing it the other config file(s) with the npy file you generated on the first go, using the -f flag.

So, the full workflow:

python doodler.py -c config1.json (config1.json has the first set of classes)

then move your outputs in data/label_images, except the .npy file(s), to another folder, and run doodler again, this time with a different config file and the npy file

python doodler.py -c config2.json -f data/label_image/my_npy_file.npy (config2.json has the second set of classes)

and repeat through all your config files with different sets of classes.

Splitting up large geoTIFF orthomosaics

August 1, 2020

Dan Buscombe

Doodler can work with really large images, but it is usually best to keep your images < 10,000 pixels in any dimension, because then the program will do CRF inference on the whole image at once rather than in chunks. This usually results in better image segmentations that are more consistent with your doodles.

So, this post is all about how you make smaller image tiles from a very large geoTIFF format orthomosaic, using python. The smaller tiles will also be written out as image tiles, with their relative position in the larger image described in the file name, for easy reassembly

We'll need a dependency not included in the doodler environment: gdal

conda install gdal

Now, in python:

import os, gdal
from gdalconst import *
from glob import glob

How large do you want your output (square) image tiles to be? (in pixels)

tilesize = 5000

What images would you like to chop up?

bigfiles = [
'Sandwich/2017-01-09_Sandwich_5cm_ortho.tif',
'Sandwich/2017-02-14_Sandwich_5cm_ortho.tif',
'Sandwich/2017-03-16_Sandwich_5cm_ortho.tif',
'Sandwich/2018-01-10_Sandwich_5cm_ortho.tif',
]

List the widths and heights of those input bigfiles

widths = [13314, 13314, 13314, 19972]
heights = [6212, 6212, 6212, 9319]

Specify a new folder for each set of image tiles (one per big image)

folders = ['Sandwich/2017-01-09_5cm', 'Sandwich/2017-02-14_5cm',\
          'Sandwich/2017-03-16_5cm','Sandwich/2017-01-10_5cm']

Make file name prefixes by borrowing the folder name:

prefixes = [f.split('/')[-1] for f in folders]

Finally, loop through each file, chop it into chunks using gdal_translate, called by an os.system() command. Then moves the tiles into their respective folders

for b,f,p in zip(bigfiles, folders, prefixes):

    # chop the image into chunks
    for i in range(0, widths[k], tilesize):
        for j in range(0, heights[k], tilesize):
            gdaltranString = "gdal_translate -of GTIFF -srcwin "+str(i)+", "+str(j)+", "+str(tilesize)+", " \
                +str(tilesize)+" "+b+" "+p+"_"+str(i)+"_"+str(j)+".tif"
            os.system(gdaltranString)

    ##move those chunks to a directory
    os.mkdir(f)
    os.system('mv '+p+'*.tif '+f)

merge a 3-band and 1-band image

July 31, 2020

Dan Buscombe

Doodler can use 1, 3, and 4-band input imagery. If the imagery is 3-band, it is assumed to be RGB and is, by default, augmented with 3 additional derivative bands.

But how do you make a 4-band image from a 3-band image and a 1-band image?

That additional 1-band might be that acquired with an additional sensor, but might more commonly be a DEM (Digital Elevation Model) corresponding to the scene.

I know of two ways. If you have gdal binaries installed, first strip the image into its component bands using gdal_translate

gdal_translate -b 1 data/images/4_rgb.png red.png
gdal_translate -b 2 data/images/4_rgb.png green.png
gdal_translate -b 3 data/images/4_rgb.png blue.png

Then merge them together using gdal_merge.py

gdal_merge.py -separate -o merged.tiff -co PHOTOMETRIC=MINISBLACK red.png green.png blue.png data/images/4_elev.png

The equivalent in python can be acheived without the gdal bindings, using the libraries already in your doodler conda environment

First, import libraries

import tifffile
import cv2
import numpy as np

Read RGB image

im1 = cv2.imread('data/images/4_rgb.png')

Read elevation and get just the first band (if this is 3-band)

im2 = cv2.imread('data/images/4_elev.png')[:,:,0]

If you had a 1-band elevation image, it would be this instead...

im2 = cv2.imread('data/images/4_elev.png')

Merge bands - creates a numpy array with 4 channels

merged = np.dstack((im1, im2))

Write the image to file

tifffile.imsave('test.tiff', merged)

You can use the following to read it back in

merged = tifffile.imread('test.tiff')

And verify with 'shape' - it should be 4 bands

merged.shape

no spaces in filenames!

July 30, 2020

Dan Buscombe

Spaces in image file names are problematic for the program because it uses filename string subsections to match images to outputs. White space is the escape character for most programming languages.

A one-liner for replacing white spaces with underscores in bash is

find -name "* *" -type f | rename 's/ /_/g'

The above command will replace spaces with underscores. Make a copy of your images beforehand for extra caution. The above code is a bash command, so on Windows you'd need git bash, WSL, or have m2-base installed (conda install m2-base)

batch "redoing"

July 29, 2020

Dan Buscombe

If you want to redo all your previous doodles with the new doodler, put all your images in data/images and put the npy files in data/label_images. Then you can call doodler like this in a loop:

for file in data/label_images/*.npy
do python doodler.py -c config.json -f $file
done

and it will cycle through each of the npy annotations, make a new label and probability image based on the current version of the CRF inference encoded in the main program.

The above code is a bash command, so on Windows you'd need git bash, WSL, or have m2-base installed (conda install m2-base)