Convolutions

Image Modeling with Keras

Ariel Rokem

Senior Data Scientist, University of Washington

Using correlations in images

Natural images contain spatial correlations
For example, pixels along a contour or edge
How can we use these correlations?

Biological inspiration

What is a convolution?

array = np.array([0, 0, 0, 0, 0, 1, 1, 1, 1, 1])

kernel = np.array([-1, 1])

conv = np.array([0, 0, 0, 0, 0, 0, 0, 0, 0])

conv[0] = (kernel * array[0:2]).sum()
conv[1] = (kernel * array[1:3]).sum()
conv[2] = (kernel * array[2:4]).sum()
...

for ii in range(8):
    conv[ii] = (kernel * array[ii:ii+2]).sum()
conv

array([0, 0, 0, 0, 1, 0, 0, 0, 0])

Convolution in one dimension

array = np.array([0, 0, 1, 1, 0, 0, 1, 1, 0, 0])
kernel = np.array([-1, 1])

conv = np.array([0, 0, 0, 0, 0, 0, 0, 0, 0])
for ii in range(8):
    conv[ii] = (kernel * array[ii:ii+2]).sum()

conv

array([ 0,  1,  0, -1,  0,  1,  0, -1,  0])

Image convolution

Two-dimensional convolution

kernel = np.array([[-1, 1], 
                   [-1, 1]])


conv = np.zeros((27, 27)

for ii in range(27):
    for jj in range(27):
        window = image[ii:ii+2, jj:jj+2] 
        conv[ii, jj] = np.sum(window * kernel)

Convolution

Let's practice!

Image Modeling with Keras