# Edge Detection¶


This numerical tour explores local differential operators (grad, div, laplacian) and their use to perform edge detection.

In [2]:
options(warn=-1) # turns off warnings, to turn on: "options(warn=0)"

library(imager)
library(png)

for (f in list.files(path="nt_toolbox/toolbox_general/", pattern="*.R")) {
source(paste("nt_toolbox/toolbox_general/", f, sep=""))
}

for (f in list.files(path="nt_toolbox/toolbox_signal/", pattern="*.R")) {
source(paste("nt_toolbox/toolbox_signal/", f, sep=""))
}

source("nt_toolbox/toolbox_wavelet_meshes/meshgrid.R")
options(repr.plot.width=3.5, repr.plot.height=3.5)


## Diffusion and Convolution¶

To obtain robust edge detection method, it is required to first remove the noise and small scale features in the image. This can be achieved using a linear blurring kernel.

Size of the image.

In [2]:
n <- 256*2


Load an image $f_0$ of $N=n \times n$ pixels.

In [3]:
f0 <- as.matrix(load_image("nt_toolbox/data/hibiscus.png",n))


Display it.

In [4]:
imageplot(f0)


Blurring is achieved using convolution: $$f \star h(x) = \sum_y f(y-x) h(x)$$ where we assume periodic boundary condition.

This can be computed in $O(N\log(N))$ operations using the FFT, since $$g = f \star h \qarrq \forall \om, \quad \hat g(\om) = \hat f(\om) \hat h(\om).$$

In [5]:
cconv <- function(f, h){
c <- fft(f)*fft(h)
return( Re(fft(c, inverse=T )/length(c)) ) }


Define a Gaussian blurring kernel of width $\si$: $$h_\si(x) = \frac{1}{Z} e^{ -\frac{x_1^2+x_2^2}{2\si^2} }$$ where $Z$ ensure that $\hat h(0)=1$.

In [6]:
t <- c(0:round(n/2), (round(-n/2)+1):(-1))
X2 <- meshgrid_2d(t, t)$X ; X1 <- meshgrid_2d(t, t)$Y
normalize <- function(h){ h/sum(h) }
h <- function(sigma){ normalize(exp(-(X1**2 + X2**2)/ (2*sigma**2))) }


Define blurring operator.

In [7]:
blur <- function(f, sigma){ cconv(f, h(sigma)) }


Exercise 1

Test blurring with several blurring size $\si$.

In [8]:
options(repr.plot.width=7, repr.plot.height=7)

source("nt_solutions/segmentation_1_edge_detection/exo1.R")

In [9]:
## Insert your code here.


The simplest edge detectors only make use of the first order derivatives.

For continuous functions, the gradient reads $$\nabla f(x) = \pa{ \pd{f(x)}{x_1}, \pd{f(x)}{x_2} } \in \RR^2.$$

We discretize this differential operator using first order finite differences. $$(\nabla f)_i = ( f_{i_1,i_2}-f_{i_1-1,i_2}, f_{i_1,i_2}-f_{i_1,i_2-1} ) \in \RR^2.$$ Note that for simplity we use periodic boundary conditions.

Compute its gradient, using (here decentered) finite differences.

In [10]:
s <- c(n, 1:(n-1))

nabla <- function(f){
dx <- f - f[s,]
dy <- f - f[,s]
}


One thus has $\nabla : \RR^N \mapsto \RR^{N \times 2}.$

In [11]:
v <- nabla(as.matrix(f0))


One can display each of its components.

In [12]:
options(repr.plot.width=7, repr.plot.height=3.5)

imageplot(v[,,1], "d/dx", c(1,2,1))
imageplot(v[,,2], "d/dy", c(1,2,2))


A very simple edge detector is obtained by simply thresholding the gradient magnitude above some $t>0$. The set $\Ee$ of edges is then $$\Ee = \enscond{x}{ d_\si(x) \geq t }$$ where we have defined $$d_\si(x) = \norm{\nabla f_\si(x)}, \qwhereq f_\si = f_0 \star h_\si.$$

Compute $d_\si$ for $\si=1$.

In [13]:
sigma <- 1
d <- sqrt(apply(nabla(blur(as.matrix(f0), sigma))**2, c(1,2), sum))


Display it.

In [14]:
options(repr.plot.width=3.5, repr.plot.height=3.5)

imageplot(d)


Exercise 2

For $\si=1$, study the influence of the threshold value $t$.

In [15]:
options(repr.plot.width=7, repr.plot.height=7)

source("nt_solutions/segmentation_1_edge_detection/exo2.R")

In [16]:
## Insert your code here.


Exercise 3

Study the influence of $\si$.

In [17]:
options(repr.plot.width=7, repr.plot.height=7)

source("nt_solutions/segmentation_1_edge_detection/exo3.R")

In [18]:
## Insert your code here.


## Zero-crossing of the Laplacian¶

Defining a Laplacian requires to define a divergence operator. The divergence operator maps vector field to images. For continuous vector fields $v(x) \in \RR^2$, it is defined as $$\text{div}(v)(x) = \pd{v_1(x)}{x_1} + \pd{v_2(x)}{x_2} \in \RR.$$ It is minus the adjoint of the gadient, i.e. $\text{div} = - \nabla^*$.

It is discretized, for $v=(v^1,v^2)$ as $$\text{div}(v)_i = v^1_{i_1+1,i_2} + v^2_{i_1,i_2+1}.$$

In [19]:
div <- function(v){
v1 <- v[,,1]
v2 <- v[,,2]
t <- c(2:n,1)
return( v1[t,] - v1 + v2[,t] - v2 )
}


The Laplacian operatore is defined as $\Delta=\text{div} \circ \nabla = -\nabla^* \circ \nabla$. It is thus a negative symmetric operator.

In [20]:
delta <- function(f){ div(nabla(f)) }


Display $\Delta f_0$.

In [21]:
options(repr.plot.width=3.5, repr.plot.height=3.5)

imageplot(delta(as.matrix(f0)))


Check that the relation $\norm{\nabla f} = - \dotp{\Delta f}{f}.$

In [22]:
dotp <- function(a,b){ sum(a*b) }
print(paste("Should be 0:", (dotp(nabla(f0), nabla(f0)) + dotp(delta(f0), f0))))

[1] "Should be 0: 2.27373675443232e-13"


The zero crossing of the Laplacian is a well known edge detector. This requires first blurring the image (which is equivalent to blurring the laplacian). The set $\Ee$ of edges is defined as: $$\Ee = \enscond{x}{ \Delta f_\si(x) = 0 } \qwhereq f_\si = f_0 \star h_\si .$$

It was proposed by Marr and Hildreth:

Marr, D. and Hildreth, E., Theory of edge detection, In Proc. of the Royal Society London B, 207:187-217, 1980.

Display the zero crossing.

In [23]:
sigma = 4

options(repr.plot.width=3.5, repr.plot.height=3.5)

plot_levelset(delta(blur(f0, sigma)), f0)


Exercise 4

Study the influence of $\si$.

In [24]:
options(repr.plot.width=7, repr.plot.height=7)

source("nt_solutions/segmentation_1_edge_detection/exo4.R")