Aims

The aims of the course are to:

• introduce the key mathematical for performing sophisticated analysis/processing of images by digital hardware

Objectives

As specific objectives, by the end of the course students should be able to:

• understand the main elements of 2-dimensional linear system theory.
• design spatial and frequency domain filters for a variety of applications.
• understand techniques for the restoration and enhancement of degraded images.
• show familiarity with the main characteristics of the human visual system with particular reference to subjective criteria for image data compression.
• understand techniques for image coding using transform methods, specifically the Discrete Cosine Transform and Wavelet Transform (as used in the JPEG coding standard).
• understand how generative AI is applied to images

Content

Sophisticated processing of images by digital hardware is now common, and ranges from special effects in video games to satellite image enhancement. Three of the main application areas are video data compression, image enhancement, and scene understanding/analysis. This module introduces the key tools for performing these tasks, and shows how these tools can be applied. 

The first 60% of the module will discuss the mathematics of basic techniques and how they are applied to images, for both enhancement and compression. The remaining 40% of the module will look at how  some of the now ubiquitous techniques in generative AI are applied to images. 

The basics of Image Processing and Image Coding (10L, Dr J Lasenby)

This course covers the following basic topics, relevant to most aspects of image processing and coding:

1. Two-dimensional linear system theory, as applied to discretely sampled systems:
   
   • The continuous 2D Fourier transform and its properties
   • Digitisation, sampling, aliasing and quantisation
   • The discrete 2D Fourier transform (DFT)
2. Spatial and Frequency domain filters
   
   • Convolution and correlation fundamentals
   • Ideal 2D filters/Zero phase filters: 
   • Windowing
3. Image Restoration/Deconvolution
   
   • Deconvolution of noiseless images -- the inverse filter
   • The Wiener filter (conventional and Bayesian derivations)
   • Non-linear techniques including Maximum Entropy deconvolution 
4. Image Enhancement
   
   • Contrast enhancement
   • Histogram equalisation/bit-plane slicing
   • Median filtering
5. Characteristics of the human visual system which are important for data compression:
   
   • Spatial and temporal frequency sensitivities
   • Distortion masking phenomena
   • Luminance and colour (chrominance) processing
6. 2D block transforms and wavelet transforms:
   
   • Discrete cosine transforms
   • Wavelet transforms
   • Energy compaction properties of transforms for typical images

Generative AI for Images (4L, Prof J Lasenby and Prof Andrew Blake)

These lectures look at using the basic knowledge acquired in the first part of the coures to understand some recent ways in which AI has been applied to images :

1. Neural Networks for Images:
   
   • Convolutional Neural Networks
   • Image classification and segmentation
2. Deep Generative Models — VAEs and the Latent Space :
   
   • The concepts of a latent space
   • Variational autoencoders as a natural bridge from compression. 
3. GANs — Adversarial Training and Image Synthesis
   
   • Image to image translation
   • Evaluation metrics
4. Diffusion Models
   
   • Score matching: forward and reverse noising/denoising
   • Stable diffusion
   • Examples