Core Task: Image classification is a central task in computer vision where an agent is given an input image and must assign it a label from a pre-defined set, such as {dog, cat, truck, plane}.
Feature Representation: Historically, classification relied on manual feature extraction like color histograms. Modern systems use CNNs to learn feature representations directly through training.
2. Biological and Historical Origins
Hubel & Wiesel (1959-1968): Their research on the receptive fields of single neurons in a cat's visual cortex identified a hierarchical organization.
Simple cells respond to light orientation.
Complex cells respond to light orientation and movement.
Hypercomplex cells respond to movement with a specific endpoint.
Neocognitron (1980): Proposed by Fukushima, this was a "sandwich" architecture consisting of simple cells with modifiable parameters and complex cells that performed pooling.
Key Milestones:
LeCun et al. (1998): Provided early illustrations of convolutional neural network structures.
AlexNet (2012): A deep convolutional neural network that achieved a breakthrough in the ImageNet classification competition.
3. Fully Connected (FC) vs. Convolution (CONV) Layers
FC Layer: In a typical fully connected layer, a 32x32x3 image is stretched into a 3072x1 vector. The activation of a neuron is the result of a dot product between a row of weights and the entire input vector.
CONV Layer: Unlike FC layers, a convolution layer preserves spatial structure.
Filters: It uses a filter (e.g., 5x5x3) that "slides" spatially over the image, computing dot products at each position.
Depth: Filters always extend the full depth of the input volume (e.g., a filter for a 3-channel RGB image must have a depth of 3).
Activation Maps: The sliding process creates an activation map representing the filter's responses at every spatial location.
Stacking: Multiple filters (e.g., 6 different filters) create multiple activation maps that are stacked to form the output volume.
Learned Templates: First-layer filters often learn local image templates such as oriented edges or opposing colors.
4. Spatial Dimensions and Parameters
Output Size Formula: The spatial dimension of the output is determined by the formula: $(N - F) / \text{stride} + 1$, where $N$ is the input size and $F$ is the filter size.
Stride: This represents how many pixels the filter moves per step. If the stride does not result in an integer when using the formula, the filter cannot be applied.
Zero Padding: It is common to pad the border of the input with zeros to prevent the spatial volume from shrinking too fast.
Size Preservation: A stride of 1 with a filter size $F$ and zero-padding of $(F-1)/2$ will preserve the spatial size of the input.
Parameter Calculation: Parameters consist of the weights (filter size $\times$ depth) plus a bias for each filter. For example, ten 5x5x3 filters involve $(5 \times 5 \times 3 + 1) \times 10 = 760$ parameters.
5. Pooling (POOL) Layer
Function: This layer makes representations smaller and more manageable.
Independence: It operates over each activation map independently.
Spatial Invariance: It contains no learnable parameters and is used to introduce spatial invariance.
Max Pooling: A common approach where a filter (e.g., 2x2 with stride 2) takes the maximum value within its window, effectively downsampling the image.
6. Architectural Trends
Typical Structure: ConvNets generally stack CONV, POOL, and FC layers. A common historical pattern is [(CONV-RELU)*N-POOL?]*M-(FC-RELU)*K, SOFTMAX.
Modern Shifts: Recent trends favor smaller filters and deeper architectures.
Simplification: There is a growing trend toward removing POOL and FC layers entirely, relying solely on CONV layers.
Advanced Models: Newer architectures like ResNet and GoogLeNet have challenged traditional historical paradigms.