Computer Vision

What is Computer Vision?

Computer Vision (CV) is the field of artificial intelligence that enables computers to extract meaningful information from images, videos, and other visual inputs. Where humans instantly perceive depth, objects, and motion, machines must learn these representations from raw pixel data.

At its core, a digital image is a 2-D grid of pixels. Each pixel stores intensity values — one number for greyscale, or three channels (R, G, B) for colour. A colour image is therefore represented as a 3-D tensor of shape [height × width × channels]. Deep learning frameworks such as PyTorch reshape this into [batch × channels × height × width] (NCHW layout) for efficient GPU processing.

The evolution of image understanding

The field has gone through three distinct eras:

Key CV Tasks

Computer Vision encompasses a wide family of tasks depending on what information needs to be extracted from the image:

Architectures

Each breakthrough architecture introduced a key idea that unlocked the next wave of progress:

Object Detection — Deep Dive

Object detection must solve two problems simultaneously: classification (what?) and localisation (where?). Several foundational concepts underpin all modern detectors.

Anchor Boxes

Most detectors pre-define a set of anchor boxes — rectangles with various aspect ratios and scales placed at regular grid positions across the feature map. The network learns to predict an offset from each anchor to the true bounding box, rather than regressing absolute coordinates. This dramatically simplifies learning because anchors already approximate the rough size and shape of common objects.

Intersection over Union (IoU)

IoU measures how well a predicted box overlaps with the ground-truth box:

A prediction is considered a true positive if IoU exceeds a threshold (commonly 0.5 for PASCAL VOC, or 0.5–0.95 averaged for COCO mAP). IoU is also used as a loss component in modern detectors (CIoU, DIoU) to jointly optimise box shape and position.

Non-Maximum Suppression (NMS)

Detectors produce many overlapping candidate boxes for the same object. NMS iteratively selects the highest-confidence box and removes all other boxes with IoU above a threshold, keeping only the best prediction per object. Soft-NMS and DIoU-NMS are improved variants that handle crowded scenes better.

One-Stage vs. Two-Stage Detectors

Segmentation

Segmentation is the task of partitioning an image into meaningful regions. There are two main flavours, plus the cutting-edge promptable approach introduced by SAM:

Code Example — Image Classification with ResNet

Load a pretrained ResNet-50 from torchvision, preprocess an image, and get the top-5 ImageNet predictions:

import torch
from torchvision import models, transforms
from torchvision.models import ResNet50_Weights
from PIL import Image
import urllib.request, json

# 1. Load a pretrained ResNet-50 (ImageNet weights)
model = models.resnet50(weights=ResNet50_Weights.IMAGENET1K_V2)
model.eval()

# 2. Standard ImageNet preprocessing pipeline
preprocess = transforms.Compose([
    transforms.Resize(256),
    transforms.CenterCrop(224),
    transforms.ToTensor(),
    transforms.Normalize(
        mean=[0.485, 0.456, 0.406],
        std=[0.229, 0.224, 0.225]
    ),
])

# 3. Load and preprocess an image
img = Image.open("dog.jpg").convert("RGB")
input_tensor = preprocess(img).unsqueeze(0)  # add batch dimension → [1, 3, 224, 224]

# 4. Forward pass — no gradients needed for inference
with torch.no_grad():
    logits = model(input_tensor)                # shape: [1, 1000]
    probs  = torch.nn.functional.softmax(logits[0], dim=0)

# 5. Fetch ImageNet class labels
url = "https://raw.githubusercontent.com/anishathalye/imagenet-simple-labels/master/imagenet-simple-labels.json"
with urllib.request.urlopen(url) as r:
    labels = json.loads(r.read())

# 6. Print top-5 predictions
top5 = torch.topk(probs, 5)
print("Top-5 predictions:")
for prob, idx in zip(top5.values, top5.indices):
    print(f"  {labels[idx]:30s}  {prob.item():.2%}")

# Example output:
#   golden retriever                91.23%
#   Labrador retriever               5.41%
#   kuvasz                           1.02%
#   cocker spaniel                   0.61%
#   clumber                          0.28%

Key Tools & Libraries

Related Topics