Stage 11 · Lesson 02/16

Computer Vision for Robotics (Vision Foundations) · Lesson 02

How Cameras See the World

Est. 50–65 min

Why

Why this lesson matters

Think of standing on a railway platform. The train coach right in front of you looks huge; the same-size coach at the far end of the platform looks tiny. Your eye did not measure the coaches wrongly — far things simply cover a smaller part of your view. A camera works the same way, and it does it with a very simple formula. In Lesson 1 we saw that a robot gets only numbers from a camera. In this lesson you learn where those numbers come from: light, a lens, a sensor of tiny pixels, and the pinhole model that maps a 3D point to a pixel (u, v).

Why should a robotics student care? Because every later step in Stage 11 depends on it. Converting a block’s pixel position into centimetres for IK (Stage 8), knowing how far one pixel is on the table, choosing a camera height and resolution, understanding why one camera cannot tell depth — all of it is the pinhole model. You will compute it by hand, check it against OpenCV, measure your own webcam, and build a one-pixel “camera” on an Arduino to feel what exposure time means.

Concept

From light to pixels: the pinhole camera

1. The camera obscura: a dark room with a small hole

Long before electronics, people noticed that light coming through a small hole into a dark room makes an image of the outside scene on the opposite wall — upside down and left-right reversed. This is the *camera obscura* (Latin for “dark chamber”). Every straight ray from a point outside passes through the hole and hits exactly one spot on the wall. A modern camera is the same idea: the hole became a lens (which gathers more light), and the wall became an electronic sensor.

Line drawing of a box camera obscura: light from a tree outside passes through a lens on the front, reflects off a mirror and forms an image on a glass screen on top
A box camera obscura drawing: light from the scene passes through a lens and makes an image that an artist can trace. A webcam is this box, made tiny, with a sensor in place of the screen.Photo: en:User:Meggar · CC BY-SA 3.0 · Source: Wikimedia Commons · resized, converted to WebP

2. The pinhole model: one formula for every pixel

We put the camera at the origin. Z points straight out of the lens, X to the right and Y downwards (this is the camera frame OpenCV uses). For a point (X, Y, Z) in front of the camera, the pixel where it appears is:

Pinhole projection (OpenCV calib3d, no distortion) · Text
u = fx · X / Z + cx
v = fy · Y / Z + cy

fx, fy = focal length in PIXELS
cx, cy = principal point (where the lens axis hits the image, near the centre)
Dividing by Z is the whole secret: double the distance and the object looks half as big.
SymbolMeaningEveryday picture
X, Y, Zposition of the point in the camera frame (for example cm)Z is “how far in front of the lens”
X / Z, Y / Zthe direction of the raythe angle at which you see the point
fx, fyfocal length measured in pixelszoom: bigger fx → objects look bigger
cx, cyprincipal point in pixelsthe image centre the lens really uses
u, vpixel column and row (top-left is 0, 0)where the point lands on the photo
Diagram of central projection: a 3D point, its straight ray through the centre of projection, and the point where the ray meets the image plane
Central projection, the idealised camera model: each 3D point is joined to the centre of projection by a straight ray, and the pixel is where that ray meets the image plane.Photo: Olaf Peters (OlafTheScientist) · CC BY-SA 4.0 · Source: Wikimedia Commons · resized, converted to WebP

3. Focal length in pixels, field of view and resolution

Lens boxes print focal length in millimetres, but vision code needs it in pixels. The OpenCV docs give both links: fx = focal_length_mm / pixel_size_mm, and from the field of view, f = sensor_width / (2 · tan(FOV / 2)). Working directly in pixels, the horizontal field of view of an image W pixels wide is FOV = 2 · atan(W / (2 · fx)). A 640-px-wide image with fx = 600 px sees about 56°; a wide-angle camera with fx = 320 px sees 90°.

If you change the resolution of the same camera — say from 640 × 480 to 1280 × 960 — fx, fy, cx and cy all scale by the same factor (×2 here). The ray for each object does not change; you just have more pixels across it. The OpenCV docs also note that lens distortion coefficients stay the same when resolution changes. So: always use intrinsics that match the resolution you are actually capturing.

4. Inside the camera: lens, sensor and colour filters

  1. Lens

    Bends light so many rays from one point meet again on the sensor — a bright, sharp image instead of the dim one a tiny pinhole gives. Real lenses also bend straight lines a little near the edges (barrel or pincushion distortion); Lesson 12 measures and removes that.

  2. Aperture and exposure

    Each pixel collects light for a short exposure time. Longer exposure = brighter, less noisy picture, but a moving hand smears into motion blur. Shorter = sharp but dark and grainy.

  3. Sensor

    A grid of light-sensitive photosites (CMOS in almost every webcam and phone). Each one turns the light it collected into a number.

  4. Colour filter (Bayer)

    A photosite cannot tell colour, so a mosaic of tiny filters sits on top: half green, a quarter red, a quarter blue (the Bayer pattern, invented by Bryce Bayer at Kodak). The camera fills in the two missing colours at each pixel from its neighbours (demosaicing).

  5. Readout

    Most cheap sensors read rows one after another — a rolling shutter — so a fast-moving object can look slanted or bent. A global shutter captures all pixels at the same instant; it is preferred for fast robots and conveyors.

Microscope view of a camera sensor showing a regular mosaic of tiny red, green and blue filter squares
An RGGB Bayer colour filter seen at ×80 on an old Sony camcorder sensor. Count them: there are twice as many green squares as red or blue.Photo: Binarysequence · CC BY-SA 3.0 · Source: Wikimedia Commons · resized, converted to WebP

Visual

Visualise it: move the block, change the focal length

Educational projection lab: focal length fx, principal point and distance Z decide where points land in the image. (Browser sim, not OpenCV, not a real lens.)

Interactive · Camera calibration lab · Educational

Camera calibration diagram

Intrinsics K map rays to pixels; bad fx / unmodeled distortion raise reprojection residual. Projection collapses Z — robots recover depth later. Estimates are educational, not a real OpenCV calibration runtime.

Code lab

Code lab: project points like a camera

pinhole_projection.py uses only NumPy. It takes a typical 640 × 480 webcam with fx = fy = 600 px and (cx, cy) = (320, 240) — example numbers, not a calibrated camera — and runs six small experiments. At the end, if OpenCV is installed, it checks the hand formula against cv.projectPoints (with zero rotation, zero translation and no distortion). Run python3 pinhole_projection.py.

pinhole_projection.py · Python
"""
pinhole_projection.py — Stage 11 Lesson 2: how a camera turns 3D points into pixels.

Uses the distortion-free pinhole model from the OpenCV calib3d docs:
    u = fx * X / Z + cx
    v = fy * Y / Z + cy
(X, Y, Z) are in the CAMERA frame: X right, Y down, Z forward out of the lens.

Needs only NumPy. If OpenCV is installed, the result is cross-checked with cv.projectPoints.
Run:  python3 pinhole_projection.py
"""
import math
import numpy as np

# A typical 640 x 480 USB webcam (example numbers, not a calibrated camera).
W, H = 640, 480
FX = FY = 600.0          # focal length in PIXELS
CX, CY = W / 2, H / 2    # principal point, near the image centre


def project(X, Y, Z, fx=FX, fy=FY, cx=CX, cy=CY):
    """3D point in camera frame (cm) -> pixel (u, v). Refuses points behind the lens."""
    if Z <= 0:
        raise ValueError(f"Z={Z}: point is behind (or on) the camera - not visible")
    return fx * X / Z + cx, fy * Y / Z + cy


def show(title):
    print("\n" + title)
    print("-" * len(title))


# 1. A 5 cm red block seen straight down from 50 cm.
show("1. A 5 cm block, camera 50 cm above the table")
left = project(-2.5, 0, 50)
right = project(2.5, 0, 50)
print(f"left edge  -> u={left[0]:6.1f} px")
print(f"right edge -> u={right[0]:6.1f} px")
print(f"block width in image = {right[0]-left[0]:.1f} px  (formula fx*5/50 = {FX*5/50:.1f})")

# 2. Same block, further away: size in pixels falls as 1/Z.
show("2. Move the camera away: width in pixels = fx * size / Z")
for Z in (25, 50, 100, 200):
    w = FX * 5 / Z
    print(f"Z={Z:4d} cm -> block is {w:6.1f} px wide, 1 px = {Z/FX*10:5.2f} mm on the table")

# 3. Depth ambiguity: different 3D points on one ray land on the same pixel.
show("3. One pixel = one ray (depth is lost)")
for Z in (30, 60, 120):
    X, Y = 0.2 * Z, -0.1 * Z     # every point with X/Z = 0.2 and Y/Z = -0.1
    u, v = project(X, Y, Z)
    print(f"point ({X:5.1f}, {Y:5.1f}, {Z:5.1f}) cm -> pixel ({u:.1f}, {v:.1f})")
print("A small near block and a big far block can look identical. Lesson 14 brings depth back.")

# 4. Field of view from fx, and fx from a datasheet FOV.
show("4. Field of view <-> focal length in pixels")
hfov = 2 * math.degrees(math.atan(W / (2 * FX)))
print(f"fx={FX:.0f} px, width={W} px -> horizontal FOV = {hfov:.1f} deg")
for fov in (60, 78, 90):
    fx = W / (2 * math.tan(math.radians(fov) / 2))
    print(f"if a camera had {fov} deg horizontal FOV at {W} px wide -> fx = {fx:.0f} px")

# 5. Changing resolution scales the intrinsics (OpenCV calib3d docs).
show("5. Same camera, 1280 x 960 instead of 640 x 480: scale fx, fy, cx, cy by 2")
u640 = project(5, 3, 50)
u1280 = project(5, 3, 50, fx=2 * FX, fy=2 * FY, cx=2 * CX, cy=2 * CY)
print(f"640x480 : ({u640[0]:.1f}, {u640[1]:.1f})")
print(f"1280x960: ({u1280[0]:.1f}, {u1280[1]:.1f})  = exactly 2x, same ray")

# 6. Things the camera cannot see.
show("6. Outside the picture or behind the lens")
for P in [(20, 0, 50), (40, 0, 50), (0, 0, -10)]:
    try:
        u, v = project(*P)
        inside = 0 <= u < W and 0 <= v < H
        print(f"{P} -> ({u:.1f}, {v:.1f}) {'inside image' if inside else 'OUTSIDE image (not in view)'}")
    except ValueError as e:
        print(f"{P} -> refused: {e}")

# 7. Cross-check with OpenCV (camera frame = world frame: rvec = tvec = 0, no distortion).
try:
    import cv2 as cv
    K = np.array([[FX, 0, CX], [0, FY, CY], [0, 0, 1]], dtype=np.float64)
    pts = np.array([[-2.5, 0, 50], [2.5, 0, 50], [5, 3, 50]], dtype=np.float64)
    img, _ = cv.projectPoints(pts, np.zeros(3), np.zeros(3), K, np.zeros(5))
    ours = np.array([project(*p) for p in pts])
    print(f"\nOpenCV projectPoints agrees with our formula: max difference = {np.abs(img.reshape(-1, 2) - ours).max():.6f} px")
except ImportError:
    print("\n(OpenCV not installed - skipped the cv.projectPoints cross-check)")

Output (run with python3 pinhole_projection.py, OpenCV 4.x installed)


1. A 5 cm block, camera 50 cm above the table
---------------------------------------------
left edge  -> u= 290.0 px
right edge -> u= 350.0 px
block width in image = 60.0 px  (formula fx*5/50 = 60.0)

2. Move the camera away: width in pixels = fx * size / Z
--------------------------------------------------------
Z=  25 cm -> block is  120.0 px wide, 1 px =  0.42 mm on the table
Z=  50 cm -> block is   60.0 px wide, 1 px =  0.83 mm on the table
Z= 100 cm -> block is   30.0 px wide, 1 px =  1.67 mm on the table
Z= 200 cm -> block is   15.0 px wide, 1 px =  3.33 mm on the table

3. One pixel = one ray (depth is lost)
--------------------------------------
point (  6.0,  -3.0,  30.0) cm -> pixel (440.0, 180.0)
point ( 12.0,  -6.0,  60.0) cm -> pixel (440.0, 180.0)
point ( 24.0, -12.0, 120.0) cm -> pixel (440.0, 180.0)
A small near block and a big far block can look identical. Lesson 14 brings depth back.

4. Field of view <-> focal length in pixels
-------------------------------------------
fx=600 px, width=640 px -> horizontal FOV = 56.1 deg
if a camera had 60 deg horizontal FOV at 640 px wide -> fx = 554 px
if a camera had 78 deg horizontal FOV at 640 px wide -> fx = 395 px
if a camera had 90 deg horizontal FOV at 640 px wide -> fx = 320 px

5. Same camera, 1280 x 960 instead of 640 x 480: scale fx, fy, cx, cy by 2
--------------------------------------------------------------------------
640x480 : (380.0, 276.0)
1280x960: (760.0, 552.0)  = exactly 2x, same ray

6. Outside the picture or behind the lens
-----------------------------------------
(20, 0, 50) -> (560.0, 240.0) inside image
(40, 0, 50) -> (800.0, 240.0) OUTSIDE image (not in view)
(0, 0, -10) -> refused: Z=-10: point is behind (or on) the camera - not visible

OpenCV projectPoints agrees with our formula: max difference = 0.000000 px

Try it: switch to projection mode, change fx and the board distance Z, and watch the projected points spread out or squeeze together. (Educational browser sim, not OpenCV.)

Interactive · Camera calibration lab · Educational

Camera calibration diagram

Intrinsics K map rays to pixels; bad fx / unmodeled distortion raise reprojection residual. Projection collapses Z — robots recover depth later. Estimates are educational, not a real OpenCV calibration runtime.

camera_check.py — measure your own webcam

Every webcam is different, and drivers do not always give the resolution you ask for. camera_check.py asks for a size with cap.set(cv.CAP_PROP_FRAME_WIDTH, …), then reads back what it really got, measures frames per second, and checks the last frame for crushed blacks and blown-out whites. With --fx-from it estimates fx from one simple measurement: hold an object of known width at a known distance, read its width in pixels from the saved image, and use fx = pixels × Z / size.

camera_check.py · Python
"""
camera_check.py — Stage 11 Lesson 2: get to know YOUR camera.

  python3 camera_check.py                      # webcam 0: asks for 640x480, reports what it really gives + FPS
  python3 camera_check.py --cam 1 --width 1280 --height 720
  python3 camera_check.py --image photo.jpg    # no camera? report on a saved photo
  python3 camera_check.py --fx-from 120 10 50  # object 10 cm wide, 50 cm away, looks 120 px wide -> fx

Follows the OpenCV "Getting Started with Videos" tutorial (VideoCapture, isOpened, read, set/get).
Press q in the window to stop early.
"""
import argparse
import math
import time

import cv2 as cv


def fx_from_measurement(width_px, size_cm, distance_cm, image_width):
    # Pinhole model: width_px = fx * size / Z  ->  fx = width_px * Z / size
    fx = width_px * distance_cm / size_cm
    hfov = 2 * math.degrees(math.atan(image_width / (2 * fx)))
    print(f"fx ~ {fx:.0f} px  ->  horizontal FOV ~ {hfov:.1f} deg at {image_width} px wide")
    print("Rough estimate only: Lesson 12 measures fx properly with a chessboard calibration.")


def report_image(img, label):
    h, w = img.shape[:2]
    print(f"{label}: {w} x {h} px, {img.shape[2] if img.ndim == 3 else 1} channel(s), dtype={img.dtype}")
    print(f"  principal point guess (cx, cy) = ({w/2:.0f}, {h/2:.0f})  (true value comes from calibration)")
    print(f"  mean brightness = {img.mean():.1f}/255")
    dark = (img.max(axis=2) if img.ndim == 3 else img) < 10
    bright = (img.min(axis=2) if img.ndim == 3 else img) > 245
    print(f"  crushed blacks = {dark.mean()*100:.1f}%   blown highlights = {bright.mean()*100:.1f}%")
    if bright.mean() > 0.05:
        print("  WARNING: >5% of pixels are pure white - too much light or exposure; detail is lost there")
    if dark.mean() > 0.30:
        print("  WARNING: >30% of pixels are near black - add light")


def run_camera(index, want_w, want_h, seconds):
    cap = cv.VideoCapture(index)
    if not cap.isOpened():
        print("Cannot open camera", index, "- close other apps using it or try --cam 1")
        return
    cap.set(cv.CAP_PROP_FRAME_WIDTH, want_w)
    cap.set(cv.CAP_PROP_FRAME_HEIGHT, want_h)
    got_w = int(cap.get(cv.CAP_PROP_FRAME_WIDTH))
    got_h = int(cap.get(cv.CAP_PROP_FRAME_HEIGHT))
    print(f"asked for {want_w}x{want_h}, camera gives {got_w}x{got_h}"
          + ("" if (got_w, got_h) == (want_w, want_h) else "  <- driver chose a supported size"))
    for _ in range(10):            # let auto-exposure settle
        cap.read()
    n, t0, frame = 0, time.time(), None
    while time.time() - t0 < seconds:
        ok, f = cap.read()
        if not ok:
            print("Can't receive frame. Exiting ...")
            break
        frame, n = f, n + 1
        cv.imshow("camera_check (q = quit)", f)
        if cv.waitKey(1) == ord("q"):
            break
    dt = time.time() - t0
    cap.release()
    cv.destroyAllWindows()
    if frame is not None:
        print(f"measured {n} frames in {dt:.1f} s = {n/dt:.1f} FPS (one frame every {1000*dt/max(n,1):.0f} ms)")
        report_image(frame, "last frame")
        cv.imwrite("camera_check_last.png", frame)
        print("saved camera_check_last.png")


if __name__ == "__main__":
    ap = argparse.ArgumentParser()
    ap.add_argument("--cam", type=int, default=0)
    ap.add_argument("--width", type=int, default=640)
    ap.add_argument("--height", type=int, default=480)
    ap.add_argument("--seconds", type=float, default=5)
    ap.add_argument("--image")
    ap.add_argument("--fx-from", nargs=3, type=float, metavar=("PIXELS", "SIZE_CM", "DIST_CM"))
    a = ap.parse_args()
    if a.fx_from:
        fx_from_measurement(*a.fx_from, image_width=a.width)
    elif a.image:
        img = cv.imread(a.image)
        if img is None:
            raise SystemExit(f"could not read {a.image}")
        report_image(img, a.image)
    else:
        run_camera(a.cam, a.width, a.height, a.seconds)
Run it · Shell
pip install opencv-python numpy
python3 camera_check.py                        # webcam 0, 5 seconds
python3 camera_check.py --width 1280 --height 720
python3 camera_check.py --image camera_check_last.png
python3 camera_check.py --fx-from 120 10 50    # 10 cm object, 50 cm away, 120 px wide
# fx ~ 600 px  ->  horizontal FOV ~ 56.1 deg at 640 px wide

Bench build: a one-pixel camera and exposure time

A photocell (LDR) is a camera with exactly one pixel. Each camera pixel collects light for an exposure time; we copy that by adding up many quick readings. one_pixel_exposure.ino takes “short” (1 reading), “medium” (8) and “long” (64) exposures, repeats each 20 times and prints the average, the noise (max − min) and the time per shot. Wiring is the voltage-divider method from the Adafruit photocell guide, the same as Lesson 1.

A photocell and a 10 kΩ resistor draw well under a milliamp — safe on the Arduino 5 V pin.
PartConnectionNote
Photocell (LDR)5 V → LDR → A1the “one pixel”
10 kΩ resistorA1 → resistor → GNDpull-down; reading goes up with more light
Optional: matchbox or black paper tubeover the LDR with a small holea real pinhole: now the pixel only sees one direction
Motors / servosnot usedif added later: separate 5–6 V supply, grounds joined, never from a GPIO
one_pixel_exposure.ino · Arduino C++
// one_pixel_exposure.ino — Stage 11 Lesson 2: a one-pixel "camera" and exposure time.
// A photocell (LDR) is a single light-sensitive pixel. A camera pixel collects light
// for an exposure time; here we copy that by ADDING UP many quick readings.
//   short exposure = 1 reading   -> fast, but noisy
//   long  exposure = 64 readings -> smooth, but slow (a moving hand gets "smeared")
// Wiring (Adafruit photocell method): 5V -> LDR -> A1 -> 10k -> GND.
// Sensors only: no motors or servos on this board. Never power a motor from a GPIO pin.

const int LDR_PIN = A1;
const int ROUNDS = 20;  // repeat each exposure 20 times to measure noise

// "Expose" for n readings spaced 1 ms apart, return the average (0..1023).
float expose(int n) {
  long sum = 0;
  for (int i = 0; i < n; i++) {
    sum += analogRead(LDR_PIN);
    delay(1);
  }
  return (float)sum / n;
}

void measure(int n) {
  float mn = 1023, mx = 0, total = 0;
  unsigned long t0 = millis();
  for (int r = 0; r < ROUNDS; r++) {
    float v = expose(n);
    if (v < mn) mn = v;
    if (v > mx) mx = v;
    total += v;
  }
  unsigned long perShot = (millis() - t0) / ROUNDS;
  Serial.print("exposure=");
  Serial.print(n);
  Serial.print(" readings  value=");
  Serial.print(total / ROUNDS, 1);
  Serial.print("  noise(max-min)=");
  Serial.print(mx - mn, 1);
  Serial.print("  time/shot=");
  Serial.print(perShot);
  Serial.println(" ms");
}

void setup() {
  Serial.begin(115200);
  Serial.println("one-pixel camera: short vs long exposure");
}

void loop() {
  measure(1);
  measure(8);
  measure(64);
  Serial.println("---");
  delay(1000);
}

Debugging

SymptomLikely causeFix
Asked for 1280 × 720 but got 640 × 480camera or driver does not support that sizeUse the size it reports; scale fx, fy, cx, cy to match
Objects look bigger/smaller than the formula sayswrong fx, or distance measured to the camera body instead of the lensMeasure Z to the lens front; estimate fx again; Lesson 12 calibrates properly
Straight table edge looks curved near the cornerslens distortionKeep the object near the image centre for now; undistort in Lesson 12
Fast hand looks slanted or bentrolling shutterSlow down, add light, or use a global-shutter camera for fast motion
Image very grainy and FPS lowtoo little light, long auto-exposureAdd a steady LED lamp, avoid relying on a flickering tube light
projectPoints error about array shape or typepoints not float64 N×3, or K not 3×3Use np.array(..., dtype=np.float64) as in the script
LDR value stuck near 0 or 1023resistor missing or on the wrong railCheck 5 V → LDR → A1 → 10 kΩ → GND

Real world

Cameras in real robots and real life

  • Phone and laptop cameras: a CMOS sensor with a Bayer filter behind a small lens — the same parts as a robot webcam. Sensors come in many sizes; bigger photosites collect more light, which helps in a dim lab.
  • Overhead camera for pick-and-place: fixing the camera straight above the table makes Z almost the same for every block, so pixel distance converts to centimetres with one number (Z / fx). That is why the Stage 11 capstone uses an overhead webcam.
  • Fast conveyors: factory vision cameras often use global shutters and short exposures with bright lights so moving parts are not blurred or slanted.
  • Pinhole cameras: a matchbox with a pin-prick hole and a piece of film is a real camera with no lens at all — it proves the pinhole model is not just maths.
Two CMOS image sensors side by side on a table: a large full-frame camera sensor and a much smaller sensor from a toy camera kit
A large full-frame camera sensor next to a small one from a hobby camera kit. Both are grids of photosites; the big one collects much more light per pixel.Photo: 根川孝太郎 (Kotaro Negawa) · CC BY-SA 3.0 · Source: Wikimedia Commons · resized, converted to WebP
A home-made pinhole camera built from a matchbox wrapped in black tape, with a film roll attached
A matchbox pinhole camera. No lens at all — just a tiny hole, a dark box and film. The image it makes follows the same projection formula as your webcam.Photo: Ian from Moscow Russia · CC BY 2.0 · Source: Wikimedia Commons · resized, converted to WebP

Think

Think about it

Think

You mount the webcam 50 cm above the table and see a 5 cm block as 60 px wide. Then you stack the block on a 20 cm box (so it is now 30 cm from the camera). How wide will it look, and what goes wrong if your code still assumes the table is 50 cm away?

Experiment

Try it in the lab

About 50 minutes. Python part needs NumPy (and optionally OpenCV); camera part needs a webcam, a ruler and an object of known width (a 10 cm scale or a book); bench part needs an Arduino UNO, a photocell and a 10 kΩ resistor.

Experiment

Pixels, distance and exposure

You will need

  • `pinhole_projection.py` (NumPy; OpenCV optional)
  • `camera_check.py` with `opencv-python` and a webcam
  • Measuring tape, an object of known width
  • Projection lab (in your browser)
  • Bench: Arduino UNO, photocell, 10 kΩ resistor, black paper or a matchbox

Steps

  1. Run `pinhole_projection.py`. Change `FX` to 400 and run again. What happens to the block width and the FOV?
  2. In Part 3, pick a point on the same ray at Z = 240 cm. Predict its pixel before running.
  3. Run `camera_check.py`. Note the real resolution and FPS. Repeat with the lights dimmed.
  4. Hold your 10 cm object at 40, 60 and 80 cm from the lens. Measure its width in pixels each time (open the saved image in any viewer) and compute fx with `--fx-from`. Are the three fx values close?
  5. In the projection lab, set Z near and far and watch how the same board changes size.
  6. Bench: run `one_pixel_exposure.ino`, compare noise for 1, 8 and 64 readings, then wave your hand during the long exposure.

Interactive · Camera calibration lab · Educational

Camera calibration diagram

Intrinsics K map rays to pixels; bad fx / unmodeled distortion raise reprojection residual. Projection collapses Z — robots recover depth later. Estimates are educational, not a real OpenCV calibration runtime.

A camera turns a ray direction into a pixel. Divide by distance and you get the image position; multiply by distance and you get back centimetres — but only if you know the distance. Longer exposure means less noise but more blur.

Knowledge check

Check your understanding

Knowledge check

1. In the pinhole model, u = …

2. A 5 cm block looks 60 px wide at 50 cm. At 100 cm it looks…

3. Why can one camera not tell a small near block from a big far block?

4. You switch the same camera from 640×480 to 1280×960. fx, fy, cx, cy should…

5. In a Bayer filter, which colour has the most filter squares?

6. A longer exposure time gives…

7. A fast propeller looks bent in a cheap webcam video. The likely cause is…

8. With fx = 600 px, how much table does one pixel cover at Z = 60 cm?

Mini assignment

Hands-on assignment

Mini assignment

Your webcam’s “ID card”

Measure your camera and write down the numbers every later Stage 11 lesson will use.

Deliverables

  • Real resolution and FPS from `camera_check.py`, in bright and dim light.
  • Three fx estimates (object at three distances) and their average; the matching horizontal FOV.
  • For your planned camera height above the arm’s table: block width in pixels for a 3 cm and a 5 cm block, and mm per pixel.
  • A sketch: camera, lens centre, image plane, a block, its ray and its pixel (u, v), with Z marked.
  • Optional (bench): a table of value and noise for 1, 8 and 64 readings from `one_pixel_exposure.ino`.

Do this, step by step

  1. Run camera_check.py in bright light and in dim light; write the real resolution and FPS each time.
  2. Hold a 10 cm object at 40, 60 and 80 cm, measure its width in pixels and run --fx-from for each.
  3. Average the three fx values and compute the horizontal FOV = 2·atan(W / (2·fx)), where W is the image width in pixels.
  4. For your camera height Z, compute mm per pixel = Z / fx and the pixel width of a 3 cm and a 5 cm block.
  5. Draw the camera, lens centre, image plane, a block, its ray and its pixel (u, v), and mark Z.

Stuck? Hints, one at a time

NoteTry the task first. Read one hint, try again, and only then read the next one.
  1. fx = (width in pixels × distance) / real width. A 10 cm object 120 px wide at 50 cm gives fx = 600 px.
  2. If the three fx values differ by more than about 10%, measure the distance from the lens, not from the back of the webcam.
  3. If the FPS drops in dim light, the camera is using a longer exposure — this is normal for auto-exposure webcams.

Challenges (pick by level)

  • 🟢 Beginner: Add a function cm_per_px(Z) to pinhole_projection.py and print it for Z = 30, 45 and 60 cm.
  • 🟡 Intermediate: Write back_project(u, v, Z) that returns (X, Y) — the inverse of project. Check that projecting then back-projecting gives the original point.
  • 🟠 Advanced: Project all 8 corners of a 5 × 5 × 5 cm cube 40 cm in front of the camera and draw the edges with cv.line on a black 640 × 480 image. Move the cube left and see perspective.
  • 🔴 Expert: Add radial distortion x″ = x′(1 + k1·r²) (from the OpenCV calib3d docs) with k1 = −0.2 and draw a straight grid. Do you get barrel or pincushion distortion?
  • ⚫ Research: Compare rolling-shutter and global-shutter sensors for a robot sorting items on a fast conveyor. Write half a page with sources.

Next step

You now know how light becomes a grid of pixels. In Lesson 3 — Images, Pixels & Color Spaces you will open that grid in Python: rows and columns, uint8 values, BGR vs RGB, grayscale, and why HSV makes colour detection much more robust to lighting.

Sources

Look back: Lesson 1: Why Robots Need Vision. Next: Lesson 3: Images, Pixels & Color Spaces. Related: Stage 11 — Computer Vision, Stage 13 — Perception, Planning & Manipulation.

Self-attested for now — keep artifacts in your lab notebook.

Complete

Today's goal

0 of 1 done today

0-day streak
Daily goal:lessons, quizzes or assignments a day

Mark this lesson complete when you have worked the experiment, knowledge check, and assignment (even roughly).

Stuck somewhere?

Ask a doubt about this lesson

Saved with this lesson's id and your email so we can reply. Privacy · or email contact@aihubstation.in