Optical flow
Many locations, one nearby frame pair.
frame 1 → frame 2
Output: a motion field over many image locations. Here the dense idea is shown sparsely with arrows.
Optical Flow vs Feature Tracking
First distinguish the two tasks. Then, once a reference feature patch is known, compare three ways to locate the corresponding patch in the next frame.
1 · Optical flow vs. feature tracking
Optical flow estimates frame-to-frame motion at many image locations. Feature tracking follows selected distinctive image features across multiple frames to form trajectories.
Feature tracking
One selected feature, followed repeatedly over time.
trajectory through frame 1
Output: a trajectory with persistent identity. Feature tracking is a common form of point tracking where the points are selected for distinctive local appearance.
Transition: once a reference feature patch is selected in frame t, how do we locate the corresponding patch in frame t+1?
2 · Three ways to locate the reference patch
The background image stays fixed in all three panels so that only the search process moves. In the multi-scale panel, each pyramid level is shown at its actual image size inside the same 640 × 408 canvas.
Global template matching
Evaluate candidate locations across the whole image.
zig-zag scan
Global search: the candidate window scans the full image in a zig-zag pattern. This makes few assumptions about target motion, but requires many comparisons.
Multi-scale template matching
Use an image pyramid: coarse, low-resolution search first; then refine only nearby at finer levels.
L2 · 160 × 102
Coarse → fine: the actual pyramid image size changes across levels. Both the image and the template window are smaller at low resolution, then grow as the search moves to finer levels.
Local refinement / LK alignment
Start from an initial guess and refine the alignment iteratively.
initial guess p₀
Local refinement: the orange dashed box is the initial guess. The offset is exaggerated here so the iterations are visible; in practice LK assumes the residual motion is small. Each update moves the patch closer to the final alignment.
current candidate / initial guessintermediate refinementfinal matchcoarse-level search region
Why LK here? Feature tracking gives a reference patch and a good initialization from the previous frame. That makes local refinement much cheaper than searching the whole image.