Optical flow

Measure apparent motion in the image plane#

An optical-flow vector assigns a horizontal and vertical image displacement or velocity to an observed location. Dense methods estimate a vector for most pixels, while sparse methods follow selected features. Flow can result from camera motion, object motion, or both.

Horn and Schunck define optical flow as the distribution of apparent velocities of brightness patterns. Their method combines the image-brightness constraint with a smoothness assumption because one local brightness measurement cannot determine both components of a two-dimensional velocity vector.

A cue for robot motion and scene motion#

A mobile robot can use flow to help estimate ego-motion, avoid approaching obstacles, or identify motion inconsistent with the static scene. Manipulators can use it to track objects or visual features between frames. RAFT is a learned dense-flow method that constructs all-pairs feature correlations and iteratively updates a flow field with a recurrent unit.

Optical flow is not the same as physical object velocity. Texture, projection, depth, and camera movement determine how a three-dimensional motion appears in pixels. Scene flow instead estimates three-dimensional motion of observed scene points.

Image evidence is ambiguous#

The aperture problem makes motion along an edge locally ambiguous. Textureless areas provide little signal, while occlusion, reflections, changing illumination, motion blur, and repeated patterns can violate common estimation assumptions. Smoothness regularisation can fill gaps but can also smear motion across a real object boundary.

Learned estimators inherit limits from their training data and image resolution. Classical and learned methods can both return confident-looking fields in regions with no valid correspondence. A robot should combine flow with depth, inertial sensing, temporal consistency, or task-specific uncertainty checks before using it for safety-critical control.