Neural radiance field

Images supervise a continuous scene function#

The original NeRF method trains a neural network from images with known camera poses. The network predicts density and view-dependent colour at sampled points. A volume-rendering calculation combines samples along a ray to reproduce an image pixel.

Unlike a point cloud, the representation is queried continuously rather than stored only as a fixed list of measured points. Unlike 3D Gaussian splatting, a standard NeRF represents the field with a network rather than an explicit collection of Gaussian primitives.

Robotics uses extend beyond view synthesis#

A robot can use a radiance field as part of scene reconstruction, camera-pose refinement, synthetic-view generation, or object geometry estimation. These uses require additional processing because the original objective is photometric view synthesis, not collision checking or control.

Dex-NeRF is one manipulation example. Its authors derive depth information from a NeRF and feed it to a grasp planner for transparent objects, which are difficult for common depth cameras. They report physical experiments with an ABB YuMi in a multi-camera workcell. That setup is evidence for the researched pipeline, not a claim that an arbitrary NeRF supplies accurate robot geometry.

Rendering quality is not geometric certainty#

Training needs multiple observations and sufficiently accurate camera poses. Changes in lighting, moving objects, sparse viewpoints, reflections, and scene motion can violate assumptions or introduce artefacts. Classic NeRF training and rendering can also be too slow for a control loop, although later methods trade quality, memory, and speed in different ways.

Density is learned to explain images and does not automatically encode a watertight surface, free space, semantics, or uncertainty. A robot that plans around a NeRF still needs to decide how to extract geometry and how much error its task can tolerate.