Light-plane triangulation

Active instead of passive

Stereo with two cameras is passive: it works with whatever light the scene reflects, and the hard part is correspondence, finding the same point in both images. On a white wall there is nothing to match. Active methods add light of known geometry. A projector is an inverse camera: instead of receiving a ray through each pixel, it sends one out. If we know which projector pixel lit a camera pixel, the two rays can be intersected exactly as in stereo, and the projector supplies the texture.

The light plane

A laser line, or one column of a projector, lights a plane through the projector center. After calibration, its equation in camera coordinates is known:

A bright pixel at $(x', y')$, measured from the principal point, sees the camera ray $x = x' z / f$, $y = y' z / f$. Substituting it into the plane equation gives

and the 3D point is $(x' z / f,\ y' z / f,\ z)$. One frame gives one profile: the curve where the plane cuts the surface. Moving the plane across the scene, by turning a mirror or moving the object on a turntable, gives the whole surface, one profile per frame.

Finding the stripe

The laser has to be found in every row. Subtracting an image taken with the laser off removes the scene's own shading; then each local maximum above a threshold is a stripe. A parabola through the maximum and its two neighbors places the peak to a fraction of a pixel: for values $a, c, b$ the center is shifted by $\delta = (a - b) / (2(a - 2c + b))$. Where the projector cannot reach, in the shadows to the left of the sphere and the box, there is no stripe, and no depth.

Speed vs robustness

One stripe per frame is slow: $N$ profiles need $N$ frames, and the scene must not move in between. Projecting $K$ stripes at once needs only $N/K$ frames, but now each peak must be matched to one of the $K$ planes. The obvious rule is to count from the left: the projector is to the right of the camera, and on a surface without occlusions the stripes keep their order. Where a stripe is missing, in a shadow or behind an object, every stripe to its right is assigned to the wrong plane, and its points end up far from the surface.

There are two ways out. Multi-frame patterns give every projector column a code that is spread over a few frames, such as binary and Gray codes: $n$ frames tell $2^n$ columns apart. Single-frame patterns make every part of one image unique, with stripes in a color sequence that never repeats or with a random dot pattern; they work on moving scenes, but a local patch has to be decoded correctly, which fails more easily on colored or strongly slanted surfaces.

Try this

• Press Play with one stripe per frame, and watch the profiles appear in the 3D view and the depth image. Orbit the 3D view to look along a light plane.
• Click the stripe on the sphere, on the box and on the wall, and compare the plane equation and the computed depth $z$ with the true depth.
• Switch to 4, 12 and 48 stripes per frame, which shows the complete sweep: fewer frames, but the share of wrong points grows. Look for the rows where they start: just right of a shadow.
• Make the baseline small: the shadows shrink, and with them the wrong points, but a small baseline also makes the rays meet at a small angle, as in stereo.
• Raise the noise and the threshold, or make the stripes wide: weak stripes on the far, grazing parts of the table get lost, and each one lost shifts all stripes to its right.