| Summary |
| One advantage of image-based rendering is scene-independent rendering cost. However, rendering a scene with a single reference depth image produces disocclusion errors: samples are lacking for surfaces that are visible in the desired view, but not in the reference view. Even small view changes produce noticeable errors. We introduce occlusion cameras, a class of non-pinhole cameras whose rays reach around occluders to gather samples that are hidden in the reference view but are likely to become visible in nearby views. These initially hidden samples alleviate disocclusion errors. Like regular depth images, occlusion-camera images have a single layer thus the number of samples they contain is bounded by the image resolution and connectivity is defined implicitly. Occlusion cameras provide fast projection so reference images are constructed efficiently with the hardware accelerated feed-forward pipeline. |
| Single Pole Occlusion Camera (SPOC) |
| The SPOC is obtained by applying a radial 3D distortion to the rays of the reference view. The distortion is centered at the occluder. A single SPOC covers the entire silhouette of the occluder, whereas many planar pinhole cameras are needed to achieve the same effect. Given a scene triangle and an SPOC, a closed form undistortion expression from the image plane to the triangle plane allows rasterization in the distorted domain, which is implemented on the GPU. |
 
SPOC images constructed on the GPU at 11 and 3fps. | 
Samples contributed by SPOC highlighted in pink. | 
Comparison between using a depth image, an SPOC image, and original geometry. |
| Depth Discontinuity Occlusion Camera (DDOC) |
| The DDOC is defined by the reference view and the geometry it encompasses. The reference planar pinhole camera is 3D distorted at depth discontinuities. The distortion is fine-grain controlled with a distortion map, which allows handling complex scenes. The camera model is customized for every reference view. The DDOC enables fast projection of 3D points, so the reference image is constructed efficiently with the feed-forward graphics pipeline. The DDOC reference image provides a high-quality, bounded-cost approximation of complex scenes. |
DDOC reference image. | 
Frames rendered from depth image and DDOC image |
| Epipolar Occlusion Camera (EOC) |
| The previous occlusion cameras were effective at reducing disocclusion events, but were unable to provide a set of viewpoints for which a valid reconstruction could take place. The epipolar occlusion camera (EOC) overcomes this disadvantages. An EOC is a non-pinhole which gathers samples of a 3D scene visible from a segment of viewpoints. The EOC is constructed by expressing the disocclusion events in the (2D) image as the sum of independent disocclusion events along (1D) epipolar lines. The EOC image has a single layer, is non-redundant, and is constructed efficiently by directly rendering the 3D scene with the EOC in feed-forward fashion, through projection followed by rasterization. |
EOC image supporting horizontal camera translation |  
 
Comparison of samples collected when using 2 PPC images (top) and a single EOC image (bottom) |
| 3D Display Rendering Acceleration |
| Volumetric 3D displays allow the user to explore a 3D scene free of joysticks, keyboards, goggles, or trackers. For nontrivial scenes, computing and transferring a 3D image to the display takes hundreds of seconds, which is a serious bottleneck for many applications. The OCRI is a compact scene representation that stores only and all scene samples that are visible from a viewing volume centered at a reference viewpoint. The OCRI enables computing and transferring the 3D image an order of magnitude faster than when the entire scene is processed. When compared with a depth image, an OCRI provides much improved scene coverage with no increase in rendering cost on the 3D display. |
  
Photos of 3D display bunny model viewed from the reference point rendered with a Depth Image, an OCRI, and the Real Geometry respectively. |  
Photo of 3D display bunny model viewed from 4” left of the reference point rendered with a Depth Image and an OCRI respectively. |
|
