| Summary |
We present a system to virtually restore damaged or historically significant objects without needing to physically change the object in any way. This work addresses both creating a restored synthetic version of the image as viewed from a camera and projecting the appropriate light patterns using digital projectors to give the illusion of the object being restored. The restoration algorithm minimizes an energy metric which enforces a set of contour criteria over the surface of the object. The visual compensation method develops a formulation that is designed to obtain bright compensations under a specified maximum amount of light.
Our system contains three steps to virtually restore an object. In an initial acquisition step, a 3D model of the object to be restored is acquired on our object acquisition stage. The stage itself consists of a computer, camera, and multiple projectors. Second, a photograph of the object is used to generate a synthetic image of a plausible appearance of the restored object. Finally, a visual compensation step determines an appropriate combination of the available projectors to generate a light-efficient compensation of the synthetically restored appearance from the second step. |
| 1. Object Acquisition |
| In the first step of our restoration process, we acquire a 3D model of the object to be restored as well as estimate the internal and external parameters of the projectors. The projectors must also be radiometrically calibrated. Our acquisition approach is based on Aliaga and Xu [2008] which performs a self-calibrating photo-geometric acquisition of the object and calibration of the projectors. Our final acquired models are of high resolution, with less than 1mm between point samples on the surface of the object. |
 |
| 2. Image Restoration |
Using a photograph of the object, a synthetic restoration of the object is performed. This process involves two steps. First, a color classification routine is employed to determine the original colors of the object as well as assign each pixel of the photograph to one of the discovered set of colors. This routine reduces the number of colors of the object to a small finite set and greatly assists in restoring the shapes of the color patches. Second, an energy minimization routine works to reshape the identified color patches. This iterative routine works to smooth out the color patches and their relationships with neighboring color patches.
In the image below, (a) shows a portion of a deteriorated object. b) and c) show the synthetic equivalent before and after restoration. d) shows the final restored image of a). |
 |
| 3. Visual Compensation |
Once a synthetic appearance of the restored object has been generated, the projectors need to project this appearance onto the physical object. Since excessive light intensity onto the object may further damage the object, selection of which projectors to illuminate different parts of the object is critical. In general, if a projector more directly illuminates a portion of the object, that projector should be depended on more heavily to illuminate that surface area. Similarly, if a projector is at a grazing angle to a portion of the object's surface, it should not contribute much light to that particular surface area. While simply selecting the optimally oriented projector per surface point of the object would achieve this goal, the resulting visual compensation would contain discontinuities due to adjacent surface points being illuminated by different projectors. Instead, we use a weighted balance of the given projectors to achieve a smooth compensation, emphasizing the optimally oriented projectors across the surface of the object.
In the image below, a) and b) show projector combination schemes which provide a sub-optimal appearance (discontinuities and too dim, respectively) while c) demonstrates our approach to balancing the contribution of our projectors. |
 |
|
