| Summary |
| A desired target appearance may not always be possible given an appearance editing setup. This leads to errors in the resulting appearance (e.g., color shifting) which can lead to potentially misleading visualizations. To achieve more compliant appearances, we modify either (or both) the projector placement or the target appearance (in progress). |
| Luminance Compliancy for Virtual Restoration |
For our virtual restoration project, we considered luminance compliancy in order to ensure that for any point on the surface, the modified color was darker than that of the surface point's albedo. This was done for two reasons: first, in general, a lower luminance color is more easily achieved than a higher luminance color (this is, however, somewhat naive thinking without considering the chromaticity of the colors involved), and second, a lower luminance color results in less projector radiance shined on the given artifact, thus reducing the risk of damaging the artifact.
In the figure below, we show a didactic example of an original set of color patches to modify. The non-compliant solution is on the left, where the red circles indicate regions where the luminance was increased. On the right is an example of a compliant result of our algorithms where all color changes result in lower luminance colors.
For more details on our virtual restoration system, see our virtual restoration page. |
 |
| Optimal Projector Placement for High Quality Appearance Editing Visualizations |
Traditionally, projector placement for an appearance editing system has been ad hoc and informal – most methods manually attempt to maximize the illumination coverage of the projectors over the object surface. However, with such a naive attempt at projector placement, the compensation compliancy of the appearance edit, or how well the desired visualization is achieved, may be sacrificed. Additionally, appearance editing systems may want use a minimal amount of light radiance (i.e., be light radiance efficient). This can be important for applications such as virtual restoration where exposure of excess light may damage the cultural heritage. Various other practical issues can also be important in an appearance editing visualization system. For example, ensuring a clear path for the viewer to observe the visualization requires additional care in the placement of projectors.
In our formulation for compensation compliancy, we consider the projector's positions as well as the projector-surface orientation and surface albedo. These considerations are factored into a system of equations which we use in an adaptive algorithm to discover optimal positions for the projectors. We define an optimal projector placement as one which achieves compensation compliancy while being light radiance efficient and while considering other practical issues. Unlike our work in virtual restoration, our work in this project considers per-RGB channel compliancy (as opposed to considering only luminance). Evaulating the projector light radiance available for all RGB channels more accurately describes the ability of the given projector light to generate a target appearance. For example, consider a pure red surface albedo color attempting to be edited to a pure green target albedo color. In general, the green color is difficult to achieve due to insufficient energy at the green wavelengths underneath the red color's spectral response curve. This may be the case even if the green color is fundamentally less bright than the red color (i.e., it is still luminance compliant). |
 |
| In the figure below, we show a result from our simulation system. With the target appearance and idealized albedo of the objects shown in a) and b), ad hoc projectors achieve the appearance seen in c). d) shows the compensation compliancy of c), with blue colors being compliant and magenta colors being incompliant. e) shows the light radiance efficiency of c), with green colors being light radiance efficient and red colors being light radiance inefficient. f-g) show the same, but with our discovered optimized projector locations. It is clear our result better matches the appearance in a) with improved compliancy while being more light radiance efficient. |
 |
| In this real world example, we apply our discovered projectors in a virtual restoration system. a) shows a photograph of the original, deteriorated cultural heritage. b) shows a photograph an example of a standard virtual restoration, and c) shows a photograph with the left artifact with annotations and the right with a visualization detailing the amount of deterioration suffered by the artifact (blue to yellow in increasing amount of deterioration). d) shows photographs of two projector placements, the left an ad hoc set of projectors and the right our optimized set of projectors. e) shows a zoom-in comparison of the 'chipped' word on the left artifact. Our optimized projector positions produce a more uniform yellow text. f) shows a zoom-in of a portion of the right artifact. Our optimized projector positions remove the blotchy appearance for a more uniform blue color. |
 |
| In this real world example, we apply our algorithms for a weather map visualization of California. a) shows a photograph of the physical map. b) is the target visualization we wish to achieve, and c) is a photograph of the resulting visualization by one projector. d-e) show two zoom-in locations of the visualization. f-g) show photographs of these locations with our optimized projector location, and h-i) show photographs of the same locations using an ad hoc projector location. For this particular example, we constrained the system to avoid placing a projector directly in front of the map. |
 |
| Perceptual Appearance Modifications to Achieve Compliant Appearances |
While optimizing projector locations may yield a compliant appearance edit, even optimally placed projectors may be incapable of supplying sufficient projector light radiance in order to achieve an appearance edit. In such a case, the only way to achieve a compliant appearance is to modify the target appearance itself (unless additional projectors are added to the setup to supply additional projector light radiance). In this work, we provide an appearance modification method which minimally alters the target appearance's colors in order to produce a perceptually similar compliant appearance. We operate in CIELAB space (Lab-space, informally) and consider both the chrominance and luminance of a target appearance's colors. Further, we attempt to maintain the color ratios of the different colors of the target appearance to maintain color constancy. If color constancy is preserved, then the perceived difference between our modified appearances and the target appearances may appear even more similar. Our results show that in comparison with compliant appearances generated by only reducing an appearance's compliancy, our approach yields compliant appearances which are both brighter and more perceptually similar to the original target appearance (as compared by ΔE).
A checkerboard pattern was imparted onto a pedestal object a) as a didactic example b). Our resulting appearance edit is shown in c). d) shows a naive application of the target appearance, clearly showing incompliancy. e) shows a compliant appearance achieved by only reducing the target appearance's luminance. f-g) show zoom-ins of the photographs. |
 |
| These visualizations show the improvement obtained by using our compliant appearances (blue-cyan-green-yellow-red in increasing difference). On the left, we compare the change in chrominance (i.e., a color shift) between our resulting appearance a) and a naive application of the target appearance b). This visualization clearly shows that the chrominance achieved in our compliant appearance is more chromatically similar than the color shift due to the appearance being incompliant. On the right, we compare the brightness of our compliant appearance c) against the brightness of a compliant appearance achieved by only reducing luminance d). Both appearances were compared against the original target appearance, and the visualizations show that our compliant appearances are overall brighter than that possible by only reducing luminance until compliancy is achieved. |
 |
| Here, we modify our pedestal object with a more complex albedo a) and attempt to impart an argyle pattern onto it b). Our resulting compliant appearance is shown in c). d) shows b) naively imparted onto the object; clear ghosting and bleeding of the underlying surface albedo is present, particularly in the lower half of the object. e) shows the resulting appearance when only luminance is considered to achieve compliancy. |
 |
| We modified another object with a flower pattern. a) shows the physical object and b) the target appearance. c) shows an appearance edit using our modified compliant appearance, d) shows an appearance naively using the target appearance, and e) shows the appearance achieved when only reducing luminance to achieve compliancy. f-g) show zoom-ins on the photographs. |
 |
| Our approach can also be applied to multi-object scenes. The key difference with a multi-object scene is that color constancy can be more loosely maintained with color patches which are farther apart. A photograph of the objects is shown on the left. a) shows a synthetic image of the target appearance, b) shows our resulting appearance edit, c) shows a compliant appearance edit achieved by only reducing the luminance of the appearance, and d) shows the resulting appearance with a naive application of the target appearance. |
 |
|
