====== Model Camera: Real-Time Modeling from Dense Color and Sparse Depth ====== ^ **Summary** ^ | The ModelCamera is a fast, easy to use, and inexpensive 3D scene modeling system. The ModelCamera acquires dense color (720x480 video frames) augmented with sparse depth (7x7 to 11x11 depth samples). The frames are registered and merged into an evolving model at the rate of five frames per second. The model is displayed continually to provide immediate operator feedback. | ^ **Structured Scene Modeling** ^^^ | Structured scenes (e.g. large pieces of furniture, walls, ceiling, and floor) are acquired freehand. The operator holds the ModelCamera in his/her hand and sweeps the scene. The frames are registered using the depth and color data. Registered frames are merged into a set of depth images created on demand. ||| | {{:wiki:projects:model_camera:image002.jpg|}} \\ Freehand ModelCamera prototype consists of a video camera \\ and a laser source whose beam is split in a 7x7 pattern \\ with a diffraction grating. Total cost $2,000. | {{:wiki:projects:model_camera:image003.jpg|}} \\ Real-time freehand modeling with operator feedback. | {{:wiki:projects:model_camera:image004.jpg|}} \\ Depth image placement shown with green dots (left) and depth image modeling (right). | ^ **Unstructured Scene Modeling** ^^^ | Unstructured scenes are acquired using a parallax-free pan-tilt bracket. Frames are stitched together using color to form a cube map. The depth samples are projected onto the cube map and triangulated in 2D. The connectivity information is applied to the depth samples to form a depth enhanced panorama (DEP), which is a texture-mapped 3D triangle mesh. A DEP supports view point translation while it maintains the advantages of color panoramas such as fast acquisition, inexpensive acquisition device, and interactive photorealistic rendering of the acquired scene. Several DEPs are combined on the GPU for better scene coverage. ||| | {{:wiki:projects:model_camera:image005.jpg|}} \\ Parallax-free pan-tilt bracket. | {{:wiki:projects:model_camera:image006.jpg|}} \\ Real-time parallax free modeling and operator feedback. | {{:wiki:projects:model_camera:image007.jpg|}} \\ Model of plant and room acquired in 5 and 15 minutes, respectively. | ^ **Large-Scale Modeling** ^^^ | We have extended the ModelCamera system with shaft encoders, a more powerful laser and a custom 11x11 diffraction grating to model large indoor environments. A team of two operators and a single acquisition device acquired in 40 hours a model spanning 1,400 m2 of floor space. The model covers corridors and 20 individual rooms over six floors of a large building on our campus. The model is assembled from room and I, L, and T corridor sections. A section is modeled by fitting proxy geometry to one or two DEPs. Corridor sections have high geometric detail where needed. Relying on the immediate feedback, the operator avoids over-sampling simple geometry. The resulting model supports realistic walkthroughs of the large indoor environment. ||| | {{:wiki:projects:model_camera:image008.jpg|}} \\ Large-scale indoor model assembled from room (blue) and corridor sections (orange). | {{:wiki:projects:model_camera:image009.png|}} \\ I, L, and T corridor sections. | {{:wiki:projects:model_camera:image011.jpg|}} \\ Corridor sections have embedded geometric detail where needed (water fountain here). | ^ **Models** ^^ ^ {{:wiki:projects:model_camera:models:image001.jpg?450|}} ^ [[http://www.cs.purdue.edu/cgvlab/modelCamera/gallery/vrml_409_1/vrml.wrz|Math 409 Office]] \\ \\ 220 519 points \\ ~420 000 triangles \\ Capture time: < 15 minutes \\ VRML size: 9 MB \\ \\ For comparison: the [[http://www.cs.purdue.edu/cgvlab/modelCamera/gallery/vrml_409_1_color_cubic_panorama/vrml_cube.wrl|same model]] without any geometry - the cubic color panorama only. \\ Notice the effect of translation away from original point of capture. ^ ^ {{:wiki:projects:model_camera:models:image003.jpg?450|}} ^ [[http://www.cs.purdue.edu/cgvlab/modelCamera/gallery/vrmlRoomTape/vrml.wrl|Math 409 after carpet installation]] \\ \\ VRML size: 15 MB ^ ^ {{:wiki:projects:model_camera:models:image005.jpg?450|}} ^ [[http://www.cs.purdue.edu/cgvlab/modelCamera/gallery/vrml_rose/vrml.wrz|Rose]] \\ \\ 10951 points \\ ~21 700 triangles \\ Capture time: < 4 minutes \\ VRML size: 0.5 MB ^ ^ {{:wiki:projects:model_camera:models:image007.jpg?450|}} ^ [[http://www.cs.purdue.edu/cgvlab/modelCamera/gallery/vrml_flower2/vrml.wrz|Flower on the table]] \\ \\ 30 568 points \\ ~59 000 triangles \\ Capture time: < 5 minutes \\ VRML size: 1.3 MB ^ ^ {{:wiki:projects:model_camera:models:image009.jpg?450|}} ^ [[http://www.cs.purdue.edu/cgvlab/modelCamera/gallery/vrml_cs146/vrml.wrz|CS 146]] \\ \\ 169 083 triangles \\ Capture time: 20 minutes \\ VRML size: 4.11 MB ^ ^ **Publications** ^ | | ^ **People** ^ | |