Automatic Scene Modeling for the 3D Camera and 3D Video
First Claim
1. A method for automatically segmenting a sequence of two-dimensional digital images into a navigable 3D model, said method including:
- a) capturing image sequences and defining nearer matte layers and/or depth maps based on proportionately greater lateral motion;
b) generating a wireframe surface for background and foreground objects from the raw video data which has been captured and processed in step (a);
c) giving depth to foreground objects using either;
silhouettes from different perspectives, center spines that protrude depthwise in proportion to the width up and down the object, or motion parallax information if available;
d) texture mapping the raw video onto the wireframe;
e) filling in occluded areas behind foreground objects, both on the background and on sides that are out of view, by stretching image edges in to the center of blank spots; and
f) sharpening surface images on nearer objects and blurring more distant images to create more depth perception, using either existing video software development kits or by writing image processing code that implements widely-known convolution masks, thereby automatically segmenting an image sequence into a 3D model.
0 Assignments
0 Petitions
Accused Products
Abstract
Single-camera image processing methods are disclosed for 3D navigation within ordinary moving video. Along with color and brightness, XYZ coordinates can be defined for every pixel. The resulting geometric models can be used to obtain measurements from digital images, as an alternative to on-site surveying and equipment such as laser range-finders. Motion parallax is used to separate foreground objects from the background. This provides a convenient method for placing video elements within different backgrounds, for product placement, and for merging video elements with computer-aided design (CAD) models and point clouds from other sources. If home users can save video fly-throughs or specific 3D elements from video, this method provides an opportunity for proactive, branded media sharing. When this image processing is used with a videoconferencing camera, the user'"'"'s movements can automatically control the viewpoint, creating 3D hologram effects on ordinary televisions and computer screens.
-
Citations
68 Claims
-
1. A method for automatically segmenting a sequence of two-dimensional digital images into a navigable 3D model, said method including:
-
a) capturing image sequences and defining nearer matte layers and/or depth maps based on proportionately greater lateral motion; b) generating a wireframe surface for background and foreground objects from the raw video data which has been captured and processed in step (a); c) giving depth to foreground objects using either;
silhouettes from different perspectives, center spines that protrude depthwise in proportion to the width up and down the object, or motion parallax information if available;d) texture mapping the raw video onto the wireframe; e) filling in occluded areas behind foreground objects, both on the background and on sides that are out of view, by stretching image edges in to the center of blank spots; and f) sharpening surface images on nearer objects and blurring more distant images to create more depth perception, using either existing video software development kits or by writing image processing code that implements widely-known convolution masks, thereby automatically segmenting an image sequence into a 3D model. - View Dependent Claims (4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 18, 19, 20, 21, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62)
-
-
2. The method for taking non-contact measurements of objects and features in a scene based on unit measures of 3D models generated from digital images, for engineering, industrial and other applications, whereby:
-
a) once the X, Y and Z coordinates have been defined for points or features, routine mathematics can be used to count or calculate distances and other measures; b) if measures, data merging or calibrating are needed in a particular scale, users can indicate as few as one length for a visible reference object in a software interface, and XYZ coordinates can be converted to those units; and c) an interface can allow the user to indicate where measurements are needed, and can show the resulting distances, volumes, or other measures. - View Dependent Claims (14, 63, 64, 65, 66, 67, 68)
-
- 3. The method for controlling navigation and viewpoint in 3D video, 3D computer games, object movies, 3D objects and panoramic VR scenes with simple body movement and gestures using a web cam to detect foreground motion of the user, which is then transmitted like mouse or keyboard inputs to control the viewpoint or to navigate.
-
39. A method for automatically segmenting a two-dimensional image sequence into a 3D model, said method including:
-
a) a video device used to capture images having two-dimensional coordinates in a digital environment; and b) a processor configured to receive, convert and process the two-dimensional images that are detected and captured from said video capturing device;
said system generating a point cloud having 3D coordinates from said two-dimensional images, defining edges from the point cloud to generate a wireframe having 3D coordinates, and adding a wiremesh to the wireframe to subsequently texture map the image from the video capturing device onto the wiremesh to display said 3D model on a screen. - View Dependent Claims (40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50)
-
Specification