Image transformation and synthesis methods

• US 20020110275A1
• Filed: 04/05/2002
• Published: 08/15/2002
• Est. Priority Date: 12/29/1994
• Status: Abandoned Application

• Current Assignee: Worldscape, Inc.
• Original Assignee: Worldscape, Inc.
• Inventors: Macintosh, David, Rogina, Peter, Ehrlich, Michael
• Application Number:
  US10/117,675
• Publication Number:
  US 20020110275A1
• Time in Patent Office: Days
• US Class Current: 382/154
• CPC Class Codes: G06T 15/10 Geometric effects
• CPC Class Codes: G06T 2207/20228 Disparity calculation for image-based rendering
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: H04N 13/117 the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
• CPC Class Codes: H04N 13/167 Synchronising or controlling image signals
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/194 Transmission of image signals
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays
• CPC Class Codes: H04N 13/366 using viewer tracking
• CPC Class Codes: H04N 13/398 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding

Abstract

In a telepresence system, a scene is captured by recording pixel data elements, each associated with a pixel ray vector having a direction and an intercept on an known locus in the frame of reference of the scene. Each pixel data element includes data representing the illumination along the pixel ray vector. For example, the pixel data elements may be captured by operating numerous video cameras pointing in different directions on a spherical locus. A virtual viewpoint image representing the image which would be seen from an arbitrary viewpoint, looking in an arbitrary direction, can be synthesized by determining the directions of synthetic pixel ray vectors from each pixel of the virtual viewpoint image through the virtual viewpoint and the intercepts of these vectors on the locus. Recorded pixel data elements having pixel ray vector directions and intercepts close to those of the synthetic pixel ray vector can be copied or interpolated to provide data representing illumination in the synthetic pixel.

Claims

1. A method of providing data defining an image of a scene, the method comprising the steps of:

(a) providing a plurality of starting pixel data elements, each said starting pixel data element incorporating data corresponding to illumination seen along a starting pixel ray vector associated with that starting pixel data element, each said starting pixel ray vector having a direction and an intercept on a locus; and

(b) forming said pixel data elements into a transform image including a plurality of direction matrices, each said direction matrix including pixel data elements associated with pixel ray vectors having directions parallel within a preselected direction tolerance range to a common ray direction.