Image transformation and synthesis methods
First Claim
1. A method of providing data defining an image of a scene, the method comprising the steps of:
- (a) providing a plurality of starting pixel data elements, each said starting pixel data element incorporating data corresponding to illumination seen along a starting pixel ray vector associated with that starting pixel data element, each said starting pixel ray vector having a direction and an intercept on a locus; and
(b) forming said pixel data elements into a transform image including a plurality of direction matrices, each said direction matrix including pixel data elements associated with pixel ray vectors having directions parallel within a preselected direction tolerance range to a common ray direction.
2 Assignments
0 Petitions
Accused Products
Abstract
In a telepresence system, a scene is captured by recording pixel data elements, each associated with a pixel ray vector having a direction and an intercept on an known locus in the frame of reference of the scene. Each pixel data element includes data representing the illumination along the pixel ray vector. For example, the pixel data elements may be captured by operating numerous video cameras pointing in different directions on a spherical locus. A virtual viewpoint image representing the image which would be seen from an arbitrary viewpoint, looking in an arbitrary direction, can be synthesized by determining the directions of synthetic pixel ray vectors from each pixel of the virtual viewpoint image through the virtual viewpoint and the intercepts of these vectors on the locus. Recorded pixel data elements having pixel ray vector directions and intercepts close to those of the synthetic pixel ray vector can be copied or interpolated to provide data representing illumination in the synthetic pixel.
-
Citations
32 Claims
-
1. A method of providing data defining an image of a scene, the method comprising the steps of:
-
(a) providing a plurality of starting pixel data elements, each said starting pixel data element incorporating data corresponding to illumination seen along a starting pixel ray vector associated with that starting pixel data element, each said starting pixel ray vector having a direction and an intercept on a locus; and
(b) forming said pixel data elements into a transform image including a plurality of direction matrices, each said direction matrix including pixel data elements associated with pixel ray vectors having directions parallel within a preselected direction tolerance range to a common ray direction. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 30)
-
-
16. A method of providing a data set for depicting a scene including the steps of providing said pixel data elements by actuating a plurality of pixel sensing elements in a plurality of different cameras, each said camera having a viewpoint and a viewpoint direction, the method further comprising the steps of capturing images using at least some of said cameras and deducing the viewpoint and view direction of at least some of said cameras from said captured images.
-
17. A method of providing a data set for depicting a scene including the steps of including the steps of providing pixel data elements by actuating a plurality of pixel sensing elements in a camera and moving the camera between a plurality of positions, each said camera position having a viewpoint and a viewpoint direction, the method further comprising the steps of actuating the camera to capture images at least some of said positions and deducing the viewpoint and view directions for at least some of said positions from said captured images.
-
18. A method of modifying a data set defining a first time sequence of telepresence scenes, to provide an altered time sequence of telepresence scenes, the method comprising the step of altering the data in said set defining said first time sequence so that the alteration changes progressively.
-
29. A method of modifying a data set defining a first time sequence of telepresence scenes to provide an altered time sequence of telepresence scenes, said data set including a group of discrete images for each scene, each said discrete image including a plurality of pixel data elements, the method comprising the step of altering the data in said set defining said first time sequence by selecting one or more of said discrete images and altering the pixel data elements in each selected discrete image.
- 31. A method of displaying a time sequence of telepresence scenes comprising the steps of monitoring a viewpoint and view direction of an observer, synthesizing a sequence of virtual viewpoint images responsive to the monitored viewpoint and view direction of the observer from a data set defining said sequence of telepresence scenes, altering the virtual viewpoint images and displaying the altered virtual viewpoint images to the observer, said synthesizing, altering and displaying steps being performed in real time so that the altered virtual viewpoint image displayed at any time corresponds to the image which would be seen from a virtual viewpoint and view direction corresponding to the viewpoint and view direction of the observer at such time, transformed by said alteration.
Specification