Depth control for stereoscopic images

• US 6,160,909 A
• Filed: 04/01/1998
• Issued: 12/12/2000
• Est. Priority Date: 04/01/1998
• Status: Expired due to Term

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Melen, Roger D.
• Primary Examiner(s): Hjerpe, Richard A.
• Assistant Examiner(s): Stevenson, Philip H.
• Application Number:
  US09/053,191
• Time in Patent Office: 986 Days
• Field of Search: 345/419, 345/427, 345/438, 382/154, 382/276, 382/293, 348/295, 348/291
• US Class Current: 382/154
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: H04N 13/111 Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

The apparent depth (416) of an apparent point (404) in a set of stereoscopic images (300) is altered by moving a region (502) in at least one image (300) which corresponds to the apparent point (404). The location of the regions (502) within each image (300) representing the apparent point (404), together with the geometry of the vantage point (306) locations, specifies the depth (316) of an actual point (304) in a scene. The region (502) corresponding to the apparent point (404) in one of the images (300) is then moved to another location in the image (300), in order to produce another set of images (300) in which the apparent depth (416) of the apparent point (404) has been altered. In other embodiments, portions of both images (300) are moved, to place the apparent point (404) at a desired location both in terms of apparent depth (416) and apparent epipolar location (314). The determination of the new apparent depth (416) for the apparent point (404) may be based on the initial apparent depth (416), or may be unrelated to the initial apparent depth (416). The new apparent depth (416) for the apparent point (404) may be selected by the user of a computer program carrying out the method. The set of stereoscopic images (300) may be still images or moving images, and may be captured digitally, scanned from photographs, or computer-generated.

Claims

1. A computer implemented method for changing the apparent depth of at least one apparent point represented in a first image and a second image, which first image and second image are members of a stereoscopic set of images, each of the first image and the second image having a vantage point associated therewith, the method comprising the steps of:

identifying in the first image a first region smaller than the first image, which first region represents an apparent point;

identifying in the second image smaller than the second image a second region, which second region represents the apparent point;

determining a first epipolar offset, where the first epipolar offset is the difference in an epipolar direction between the location of the second region relative to a characteristic point of the second image and the location of the first region relative to a characteristic point of the first image, which epipolar direction is parallel to a vector from the vantage point of the first image to the vantage point of the second image; and

changing the location of at least one of the first region and the second region such that, after changing, the first region and the second region define a second epipolar offset, which second epipolar offset is the difference in the epipolar direction between the location of the second region relative to the characteristic point of the second image and the location of the first region relative to the characteristic point of the first image, the change of location being selected such that the second epipolar offset is a desired value.

2. The method of claim 1, wherein the relation of the characteristic point of the first image to the first image is the same as the relation of the characteristic point of the second image to the second image.

3. The method of claim 1, wherein the desired value of the second epipolar offset is the product of the first epipolar offset multiplied by a predetermined constant.

4. The method of claim 1, wherein the desired value of the second epipolar offset is the product of the first epipolar offset multiplied by a constant which is selected by a user.

5. The method of claim 1, wherein the desired value for the second epipolar offset is determined with reference to the first epipolar offset.

6. The method of claim 5, wherein the desired value for the second epipolar offset is determined with reference to the distance in the epipolar direction between the vantage point of the first image and the vantage point of the second image.

7. The method of claim 1, wherein one of the first image and the second image is a scanned photograph of a physical scene.

8. The method of claim 1, wherein one of the first image and the second image is an image of a physical scene which has been captured in digital form.

9. The method of claim 1, wherein one of the first image and the second image is a computer generated image.

10. The method of claim 1, wherein one of the first image and the second image is a moving image.

11. The method of claim 1, wherein changing the position of at least one of the first region and the second region comprises changing the position of the first region and the second region such that the apparent location of the apparent point is not changed in the epipolar direction.

12. A computer apparatus comprising:

a central processing unit (CPU);

an image memory coupled to the CPU, for storing a first image of an apparent point and a second image of the apparent point, which first image and second image are members of a stereoscopic set of images, each of the first image and the second image having a vantage point associated therewith; and

a program memory coupled to the CPU, for storing an array of instructions, which instructions, when executed by the CPU, cause the CPU to;

identify in the first image a first region, smaller than the first image which first region represents the apparent point;

identify in the second image a second region smaller than the second image, which second region represents the apparent point;

determine a first epipolar offset, where the first epipolar offset is the difference in an epipolar direction between the location of the second region relative to a characteristic point of the second image and the location of the first region relative to a characteristic point of the first image, which epipolar direction is parallel to a vector from the vantage point of the first image to the vantage point of the second image; and

change the location of at least one of the first region and the second region such that, after changing, the first region and the second region define a second epipolar offset, which second epipolar offset is the difference in the epipolar direction between the location of the second region relative to the characteristic point of the second image and the location of the first region relative to the characteristic point of the first image, the change of location being selected such that the second epipolar offset is a desired value.

13. The apparatus of claim 12, wherein the relation of the characteristic point of the first image to the first image is the same as the relation of the characteristic point of the second image to the second image.

14. The apparatus of claim 12, wherein the desired value of the second epipolar offset is the product of the first epipolar offset multiplied by a predetermined constant.

15. The apparatus of claim 12, wherein the desired value for the second epipolar offset is determined with reference to the first epipolar offset.

16. The apparatus of claim 15, wherein the desired value for the second epipolar offset is determined with reference to the distance in the epipolar direction between the vantage point of the first image and the vantage point of the second image.

17. The apparatus of claim 12, wherein one of the first image and the second image is a moving image.

18. The apparatus of claim 12, wherein changing the position of at least one of the first region and the second region comprises changing the position of the first region and the second region such that the apparent location of the point in the epipolar direction is not changed.

19. A computer readable medium containing a computer program for changing the apparent location of at least one apparent point represented in a first image and a second image, which first image and second image are members of a stereoscopic set of images, each of the first image and the second image having a vantage point associated therewith, the changing of the apparent location of the apparent point performed by a method comprising the steps of:

identifying in the first image a first region smaller than the first image, which first region represents an apparent point;

identifying in the second image a second region smaller than the second image, which second region represents the apparent point;

determining a first epipolar offset, where the first epipolar offset is the difference in an epipolar direction between the location of the second region relative to a characteristic point of the second image and the location of the first region relative to a characteristic point of the first image, which epipolar direction is parallel to a vector from the vantage point of the first image to the vantage point of the second image; and

changing the location of at least one of the first region and the second region such that, after changing, the first region and the second region define a second epipolar offset, which second epipolar offset is the difference in the epipolar direction between the location of the second region relative to the characteristic point of the second image and the location of the first region relative to the characteristic point of the first image, the change of location being selected such that the second epipolar offset is a desired value.

20. The computer readable medium of claim 19, wherein the relation of the characteristic point of the first image to the first image is the same as the relation of the characteristic point of the second image to the second image.