Image segmentation by means of temporal parallax difference induction

• US 7,162,083 B2
• Filed: 07/05/2002
• Issued: 01/09/2007
• Est. Priority Date: 07/06/2001
• Status: Active Grant

• Current Assignee: Vision III Imaging, Inc.
• Original Assignee: Vision III Imaging, Inc.
• Inventors: Martin, Michael B., Mayhew, Christopher A.
• Primary Examiner(s): SHERALI, ISHRAT I
• Application Number:
  US10/188,872
• Publication Number:
  US 20030007560A1
• Time in Patent Office: 1,649 Days
• Field of Search: 382/103, 382/106, 382/107, 382/154, 382/173, 382/162, 348/135, 348/139, 348/446, 348/50, 348/586, 348/587, 348/42, 348/208.1, 345/419, 345/600, 396/324, 396/429, 356/608, 359/470, 359/477
• US Class Current: 382/173
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/579 from motion
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 5/275 Generation of keying signals

Abstract

An image compositing and compression method based on the creation and processing of parallax differences in motion photography. A parallax scanning MOE lens creates discrete parallax differences in the objects in the recorded scene that are perceived by the viewer as enhanced texture and depth when displayed. Using parallax differences in a captured scene, a computer can detect objects for the purpose of creating image compositing mattes. This method allows matte passes to be filmed on location at the time of principal photography, thereby saving costly additional blue/green stage production shoot days associated with traveling matte techniques. In addition, because the mattes are based on parallax scan differences in the recorded scene and not on a uniform color and luminance process, certain conflicting scene subject colors will not have to be avoided. Also, because the matte scenes are recorded on location, the lighting in each of the various elements matches in the final composited image.

Claims

1. A method of autostereoscopic parallax scan imaging comprising the steps of:

providing an imaging plane;

providing a lens having an optical axis;

directing the optical axis toward an object;

creating a lens aperture at a succession of disparity positions offset from the optical axis;

observing a succession of images of the object appearing on the imaging plane; and

generating flow maps, for one or more of the succession of images, which specify how pixels move between adjacent images in the succession of images.

2. The method of claim 1, further comprising:

generating motion patterns, for one or more of the succession of images, which specify how pixels move over the succession of images;

generating a matte ordering using parallax scan amplitude and phase values from the motion patterns;

separating groups of pixels into distinct regions by combining pixels with similar flow map values; and

generating a matte for the distinct regions of the one or more of the succession of images to provide a succession of mattes.

3. The method of claim 2, further comprising selecting and removing selected mattes from the succession of mattes.

4. The method of claim 3, further comprising using a computer or control device to automatically select the mattes.

5. The method of claim 3, further comprising manually selecting the mattes.

6. The method of claim 2, further comprising performing region edge processing on at least one matte of the succession of mattes.

7. The method of claim 2, further comprising:

generating color layers for the one or more of the succession of images by multiplying the succession of mattes by corresponding original image information.

8. The method of claim 7 further comprising:

inserting additional image layers into the color layers generated for the one or more of the succession of images.

9. The method of claim 7 further comprising:

ordering the color layers.

10. The method of claim 7, further comprising:

generating a composite image by adding together at least two of the color layers corresponding to the one or more of the succession of images.

11. The method of claim 10, further comprising:

generating a final composite image by adjusting the composite image for brightness or contrast.

12. The method of claim 1 in which a parallax scan data file is created and stored when each of the succession of images is observed.

13. The method of claim 2 in which a parallax scan data file is used to assist in the generation of motion patterns.

14. The method of claim 1 further comprising:

using the flow maps to estimate physical distances from the image recording device to the objects comprising each matte.