Image sequence depth enhancement system and method

• US 7,907,793 B1
• Filed: 08/17/2009
• Issued: 03/15/2011
• Est. Priority Date: 05/04/2001
• Status: Expired due to Fees

• Current Assignee: Legend3D, Inc.
• Original Assignee: Legend Films Incorporated (Legend3D, Inc.)
• Inventors: Sandrew, Barry
• Primary Examiner(s): Kassa; Yosef
• Application Number:
  US12/542,498
• Time in Patent Office: 575 Days
• Field of Search: 382/154, 382/284, 382/294, 382/307, 348/576, 348/586
• US Class Current: 382/284
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: H04N 9/43 Conversion of monochrome picture signals to colour picture signals for colour picture display

Abstract

Motion picture scenes to be colorized/depth enhanced (2D→3D) are broken into separate elements, backgrounds/sets or motion/onscreen-action. Background and motion elements are combined separately into single frame representations of multiple frames which becomes a visual reference database that includes data for all frame offsets used later for the computer controlled application of masks within a sequence of frames. Each pixel address within the database corresponds to a mask/lookup table address within the digital frame and X, Y, Z location of subsequent frames. Masks are applied to subsequent frames of motion objects based on various differentiating image processing methods, including automated mask fitting of all masks or single masks in an entire frame, bezier and polygon tracing of selected regions with edge detected shaping and operator directed detection of subsequent regions. Colors and/or depths are automatically applied to masks throughout a scene from the composite background and to motion objects.

Claims

1. A method for modifying a set of time ordered digital images comprising:

associating a plurality of images comprising common background elements;

obtaining a composite background comprising said plurality of images wherein said composite background excludes a set of motion objects moving in relation to said common background elements;

setting at least one depth parameter associated with a region within said composite background;

applying said at least one depth parameter to at least one image selected from said plurality of images using a computer.

2. The method of claim 1 wherein said setting said at least one depth parameter comprises assigning a depth value for at least one of said common background elements.

3. The method of claim 1 wherein said setting said at least one depth parameter comprises selecting a three-dimensional geometric depth shape for at least one of said common background elements.

4. The method of claim 1 wherein said setting said at least one depth parameter comprises translating a three-dimensional geometric depth shape for at least one of said common background elements.

5. The method of claim 1 wherein said setting said at least one depth parameter comprises rotating a three-dimensional geometric depth shape for at least one of said common background elements.

6. The method of claim 1 wherein said setting said at least one depth parameter comprises changing a size of a three-dimensional geometric depth shape for at least one of said common background elements.

7. The method of claim 1 wherein said setting said at least one depth parameter comprises associating a transparency mask with said at least one common background elements and assigning depth to said transparency mask.

8. The method of claim 1 wherein said obtaining a composite background further comprises:

obtaining a camera angle value for said at least one image selected from said plurality of images.

9. The method of claim 1 wherein said obtaining a composite background further comprises:

obtaining a camera location value for said at least one image selected from said plurality of images.

10. The method of claim 1 wherein said obtaining a composite background further comprises:

estimating a camera focal length value for said at least one image.

11. The method of claim 1 wherein said obtaining a composite background further comprises:

estimating a zoom value for said at least one image.

12. The method of claim 1 further comprising:

automatically separating said plurality of images at scene changes.

13. A method for modifying a set of time ordered digital images comprising:

associating a first mask with a motion object in a first image;

copying said first mask to create a second mask associated with a second image;

moving said second mask to a location of said motion object in said second image;

reshaping said second mask to fit said motion object in said second image using a computer;

setting at least one depth parameter associated with said first mask;

applying said at least one depth parameter to said second image using said computer.

14. The method of claim 13 wherein said setting said at least one depth parameter comprises selecting a numeric depth value for said first mask.

15. The method of claim 13 wherein said setting said at least one depth parameter comprises selecting a three-dimensional geometric depth shape for said first mask.

16. The method of claim 13 wherein said setting said at least one depth parameter comprises translating a three-dimensional geometric depth shape for said first mask.

17. The method of claim 13 wherein said setting said at least one depth parameter comprises rotating a three-dimensional geometric depth shape for said first mask.

18. The method of claim 13 wherein said setting said at least one depth parameter comprises changing a size of a three-dimensional geometric depth shape for said first mask.

19. The method of claim 13 wherein said setting said at least one depth parameter comprises associating a transparency mask with said first mask and assigning depth to said transparency mask.

20. A method for modifying a set of time ordered digital images comprising:

associating a plurality of images comprising common background elements;

obtaining a composite background comprising said plurality of images wherein said composite background excludes a set of motion objects moving in relation to said common background elements;

setting at least one depth parameter associated with a region within said composite background;

applying said at least one depth parameter to at least one non-composited image selected from said plurality of images using a computer;

associating a first mask with a motion object selected from said set of motion object in a first image selected from the plurality of images;

copying said first mask to create a second mask associated with a second image selected from the plurality of images;

moving said second mask to location of said motion object in said second image;

reshaping said second mask to fit said motion object in said second image using said computer;

setting at least one motion object depth parameter associated with said first mask;

applying said at least one motion object depth parameter to said second image using said computer.