System and method for using feature tracking techniques for the generation of masks in the conversion of two-dimensional images to three-dimensional images

• US 8,878,835 B2
• Filed: 07/12/2012
• Issued: 11/04/2014
• Est. Priority Date: 03/12/2007
• Status: Active Grant

• Current Assignee: Intellectual Discovery Co., Ltd.
• Original Assignee: Intellectual Discovery Co., Ltd.
• Inventors: Keech, Gregory R., Simmons, Christopher L., SPOONER, DAVID A., Lowe, Danny D., Birtwistle, Steven, Wallner, Natascha
• Primary Examiner(s): AMIN, JWALANT B
• Application Number:
  US13/547,817
• Publication Number:
  US 20120275652A1
• Time in Patent Office: 845 Days
• Field of Search: None
• US Class Current: 345/418
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal

Abstract

The present invention is directed to systems and methods for controlling 2-D to 3-D image conversion and/or generation. The methods and systems use auto-fitting techniques to create a mask based upon tracking features from frame to frame. When features are determined to be missing they are added prior to auto-fitting the mask.

Claims

1. A method of generating a mask for use in generating 2-D to 3-D conversion of an object within an image, said image being present as an image-set across multiple frames of said image, said method comprising:

establishing, on a processor-based system, a polygon of said object within a subset of said image-set;

selecting, on the processor-based system, at least one feature of said image-set;

tracking all endpoints of vertices of said polygon for all frames of said multiple frames of said image-set;

determining if said at least one selected feature is missing from certain frames of said multiple frames of said image-set;

auto-correcting a vertex of said polygon associated with said at least one selected feature into said certain frames when said at least one selected feature is determined by the processor-based system to be missing from said certain frames; and

generating a mask of said object to fit said at least one selected feature.

2. The method of claim 1

wherein said generating a mask comprises auto-fitting the mask by iteratively tracking all endpoints of vertices of said polygon for all frames of said multiple frames of said image-set and determining if said at least one selected feature is missing from certain frames of said multiple frames of said image-set.

3. The method of claim 1 wherein said polygon comprises a polycurve polygon.

4. The method of claim 3 wherein said subset represents an equal distribution of a total set of said image-set.

5. The method of claim 4 further comprising:

manually specifying a vertex of said polycurve polygon for at least one frame of said multiple frames,

wherein said equal distribution comprises at least a first, a middle and a last one of said image-set.

6. The method of claim 3 wherein said tracking assumes that every end point of a polycurve of said polygon corresponds to a feature in said image.

7. The method of claim 6 further comprising:

refining one or more control vertices such that a position of the one or more control vertices remain relative to all endpoints of vertices of said polygon.

8. The method of claim 7 wherein said refining comprises using image analysis tools.

9. The method of claim 1 wherein said tracking occurs in a sequential manner from one end of said image-set to another end.

10. A system for establishing a mask for use in generating 2-D to 3-D conversion of an object within an image, said image being present as an image-set across multiple frames of said image, said system comprising:

a memory storing executable instructions;

one or more processors operatively coupled to the memory to execute the executable instructions from the memory, the one or more processors being configured to;

execute control sequences for establishing a polygon of said object within a subset of said image-set;

execute control sequences for selecting at least one feature of said image-set;

execute control sequences for tracking all endpoints of vertices of said polygon for all frames of said multiple frames of said image-set;

execute control sequences for determining if said at least one selected feature is missing from certain frames of said multiple frames of said image-set;

execute control sequences for auto-correcting a vertex of said polygon associated with said at least one selected feature into said certain frames when said at least one selected feature is determined to be missing from said certain frames; and

execute control sequences for generating a mask of said object to fit said at least one selected feature.

11. The system of claim 10 wherein said generating a mask comprises auto-fitting the mask by iteratively executing control sequences for tracking all endpoints of vertices of said polygon for all frames of said multiple frames of said image-set and determining if said at least one selected feature is missing from certain frames of said multiple frames of said image-set.

12. The system of claim 11 wherein said polygon comprises a polycurve polygon.

13. The system of claim 10 wherein said control sequences for tracking assumes that every end point of a polycurve of said polygon corresponds to a feature in said image.

14. A non-transitory, machine-readable storage medium comprising instructions stored therein, the instructions executable by one or more processors to facilitate performing a method of generating a mask for use in generating 2-D to 3-D conversion of an object within an image, said image being present as an image-set across multiple frames of said image, said method comprising:

establishing a polygon of said object within a subset of said image-set;

selecting at least one feature of said image-set;

tracking all endpoints of vertices of said polygon for all frames of said multiple frames of said image-set;

determining if said at least one selected feature is missing from certain frames of said multiple frames of said image-set;

auto-correcting a vertex of said polygon associated with said at least one selected feature into said certain frames when said at least one selected feature is determined to be missing from said certain frames; and

generating a mask of said object to fit said at least one selected feature.

15. The non-transitory, machine-readable storage medium of claim 14 wherein said generating a mask comprises auto-fitting the mask by iteratively tracking all endpoints of vertices of said polygon for all frames of said multiple frames of said image-set and determining if said at least one selected feature is missing from certain frames of said multiple frames of said image-set.

16. The non-transitory, machine-readable storage medium of claim 15 wherein said polygon comprises a polycurve polygon;

and wherein the method further comprises temporarily inhibiting said auto-fitting when said at least one selected feature is determined to be missing from said certain frames.

17. The non-transitory, machine-readable storage medium of claim 16 wherein said subset represents an equal distribution of a total set of said image-set.

18. The non-transitory, machine-readable storage medium of claim 17 wherein said equal distribution comprises at least a first, a middle and a last one of said image-set.

19. The non-transitory, machine-readable storage medium of claim 16 wherein said tracking assumes that every end point of a polycurve of said polygon corresponds to a feature in said image.