Image sequence enhancement and motion picture project management system

• US 9,286,941 B2
• Filed: 05/11/2015
• Issued: 03/15/2016
• Est. Priority Date: 05/04/2001
• Status: Expired due to Fees

• Current Assignee: Legend3D, Inc.
• Original Assignee: Legend3D, Inc.
• Inventors: Sandrew, Barry, Baldridge, Tony, Sandrew, Jared, Lopez, Anthony
• Primary Examiner(s): JONES, HEATHER RAE
• Application Number:
  US14/709,300
• Publication Number:
  US 20150243324A1
• Time in Patent Office: 309 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: H04N 13/257 Colour aspects
• CPC Class Codes: H04N 13/261 with monoscopic-to-stereoscopic image conversion
• CPC Class Codes: H04N 13/266 by scanning a film

Abstract

The system enables conversion of black and white images to color images and/or two-dimensional images into three-dimensional images based on adding color and/or depth to images using masks for regions in the images, as well as reshaping of masks to cover objects that have moved and changed shape as the objects move in a sequence of images. Also, includes motion picture project management system for reviewers, coordinators and artists. Artists utilize image analysis and image enhancement and computer graphics processing for example to convert two-dimensional images into three-dimensional images or otherwise create or alter motion pictures. Enables the efficient management of projects related to motion pictures to enable enterprises to manage assets, control costs, predict budgets and profit margins, reduce archival storage and otherwise provide displays tailored to specific roles to increase worker efficiency.

Claims

1. A system configured to modify a set of time ordered digital images comprising a computer having memory and a program stored in said memory that is configured to:

associate a first mask with a motion object in a first image at a first mask location;

copy said first mask to create a second mask associated with a second image;

move said second mask to a second mask location associated with said motion object in said second image; and

,

reshape said second mask to fit said motion object in said second image based on underlying image data in said second image.

2. The system of claim 1 wherein said computer is further configured to:

accept user input of at least one threshold associated with said underlying image data;

create said first mask based on said at least one threshold associated with said underlying image data at said first mask location;

wherein said associate said first mask with said motion object comprises storing said first mask in said memory.

3. The system of claim 1 wherein said computer is further configured to:

move said second mask based on said underlying image data in said second image.

4. The system of claim 1 wherein said computer is further configured to:

reshape said second mask through a search near said second mask location in said second image based on said underlying image data contained within said first mask in said first image.

5. The system of claim 1 wherein said computer is further configured to:

track underlying image data in said second image that is contained within said first mask in said first image to adjust said second mask shape in said second image.

6. The system of claim 1 wherein said computer is further configured to:

reshape said second mask through a search via gradient descent within a bounding box around said second mask location in said second image based on said underlying image data contained within said first mask in said first image.

7. The system of claim 6 wherein said computer is further configured to:

create a fit box for said second mask in said second image; and

,

modify a mask shape of said second mask using a gradient descent of minimum errors within said fit box based on said underlying image data.

8. The system of claim 7 wherein said modify of the mask shape of said second mask further comprises a modification of said shape using Bezier curve animation with edge detection.

9. The system of claim 7 wherein said modify the mask shape of said second mask further comprises a modification of said shape via polygon animation with edge detection.

10. The system of claim 7 wherein said computer is further configured to:

calculate a weighted index for said first mask in said first image using bilinear interpolation;

calculate said minimum errors based on said weighted index for said first mask and on said underlying image data.

11. The system of claim 1 wherein said computer is further configured to:

wherein said second mask comprises a polygon or curve and wherein said move or reshape or both said move and said reshape of said second mask comprises a change in location of at least one pixel of said mask or one or more vertices or control points of said mask.

12. The system of claim 11 wherein said reshape of said second mask further comprises a modification of said shape based on Bezier curve animation with edge detection.

13. The system of claim 11 wherein said reshape of said second mask further comprises a modification of said shape based on polygon animation with edge detection.

14. The system of claim 1 wherein said computer is further configured to:

accept at least one depth parameter associated with said first mask in said first image;

apply said at least one depth parameter to said first mask in said first image.

15. The system of claim 14 wherein said computer is further configured to:

generate at least one translation map associated with said first image;

modify said first mask associated with said at least one translation map to create a modified mask; and

,

update a left viewpoint image or a right viewpoint image or both based on said at least one translation map and said modified mask.

16. The system of claim 1 wherein said computer is further configured to:

accept user input to manually adjust said second mask.

17. The system of claim 1 said computer is further configured to:

import a first alpha mask for a computer-generated element in said first image and import a second alpha mask for said second image and further reshape said first mask and reshape said second mask to fit a border of said alpha mask for said computer-generated element.

18. The system of claim 1 wherein said computer is further configured to calculate amount of disk space that may be utilized to archive a movie conversion project and signify at least one asset that may be rebuilt from other assets to avoid archival of said at least one asset.

19. The system of claim 1 wherein said underling image data is a feature or image feature or image characteristic associated with one or more pixels.

20. The system of claim 1 wherein said computer is further configured to:

utilize said underlying image data in said first mask to determine at least one depth parameter associated with said first image;

apply said at least one depth parameter to said first mask in said first image.

21. The system of claim 20 wherein said computer is further configured to:

generate at least one translation map associated with said first image;

modify said first mask associated with said at least one translation map to create a modified mask; and

,

update a left viewpoint image or a right viewpoint image or both based on said at least one translation map and said modified mask.

22. The system of claim 1 wherein said computer is further configured to:

create said first mask that comprises said associate said first mask with said motion object and wherein said first mask is associated with at least one characteristic of the first image and associated with at least a portion of the first image, including

obtain a selection of underlying image data and determine values of said underlying image data associated with a plurality of pixel locations in a portion of the first image,

determine a left or a right shift of the plurality of pixel locations of the portion of the first image,

apply the first mask to the left or the right shift of the portion of the first image to create a weighted displacement mask having a displacement vector associated with the plurality of pixel locations of the portion of the first image, the displacement vector having a magnitude and direction,

displace each of the plurality of pixel locations of the portion of the first image in accordance with the magnitude of the displacement vector associated with that pixel to create a left or right viewpoint image or both.

23. The system of claim 22 wherein said computer is further configured to:

generate at least one translation map associated with said first image;

modify said first mask associated with said at least one translation map to create a modified mask; and

,

update said left viewpoint image or said right viewpoint image or both based on said at least one translation map and said modified mask.

24. A system configured to modify a set of time ordered digital images comprising a computer having memory and a program stored in said memory that is configured to:

associate a first mask with a motion object in a first image at a first mask location;

copy said first mask to create a second mask associated with a second image;

move said second mask to a second mask location associated with said motion object in said second image based on underlying image data in said second image; and

,

reshape said second mask to fit said motion object in said second image based on said underlying image data in said second image through a search near said second mask location in said second image based on said underlying image data contained within said first mask in said first image.

25. The system of claim 24 wherein said computer is further configured to:

accept at least one depth parameter associated with said first mask in said first image;

apply said at least one depth parameter to said first mask in said first image.

26. The system of claim 25 wherein said computer is further configured to:

generate at least one translation map associated with said first image;

modify said first mask associated with said at least one translation map to create a modified mask; and

,

update a left viewpoint image or a right viewpoint image or both based on said at least one translation map and said modified mask.

27. The system of claim 24 wherein said computer is further configured to:

utilize said underlying image data in said first mask to determine at least one depth parameter associated with said first image;

apply said at least one depth parameter to said first mask in said first image.

28. The system of claim 27 wherein said computer is further configured to:

generate at least one translation map associated with said first image;

modify said first mask associated with said at least one translation map to create a modified mask; and

,

update a left viewpoint image or a right viewpoint image or both based on said at least one translation map and said modified mask.

29. The system of claim 24 wherein said computer is further configured to:

create said first mask that comprises said associate said first mask with said motion object and wherein said first mask is associated with at least one characteristic of the first image and associated with at least a portion of the first image, including

obtain a selection of underlying image data and determine values of said underlying image data associated with a plurality of pixel locations in a portion of the first image,

determine a left or a right shift of the plurality of pixel locations of the portion of the first image,

apply the first mask to the left or the right shift of the portion of the first image to create a weighted displacement mask having a displacement vector associated with the plurality of pixel locations of the portion of the first image, the displacement vector having a magnitude and direction,

displace each of the plurality of pixel locations of the portion of the first image in accordance with the magnitude of the displacement vector associated with that pixel to create a left or right viewpoint image or both.

30. The system of claim 29 wherein said computer is further configured to:

generate at least one translation map associated with said first image;

modify said first mask associated with said at least one translation map to create a modified mask; and

,

update said left viewpoint image or said right viewpoint image or both based on said at least one translation map and said modified mask.