Registration of separations

• US 7,486,842 B2
• Filed: 09/17/2007
• Issued: 02/03/2009
• Est. Priority Date: 01/04/2002
• Status: Expired due to Term

• Current Assignee: AOL LLC (Apollo Global Management, Inc.), Warner Bros Entertainment Incorporated (Warner Bros. Discovery, Inc.)
• Original Assignee: AOL LLC (Apollo Global Management, Inc.), Warner Bros Entertainment Incorporated (Warner Bros. Discovery, Inc.)
• Inventors: Perlmutter, Sharon M., Perlmutter, Keren O., Wang, Eric, Gasparri, Massimiliano, Klamer, Paul R., Cookson, Christopher J.
• Primary Examiner(s): Kassa; Yosef
• Application Number:
  US11/856,541
• Publication Number:
  US 20080056614A1
• Time in Patent Office: 505 Days
• Field of Search: 382/282, 382/284, 382/294
• US Class Current: 382/294
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20064 Wavelet transform [DWT]
• CPC Class Codes: G06T 2207/20192 Edge enhancement; Edge preservation
• CPC Class Codes: G06T 3/403 Edge-driven scaling; Edge-based scaling
• CPC Class Codes: G06T 3/4084 in the transform domain, e.g. fast Fourier transform [FFT] domain scaling
• CPC Class Codes: G06T 5/003 Deblurring; Sharpening vibration or motion blur correction for cameras comprising an electronic image sensor H04N23/682
• CPC Class Codes: G06T 5/10 using non-spatial domain filtering
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/38 Registration of image sequences
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image
• CPC Class Codes: H04N 1/58 Edge or detail enhancement; Noise or error suppression, e.g. colour misregistration correction H04N1/62 takes precedence
• CPC Class Codes: H04N 23/15 Image signal generation with circuitry for avoiding or correcting image misregistration
• CPC Class Codes: H04N 5/262 Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
• CPC Class Codes: H04N 9/11 Scanning of colour motion picture films, e.g. for telecine

Abstract

Separations or images relating to film or other fields may be registered using a variety of features, such as, for example: (1) correcting one or more film distortions; (2) automatically determining a transformation to reduce a film distortion; (3) applying multiple criteria of merit to a set of features to determine a set of features to use in determining a transformation; (4) determining transformations for areas in an image or a separation in a radial order; (5) comparing areas in images or separations by weighting feature pixels differently than non-feature pixels; (6) determining distortion values for transformations by applying a partial distortion measure and/or using a spiral search configuration; (7) determining transformations by using different sets of features to determine corresponding transformation parameters in an iterative manner; and (8) applying a feathering technique to neighboring areas within an image or separation.

Claims

1. A computer-implemented method for automatically determining an alignment vector for a region within an image, the method comprising:

defining multiple regions within an image;

selecting, from among the multiple regions, a first region within the image;

identifying, from among the multiple regions, a second region within the image that is related to the first region within the image;

accessing an alignment vector determined for the second region within the image, the alignment vector determined for the second region within the image being descriptive of a transformation that aligns the second region within the image with a second region within a reference image that corresponds to the second region within the image;

identifying a first region within the reference image that corresponds to the first region within the image;

comparing the first region within the image to the first region within the reference image;

based on the comparison, determining a registration metric for the first region within the image; and

automatically determining an alignment vector for the first region within the image based on the accessed alignment vector determined for the second region within the image and the determined registration metric for the first region within the image, the alignment vector determined for the first region within the image being descriptive of a transformation that aligns the first region within the image with the first region within the reference image that corresponds to the first region within the image.

2. The computer-implemented method of claim 1:

wherein defining multiple regions within an image comprises defining a set of blocks within a first frame of a first color film separation and grouping the set of blocks into multiple concentric rings;

wherein selecting, from among the multiple regions, a first region within the image comprises selecting, from among the set of blocks, a first block within the first frame of the first color film separation that is grouped into a first ring;

wherein identifying, from among the multiple regions, a second region within the image that is related to the first region within the image comprises identifying, from among the set of blocks, a second block within the first frame of the first color film separation that is grouped into a second ring with a second radius that is shorten than a first radius of the first ring into which the first block is grouped;

wherein accessing an alignment vector determined for the second region within the image, the alignment vector determined for the second region within the image being descriptive of a transformation that aligns the second region within the image with a second region within a reference image that corresponds to the second region within the image comprises accessing an alignment vector determined for the second block within the first frame of the first color film separation, the alignment vector determined for the second block within the first frame of the first color film separation being descriptive of a transformation that aligns the second block within the first frame of the first color film separation with a second block within a second frame of a second color film separation that corresponds to the second block within the first frame of the first color film separation;

wherein identifying a first region within the reference image that corresponds to the first region within the image comprises identifying a first block within the second frame of the second color film separation that corresponds to the first block within the first frame of the first color film separation;

wherein comparing the first region within the image to the first region within the reference image comprises comparing the first block within the first frame of the first color film separation to the first block within the second frame of the second color film separation;

wherein determining a registration metric for the first region within the image comprises determining a registration metric for the first block within the first frame of the first color film separation; and

wherein automatically determining an alignment vector for the first region within the image based on the accessed alignment vector determined for the second region within the image and the determined registration metric for the first region within the image, the alignment vector determined for the first region within the image being descriptive of a transformation that aligns the first region within the image with the first region within the reference image that corresponds to the first region within the image comprises automatically determining an alignment vector for the first block within the first frame of the first color film separation based on the accessed alignment vector determined for the second block within the first frame of the first color film separation and the determined registration metric for the first block within the first frame of the first color film separation, the alignment vector determined for the first block within the first frame of the first color film separation being descriptive of a transformation that aligns the first block within the first frame of the first color film separation with the first block within the second frame of the second color film separation that corresponds to the first block within the first frame of the first color film separation.

3. The computer-implemented method of claim 1 wherein the image is a frame of a first color film separation and the reference image is a frame of a second color film separation that is different than the first color film separation, further comprising:

combining the frame of the first color film separation and the frame of the second color film separation to form a composite color frame by applying the alignment vector determined for the first region within the image.

4. The computer-implemented method of claim 1 wherein determining a registration metric for the first region within the image comprises determining a similarity measurement that is indicative of a similarity between the first region within the image and the first region within the reference image.

5. The computer-implemented method of claim 1 wherein determining a registration metric for the first region within the image comprises determining a distortion measurement that is indicative of a distortion between the first region within the image and the first region within the reference image.

6. The computer-implemented method of claim 1 wherein determining a registration metric for the first region within the image comprises:

identifying a feature within the first region within the image;

identifying the feature within the first region within the reference image;

determining a first registration measure based on a comparison of at least one feature pixel that corresponds to the identified feature within the first region within the image to at least one feature pixel that corresponds to the identified feature within the first region within the reference image;

determining a second registration measure based on a comparison of at least one non-feature pixel that does not correspond to the identified feature within the first region within the image to at least one non-feature pixel that does not corresponds to the identified feature within the first region within the reference image;

computing the registration metric for the first region within the image by combining the first registration measure and the second registration measure in a manner that weights the first registration measure more heavily than the second registration measure.

7. The computer-implemented method of claim 1 wherein identifying, from among the multiple regions, a second region within the image that is related to the first region within the image comprises identifying, as the second region within the image, a neighboring region that has already been processed and that shares at least one common pixel with the first region within the image.

8. The computer-implemented method of claim 1 wherein identifying, from among the multiple regions, a second region within the image that is related to the first region within the image comprises identifying, as the second region within the image, a neighboring region that has already been processed and that shares a common border with the first region within the image.

9. The computer-implemented method of claim 1:

wherein defining multiple regions within an image comprises grouping the multiple regions into a set of concentric rings; and

wherein identifying, from among the multiple regions, a second region within the image that is related to the first region within the image comprises identifying, as the second region within the image, an inward radial neighboring region that is no further from a particular reference point than the first region within the image.

10. The computer-implemented method of claim 9 wherein identifying, as the second region within the image, an inward radial neighboring region that is no further from a particular reference point than the first region within the image comprises identifying, as the second region within the image, an inward radial neighboring region that is no further from a center of the concentric rings than the first region within the image.

11. The computer-implemented method of claim 1 wherein automatically determining an alignment vector for the first region within the image based on the accessed alignment vector determined for the second region within the image and the determined registration metric for the first region within the image comprises:

automatically determining an initial alignment vector based on the accessed alignment vector determined for the second region within the image; and

automatically adjusting the initial alignment vector based on the determined registration metric for the first region within the image.

12. The computer-implemented method of claim 11 wherein automatically determining an initial alignment vector based on the accessed alignment vector determined for the second region within the image comprises automatically determining an initial alignment vector based solely on the accessed alignment vector determined for the second region within the image.

13. An apparatus for automatically determining an alignment vector for a region within an image, the apparatus comprising at least one processor programmed to perform operations comprising:

defining multiple regions within an image;

selecting, from among the multiple regions, a first region within the image;

identifying, from among the multiple regions, a second region within the image that is related to the first region within the image;

accessing an alignment vector determined for the second region within the image, the alignment vector determined for the second region within the image being descriptive of a transformation that aligns the second region within the image with a second region within a reference image that corresponds to the second region within the image;

identifying a first region within the reference image that corresponds to the first region within the image;

comparing the first region within the image to the first region within the reference image;

based on the comparison, determining a registration metric for the first region within the image; and

automatically determining an alignment vector for the first region within the image based on the accessed alignment vector determined for the second region within the image and the determined registration metric for the first region within the image, the alignment vector determined for the first region within the image being descriptive of a transformation that aligns the first region within the image with the first region within the reference image that corresponds to the first region within the image.

14. The apparatus of claim 13:

wherein defining multiple regions within an image comprises defining a set of blocks within a first frame of a first color film separation and grouping the set of blocks into multiple concentric rings;

wherein selecting, from among the multiple regions, a first region within the image comprises selecting, from among the set of blocks, a first block within the first frame of the first color film separation that is grouped into a first ring;

wherein identifying, from among the multiple regions, a second region within the image that is related to the first region within the image comprises identifying, from among the set of blocks, a second block within the first frame of the first color film separation that is grouped into a second ring with a second radius that is shorten than a first radius of the first ring into which the first block is grouped;

wherein accessing an alignment vector determined for the second region within the image, the alignment vector determined for the second region within the image being descriptive of a transformation that aligns the second region within the image with a second region within a reference image that corresponds to the second region within the image comprises accessing an alignment vector determined for the second block within the first frame of the first color film separation, the alignment vector determined for the second block within the first frame of the first color film separation being descriptive of a transformation that aligns the second block within the first frame of the first color film separation with a second block within a second frame of a second color film separation that corresponds to the second block within the first frame of the first color film separation;

wherein identifying a first region within the reference image that corresponds to the first region within the image comprises identifying a first block within the second frame of the second color film separation that corresponds to the first block within the first frame of the first color film separation;

wherein comparing the first region within the image to the first region within the reference image comprises comparing the first block within the first frame of the first color film separation to the first block within the second frame of the second color film separation;

wherein determining a registration metric for the first region within the image comprises determining a registration metric for the first block within the first frame of the first color film separation; and

wherein automatically determining an alignment vector for the first region within the image based on the accessed alignment vector determined for the second region within the image and the determined registration metric for the first region within the image, the alignment vector determined for the first region within the image being descriptive of a transformation that aligns the first region within the image with the first region within the reference image that corresponds to the first region within the image comprises automatically determining an alignment vector for the first block within the first frame of the first color film separation based on the accessed alignment vector determined for the second block within the first frame of the first color film separation and the determined registration metric for the first block within the first frame of the first color film separation, the alignment vector determined for the first block within the first frame of the first color film separation being descriptive of a transformation that aligns the first block within the first frame of the first color film separation with the first block within the second frame of the second color film separation that corresponds to the first block within the first frame of the first color film separation.

15. The apparatus of claim 13 wherein the image is a frame of a first color film separation and the reference image is a frame of a second color film separation that is different than the first color film separation, and the at least one processor is further programmed to perform operations comprising:

combining the frame of the first color film separation and the frame of the second color film separation to form a composite color frame by applying the alignment vector determined for the first region within the image.

16. The apparatus of claim 13:

wherein defining multiple regions within an image comprises grouping the multiple regions into a set of concentric rings; and

wherein identifying, from among the multiple regions, a second region within the image that is related to the first region within the image comprises identifying, as the second region within the image, an inward radial neighboring region that is no further from a particular reference point than the first region within the image.

17. A computer-readable medium encoded with a computer program comprising instructions that, when executed, operate to cause a computer to perform operations comprising:

defining multiple regions within an image;

selecting, from among the multiple regions, a first region within the image;

identifying, from among the multiple regions, a second region within the image that is related to the first region within the image;

accessing an alignment vector determined for the second region within the image, the alignment vector determined for the second region within the image being descriptive of a transformation that aligns the second region within the image with a second region within a reference image that corresponds to the second region within the image;

identifying a first region within the reference image that corresponds to the first region within the image;

comparing the first region within the image to the first region within the reference image;

based on the comparison, determining a registration metric for the first region within the image; and

automatically determining an alignment vector for the first region within the image based on the accessed alignment vector determined for the second region within the image and the determined registration metric for the first region within the image, the alignment vector determined for the first region within the image being descriptive of a transformation that aligns the first region within the image with the first region within the reference image that corresponds to the first region within the image.

18. The computer-readable medium of claim 17:

wherein defining multiple regions within an image comprises defining a set of blocks within a first frame of a first color film separation and grouping the set of blocks into multiple concentric rings;

wherein selecting, from among the multiple regions, a first region within the image comprises selecting, from among the set of blocks, a first block within the first frame of the first color film separation that is grouped into a first ring;

wherein identifying, from among the multiple regions, a second region within the image that is related to the first region within the image comprises identifying, from among the set of blocks, a second block within the first frame of the first color film separation that is grouped into a second ring with a second radius that is shorten than a first radius of the first ring into which the first block is grouped;

wherein accessing an alignment vector determined for the second region within the image, the alignment vector determined for the second region within the image being descriptive of a transformation that aligns the second region within the image with a second region within a reference image that corresponds to the second region within the image comprises accessing an alignment vector determined for the second block within the first frame of the first color film separation, the alignment vector determined for the second block within the first frame of the first color film separation being descriptive of a transformation that aligns the second block within the first frame of the first color film separation with a second block within a second frame of a second color film separation that corresponds to the second block within the first frame of the first color film separation;

wherein identifying a first region within the reference image that corresponds to the first region within the image comprises identifying a first block within the second frame of the second color film separation that corresponds to the first block within the first frame of the first color film separation;

wherein comparing the first region within the image to the first region within the reference image comprises comparing the first block within the first frame of the first color film separation to the first block within the second frame of the second color film separation;

wherein determining a registration metric for the first region within the image comprises determining a registration metric for the first block within the first frame of the first color film separation; and

wherein automatically determining an alignment vector for the first region within the image based on the accessed alignment vector determined for the second region within the image and the determined registration metric for the first region within the image, the alignment vector determined for the first region within the image being descriptive of a transformation that aligns the first region within the image with the first region within the reference image that corresponds to the first region within the image comprises automatically determining an alignment vector for the first block within the first frame of the first color film separation based on the accessed alignment vector determined for the second block within the first frame of the first color film separation and the determined registration metric for the first block within the first frame of the first color film separation, the alignment vector determined for the first block within the first frame of the first color film separation being descriptive of a transformation that aligns the first block within the first frame of the first color film separation with the first block within the second frame of the second color film separation that corresponds to the first block within the first frame of the first color film separation.

19. The computer-readable medium of claim 17 wherein the image is a frame of a first color film separation and the reference image is a frame of a second color film separation that is different than the first color film separation, and the computer program further comprises instructions that, when executed, operate to cause a computer to perform operations comprising:

combining the frame of the first color film separation and the frame of the second color film separation to form a composite color frame by applying the alignment vector determined for the first region within the image.

20. The computer-readable medium of claim 17:

wherein defining multiple regions within an image comprises grouping the multiple regions into a set of concentric rings; and

wherein identifying, from among the multiple regions, a second region within the image that is related to the first region within the image comprises identifying, as the second region within the image, an inward radial neighboring region that is no further from a particular reference point than the first region within the image.