System and method for real-time image retensioning and loop error correction

• US 8,437,497 B2
• Filed: 01/27/2011
• Issued: 05/07/2013
• Est. Priority Date: 01/27/2011
• Status: Active Grant

• Current Assignee: Seiko Epson Corporation (Seiko Group)
• Original Assignee: Seiko Epson Corporation (Seiko Group)
• Inventors: Fu, Guoyi, Brusnitsyn, Mikhail
• Primary Examiner(s): Kassa, Yosef
• Application Number:
  US13/015,285
• Publication Number:
  US 20120195523A1
• Time in Patent Office: 831 Days
• Field of Search: 382/274, 382/275, 382/278, 382/282, 382/313, 382/318, 382/100, 382/260, 382/284, 382/294, 358/3.26, 358/3.27, 358/463, 358/505, 358/540, 358/450
• US Class Current: 382/100
• CPC Class Codes: G06T 2207/10008 from scanner, fax or copier
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/30176 Document
• CPC Class Codes: G06T 3/14 Transformations for image registration, e.g. adjusting or mapping for alignment of images
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: H04N 1/107 with manual scanning
• CPC Class Codes: H04N 1/387 Composing, repositioning or otherwise geometrically modifying originals
• CPC Class Codes: H04N 1/3871 the composed originals being of different kinds, e.g. low- and high-resolution originals
• CPC Class Codes: H04N 1/3876 Recombination of partial images to recreate the original image
• CPC Class Codes: H04N 2201/04787 by changing or controlling the addresses or values of pixels, e.g. in an array, in a memory, by interpolation

Abstract

Due to an accumulated error from the pair-wise registration, the stitched image may be blurred or have a gap when a loop is encountered. In order to remove the accumulated error, we identify a closed loop where a first image frame overlaps with a second image frame, the second image frame being captured earlier in a scanning sequence than the first image frame; register the first image frame with the second image frame; and apply a global optimization to adjust registration parameters for the plurality of pair-wise registrations of image frames within the closed loop using global constraints.

Claims

1. A method for correcting an error accumulated from a plurality of pair-wise registrations of image frames captured in sequence by a scanning device, comprising:

using a processor to;

identify a closed loop where a first image frame overlaps with a second image frame, the second image frame being captured earlier in the sequence than the first image frame;

register the first image frame with the second image frame;

apply a global optimization to adjust registration parameters for the plurality of pair-wise registrations of image frames within the closed loop using global constraints;

identifying a registration break in the closed loop, the registration break being between two consecutive frames i and i+1 in the sequence;

assigning all registration errors in the closed loop to a pair-wise registration between frames i and i+1; and

updating pair-wise transformation parameters according to a transformation matrix for frames i and i+1.

2. A method as in claim 1 wherein adjusting the registration parameters comprises calculating an error measurement that includes multiple global constraints.

3. A method as in claim 2 wherein the global constraints comprise three independent scalar constraints.

4. A method as in claim 2 further comprising extracting matched feature point pairs for the plurality of pair-wise registrations.

5. A method as in claim 4 wherein calculating the error measurement further includes matched feature point errors.

6. A method for correcting an error accumulated from a plurality of pair-wise registrations of image frames captured in sequence by a scanning device, comprising:

using a processor to;

identify a closed loop where a first image frame overlaps with a second image frame, the second image frame being captured earlier in the sequence than the first image frame;

register the first image frame with the second image frame; and

apply a global optimization to adjust registration parameters for the plurality of pair-wise registrations of image frames within the closed loop using global constraints;

the method further comprising marking all pair-wise registrations of image frames within the closed loop as fixed;

wherein the registration parameters of pair-wise registrations of image frames within the closed loop that have been marked as fixed are not adjusted during the global optimization.

7. An apparatus for correcting an error accumulated from a plurality of pair-wise registrations of image frames captured in sequence by a scanning device, comprising:

a processor that;

identifies a closed loop where a first image frame overlaps with a second image frame, the second image frame being captured earlier in the sequence than the first image frame;

registers the first image frame with the second image frame;

applies a global optimization to adjust registration parameters for the plurality of pair-wise registrations of image frames within the closed loop using global constraints;

identifies a registration break in the closed loop, the registration break being two consecutive frames i and i+1 in the sequence;

assigns all registration errors in the closed loop to a pair-wise registration between frames i and i+1; and

updates pair-wise transformation parameters according to a transformation matrix for frames i and i+1.

8. An apparatus as in claim 7 wherein adjusting the registration parameters comprises calculating an error measurement that includes multiple global constraints.

9. An apparatus as in claim 8 wherein the global constraints comprise three independent scalar constraints.

10. An apparatus as in claim 8 wherein the processor further extracts matched feature point pairs for the plurality of pair-wise registrations.

11. An apparatus as in claim 10 wherein calculating the error measurement further includes matched feature point errors.

12. An apparatus for correcting an error accumulated from a plurality of pair-wise registrations of image frames captured in sequence by a scanning device, comprising:

a processor that;

identifies a closed loop where a first image frame overlaps with a second image frame, the second image frame being captured earlier in the sequence than the first image frame;

registers the first image frame with the second image frame;

applies a global optimization to adjust registration parameters for the plurality of pair-wise registrations of image frames within the closed loop using global constraints; and

marks all pair-wise registrations of image frames within the closed loop as fixed;

wherein the registration parameters of pair-wise registrations of image frames within the closed loop that have been marked as fixed are not adjusted during the global optimization.

13. One or more tangible, non-transitory computer-readable media having computer-readable instructions thereon, which, when executed by a processor, correct an error accumulated from a plurality of pair-wise registrations of image frames captured in sequence by a scanning device, wherein the processor:

identifies a closed loop where a first image frame overlaps with a second image frame, the second image frame being captured earlier in the sequence than the first image frame;

registers the first image frame with the second image frame;

applies a global optimization to adjust registration parameters for the plurality of pair-wise registrations of image frames within the closed loop using global constraints;

identifies a registration break in the closed loop, the registration break being between two consecutive frames i and i+1 in the sequence;

assigns all registration errors in the closed loop to a pair-wise registration between frames i and i+1; and

updates pair-wise transformation parameters according to a transformation matrix for frames i and i+1.

14. One or more tangible, non-transitory computer-readable media as in claim 13 wherein adjusting the registration parameters comprises calculating an error measurement that includes multiple global constraints.

15. One or more tangible, non-transitory computer-readable media as in claim 14 further comprising extracting matched feature point pairs for the plurality of pair-wise registrations.

16. One or more tangible, non-transitory computer-readable media as in claim 14 wherein the processor further extracts matched feature point pairs for the plurality of pair-wise registrations.

17. One or more tangible, non-transitory computer-readable media as in claim 16 wherein calculating the error measurement further includes matched feature point errors.

18. One or more tangible, non-transitory computer-readable media having computer-readable instructions thereon, which, when executed by a processor, correct an error accumulated from a plurality of pair-wise registrations of image frames captured in sequence by a scanning device, wherein the processor:

identifies a closed loop where a first image frame overlaps with a second image frame, the second image frame being captured earlier in the sequence than the first image frame;

registers the first image frame with the second image frame;

applies a global optimization to adjust registration parameters for the plurality of pair-wise registrations of image frames within the closed loop using global constraints; and

marks all pair-wise registrations of image frames within the closed loop as fixed;

wherein the registration parameters of pair-wise registrations of image frames within the closed loop that have been marked as fixed are not adjusted during the global optimization.