System and method for integrated pair-wise registration of images using image based information and sensor coordinate and error information

• US 8,279,500 B2
• Filed: 01/27/2011
• Issued: 10/02/2012
• Est. Priority Date: 01/27/2011
• Status: Active Grant

• Current Assignee: Seiko Epson Corporation (Seiko Group)
• Original Assignee: Seiko Epson Corporation (Seiko Group)
• Inventors: Fu, Guoyi, Brusnitsyn, Mikhail
• Primary Examiner(s): Worku, Negussie
• Application Number:
  US13/015,355
• Publication Number:
  US 20120194879A1
• Time in Patent Office: 614 Days
• Field of Search: 358/474, 358/488, 358/449, 358/468, 358/486, 382/141, 382/113, 382/112, 382/238
• US Class Current: 358/474
• CPC Class Codes: H04N 1/047 Detection, control or error compensation of scanning velocity or position H04N1/0402 and H04N1/17 take precedence
• CPC Class Codes: H04N 1/107 with manual scanning
• CPC Class Codes: H04N 1/387 Composing, repositioning or otherwise geometrically modifying originals
• CPC Class Codes: H04N 2201/0414 Scanning an image in a series of overlapping zones
• CPC Class Codes: H04N 2201/0471 using dedicated detectors
• CPC Class Codes: H04N 2201/04732 Detecting at infrequent intervals, e.g. once or twice per line for main-scan control
• CPC Class Codes: H04N 2201/04787 by changing or controlling the addresses or values of pixels, e.g. in an array, in a memory, by interpolation

Abstract

Two images are stitched together through minimization of a cost function that consists of registration errors from image data of the two images, as well as the estimated errors from a set of sensors. The weight function in the cost function is derived from the confidence value of sensor estimation that considers the sensor errors including lift and off page as well as a measure of accuracy of the sensor readings. Weights are used to adjust image registration accuracy against sensor accuracy to produce a set of registration parameters that would best stitch the two images together. In order to handle large errors for initial registration parameters and to avoid local minima in the minimization process, the image pair may be registered in a lower resolution and then refined in a higher resolution.

Claims

1. A method for registering two image frames captured by a scanning device, the scanning device comprising an image capturing unit and a plurality of position sensors that supply position data indicating the relative position of the image capturing device when the image capturing device captures each of the two image frames, comprising:

reading position data from each of the plurality of position sensors when a first image frame i is captured and when a second image frame i+1 is captured;

estimating transformation parameters from the sensor position data that relate an image center of image frame i+1 to image frame i;

extracting an overlapping area between image frame i+1 and image frame i using the estimated transformation parameters from the sensor position data; and

refining the transformation parameters by minimizing a cost function integrating a registration error ES from sensor position data and a registration error EI from image data of the overlapping area to register image frame i+1 with image frame i.

2. A method as in claim 1 wherein integrating the registration error ES and the registration error EI comprises weighting at least one of the registration error ES or the registration error EI.

3. A method as in claim 2 wherein the weighting is a function of confidence in the sensor position data.

4. A method as in claim 3 wherein the confidence is a measure of a semi-axis of error ellipse.

5. A method as in claim 1 wherein the steps of reading, estimating, extracting, and refining are performed first in a first resolution and then performed again in a second resolution that is higher than the first resolution.

6. A method as in claim 5 wherein the first resolution is selected based upon confidence in the sensor position data.

7. A scanning device comprising:

an image capturing unit;

a plurality of position sensors that supply position data indicating the relative position of the image capturing device when the image capturing device captures one of a plurality of image frames; and

a processor that;

reads position data from each of the plurality of position sensors when a first image frame i is captured and when a second image frame i+1 is captured;

estimates transformation parameters from the sensor position data that relate an image center of image frame i+1 to image frame i; and

extracts an overlapping area between image frame i+1 and image frame i using the estimated transformation parameters from the sensor position data; and

refines the transformation parameters by minimizing a cost function integrating a registration error ES from sensor position data and a registration error EI from image data of the overlapping area to register image frame i+1 with image frame i.

8. A scanning device as in claim 7 wherein integrating the registration error ES and the registration error EI comprises weighting at least one of the registration error ES or the registration error EI.

9. A scanning device as in claim 8 wherein the weighting is a function of confidence in the sensor position data.

10. A scanning device as in claim 9 wherein the confidence is a measure of a semi-axis of error ellipse.

11. A scanning device as in claim 7 wherein the steps of reading, estimating, extracting, and refining are performed first in a first resolution and then performed again in a second resolution that is higher than the first resolution.

12. A scanning device as in claim 11 wherein the first resolution is selected based upon confidence in the sensor position data.

13. A non-transitory computer-readable medium encoded with computer-executable instructions for performing the following:

read position data from each of a plurality of position sensors when a first image frame i is captured and when a second image frame i+1 is captured;

estimates transformation parameters from the sensor position data that relate an image center of image frame i+1 to image frame i;

extract an overlapping area between image frame i+1 and image frame i using the estimated transformation parameters from the sensor position data; and

refine the transformation parameters by minimizing a cost function integrating a registration error ES from sensor position data and a registration error EI from image data of the overlapping area to register image frame i+1 with image frame i.

14. The non-transitory computer-readable medium as in claim 13 wherein integrating the registration error ES and the registration error EI comprises weighting at least one of the registration error ES or the registration error EI.

15. The non-transitory computer-readable medium as in claim 14 wherein the weighting is a function of confidence in the sensor position data.

16. The non-transitory computer-readable medium as in claim 15 wherein the confidence is a measure of a semi-axis of error ellipse.

17. The non-transitory computer-readable medium as in claim 13 wherein the steps of reading, estimating, extracting, and refining are performed first in a first resolution and then performed again in a second resolution that is higher than the first resolution.

18. The non-transitory computer-readable medium as in claim 17 wherein the first resolution is selected based upon confidence in the sensor position data.