Single image rectification

• US 10,489,893 B2
• Filed: 01/29/2018
• Issued: 11/26/2019
• Est. Priority Date: 12/18/2015
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Moraleda, Jorge, Prashnani, Ekta, Gormish, Michael J., Berkner, Kathrin, Savarese, Silvio
• Primary Examiner(s): Liu, Li
• Application Number:
  US15/882,976
• Publication Number:
  US 20180150945A1
• Time in Patent Office: 666 Days
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20048 Transform domain processing
• CPC Class Codes: G06T 2207/20061 Hough transform
• CPC Class Codes: G06T 2207/20212 Image combination
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image
• CPC Class Codes: G06T 3/02 Affine transformations for image registration G06T3/147; for image mosaicing G06T3/4038
• CPC Class Codes: G06T 5/10 using non-spatial domain filtering
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06T 7/62 of area, perimeter, diameter or volume
• CPC Class Codes: G06V 10/16 using multiple overlapping images; Image stitching
• CPC Class Codes: G06V 10/243 by compensating for image skew or non-uniform image deformations

Abstract

The disclosure includes a system and method for performing image rectification using a single image and information identified from the single image. An image recognition application receives an input image, identifies a plurality of objects in the input image, estimates rectification parameters for the plurality of objects, identifies a plurality of candidate rectification parameters using a voting procedure on the rectification parameters for the plurality of objects, estimates final rectification parameters based on the plurality of candidate rectification parameters, computes a global transformation matrix using the final rectification parameters, and performs image rectification on the input image using the global transformation matrix.

Claims

1. A method comprising:

receiving, by one or more processors, an input image;

identifying, by the one or more processors, a plurality of objects in the input image;

determining a region of interest associated with an object in the input image;

retrieving an indexed image of the object from a database;

retrieving a region of interest associated with the object in the indexed image;

mapping the region of interest of the object in the input image to the region of interest of the object in the indexed image;

estimating a local transformation matrix based on the mapping;

estimating, by the one or more processors, rectification parameters for the object based on the local transformation matrix;

computing, by the one or more processors, a global transformation matrix based on the rectification parameters; and

performing, by the one or more processors, image rectification on the input image using the global transformation matrix.

2. The computer-implemented method of claim 1, further comprising:

identifying, by the one or more processors, a plurality of candidate rectification parameters using a voting procedure on rectification parameters for the plurality of objects; and

estimating, by the one or more processors, final rectification parameters based on the plurality of candidate rectification parameters, wherein the global transformation matrix is based on the final rectification parameters.

3. The computer-implemented method of claim 2, wherein estimating the final rectification parameters based on the plurality of candidate rectification parameters comprises:

identifying, from the plurality of objects, a set of objects corresponding to the plurality of candidate rectification parameters; and

calculating the final rectification parameters based on the set of objects.

4. The computer-implemented method of claim 2, wherein estimating the rectification parameters for the plurality of objects comprises:

estimating a local transformation matrix for each of the plurality of objects; and

calculating the rectification parameters for the plurality of objects using the local transformation matrix.

5. The computer-implemented method of claim 1, wherein the local transformation matrix is an affine transformation matrix.

6. The computer-implemented method of claim 1, further comprising:

performing image rectification on a plurality of input images; and

performing image stitching on the rectified input images.

7. The computer-implemented method of claim 1, further comprising performing image recognition on the rectified input image.

8. A system comprising;

one or more processors; and

a memory, the memory storing instructions, which when executed cause the one or more processor to;

receive an input image;

identify a plurality of objects in the input image;

determine a region of interest associated with an object in the input image;

retrieve an indexed image of the object from a database;

retrieve a region of interest associated with the object in the indexed image;

map the region of interest of the object in the input image to the region of interest of the object in the indexed image;

estimate a local transformation matrix based on the mapping;

estimate rectification parameters for the object based on the local transformation matrix;

compute a global transformation matrix based on the rectification parameters; and

perform image rectification on the input image using the global transformation matrix.

9. The system of claim 8, wherein the instructions further cause the one or more processors to:

identify a plurality of candidate rectification parameters using a voting procedure on rectification parameters for the plurality of objects; and

estimate final rectification parameters based on the plurality of candidate rectification parameters, wherein the global transformation matrix is based on the final rectification parameters.

10. The system of claim 9, wherein to estimate the final rectification parameters based on the plurality of candidate rectification parameters, the instructions cause the one or more processors to:

identify, from the plurality of objects, a set of objects corresponding to the plurality of candidate rectification parameters; and

calculate the final rectification parameters based on the set of objects.

11. The system of claim 9, wherein to estimate the rectification parameters for the plurality of objects, the instructions cause the one or more processors to:

estimate a local transformation matrix for each of the plurality of objects; and

calculate the rectification parameters for the plurality of objects using the local transformation matrix.

12. The system of claim 9, wherein the local transformation matrix is an affine transformation matrix.

13. The system of claim 8, wherein the instructions cause the one or more processors to:

perform image rectification on a plurality of input images; and

perform image stitching on the rectified input images.

14. The system of claim 8, wherein the instructions cause the one or more processors to perform image recognition on the rectified input image.

15. A computer program product comprising a non-transitory computer readable medium storing a computer readable program, wherein the computer readable program when executed causes a computer to:

receive an input image;

identify a plurality of objects in the input image;

determine a region of interest associated with an object in the input image;

retrieve an indexed image of the object from a database;

retrieve a region of interest associated with the object in the indexed image;

map the region of interest of the object in the input image to the region of interest of the object in the indexed image;

estimate a local transformation matrix based on the mapping;

estimate rectification parameters for the object based on the local transformation matrix;

compute a global transformation matrix based on the rectification parameters; and

perform image rectification on the input image using the global transformation matrix.

16. The computer program product of claim 15, wherein the computer readable program further causes the computer to:

identify a plurality of candidate rectification parameters using a voting procedure on rectification parameters for the plurality of objects; and

estimate final rectification parameters based on the plurality of candidate rectification parameters, wherein the global transformation matrix is based on the final rectification parameters.

17. The computer program product of claim 16, wherein to estimate the final rectification parameters based on the plurality of candidate rectification parameters, the computer readable program causes the computer to:

identify, from the plurality of objects, a set of objects corresponding to the plurality of candidate rectification parameters; and

calculate the final rectification parameters based on the set of objects.

18. The computer program product of claim 16, wherein to estimate the rectification parameters for the plurality of objects, the computer readable program causes the computer to:

estimate a local transformation matrix for each of the plurality of objects; and

calculate the rectification parameters for the plurality of objects using the local transformation matrix.

19. The computer program product of claim 15, wherein the computer readable program causes the computer to:

perform image rectification on a plurality of input images; and

perform image stitching on the rectified input images.

20. The computer program product of claim 15, wherein the computer readable program causes the computer to perform image recognition on the rectified input image.