Single image rectification

• US 9,904,990 B2
• Filed: 12/18/2015
• Issued: 02/27/2018
• Est. Priority Date: 12/18/2015
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Moraleda, Jorge, Prashnani, Ekta, Gormish, Michael J., Berkner, Kathrin, Savarese, Silvio
• Primary Examiner(s): Liu, Li
• Application Number:
  US14/975,644
• Publication Number:
  US 20170178301A1
• Time in Patent Office: 802 Days
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20048 Transform domain processing
• CPC Class Codes: G06T 2207/20061 Hough transform
• CPC Class Codes: G06T 2207/20212 Image combination
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image
• CPC Class Codes: G06T 3/02 Affine transformations for image registration G06T3/147; for image mosaicing G06T3/4038
• CPC Class Codes: G06T 5/10 using non-spatial domain filtering
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06T 7/62 of area, perimeter, diameter or volume
• CPC Class Codes: G06V 10/16 using multiple overlapping images; Image stitching
• CPC Class Codes: G06V 10/243 by compensating for image skew or non-uniform image deformations

Abstract

The disclosure includes a system and method for performing image rectification using a single image and information identified from the single image. An image recognition application receives an input image, identifies a plurality of objects in the input image, estimates rectification parameters for the plurality of objects, identifies a plurality of candidate rectification parameters using a voting procedure on the rectification parameters for the plurality of objects, estimates final rectification parameters based on the plurality of candidate rectification parameters, computes a global transformation matrix using the final rectification parameters, and performs image rectification on the input image using the global transformation matrix.

Claims

1. A method comprising:

receiving, by one or more processors, an input image;

identifying, by the one or more processors, a plurality of objects in the input image;

estimating, by the one or more processors, a local transformation matrix for each of the plurality of objects;

calculating rectification parameters for the plurality of objects using the local transformation matrix;

identifying, by the one or more processors, a plurality of candidate rectification parameters using a voting procedure on the rectification parameters for the plurality of objects;

estimating, by the one or more processors, final rectification parameters based on the plurality of candidate rectification parameters;

computing, by the one or more processors, a global transformation matrix using the final rectification parameters; and

performing, by the one or more processors, image rectification on the input image using the global transformation matrix.

2. The computer-implemented method of claim 1, wherein estimating the final rectification parameters based on the plurality of candidate rectification parameters comprises:

identifying, from the plurality of objects, a set of objects corresponding to the plurality of candidate rectification parameters; and

calculating the final rectification parameters based on the set of objects.

3. The computer-implemented method of claim 1, wherein identifying the plurality of candidate rectification parameters using the voting procedure on the rectification parameters for the plurality of objects comprises identifying the candidate rectification parameters with a maximum number of votes.

4. The computer-implemented method of claim 1, wherein estimating the local transformation matrix for each of the plurality of objects comprises:

determining a region of interest associated with an object in the input image;

retrieving an indexed image of the object from a database;

retrieving a region of interest associated with the object in the indexed image;

mapping the region of interest of the object in the input image to the region of interest of the object in the indexed image; and

estimating the local transformation matrix based on the mapping.

5. The computer-implemented method of claim 1, wherein the local transformation matrix is an affine transformation matrix.

6. The computer-implemented method of claim 1, further comprising:

performing image rectification on a plurality of input images; and

performing image stitching on the rectified input images.

7. The computer-implemented method of claim 1, further comprising performing image recognition on the rectified input image.

8. A system comprising;

one or more processors; and

a memory, the memory storing instructions, which when executed cause the one or more processor to;

receive an input image;

identify a plurality of objects in the input image;

estimate a local transformation matrix for each of the plurality of objects;

calculate rectification parameters for the plurality of objects using the local transformation matrix;

identify a plurality of candidate rectification parameters using a voting procedure on the rectification parameters for the plurality of objects;

estimate final rectification parameters based on the plurality of candidate rectification parameters;

compute a global transformation matrix using the final rectification parameters; and

perform image rectification on the input image using the global transformation matrix.

9. The system of claim 8, wherein to estimate the final rectification parameters based on the plurality of candidate rectification parameters, the instructions cause the one or more processors to:

identify, from the plurality of objects, a set of objects corresponding to the plurality of candidate rectification parameters; and

calculate the final rectification parameters based on the set of objects.

10. The system of claim 8, wherein to identify the plurality of candidate rectification parameters using the voting procedure on the rectification parameters for the plurality of objects, the instructions cause the one or more processors to identify the candidate rectification parameters with a maximum number of votes.

11. The system of claim 8, wherein to estimate the local transformation matrix for each of the plurality of objects, the instructions cause the one or more processors to:

determine a region of interest associated with an object in the input image;

retrieve an indexed image of the object from a database;

retrieve a region of interest associated with the object in the indexed image;

map the region of interest of the object in the input image to the region of interest of the object in the indexed image; and

estimate the local transformation matrix based on the mapping.

12. The system of claim 8, wherein the local transformation matrix is an affine transformation matrix.

13. The system of claim 8, wherein the instructions cause the one or more processors to:

perform image rectification on a plurality of input images; and

perform image stitching on the rectified input images.

14. The system of claim 8, wherein the instructions cause the one or more processors to perform image recognition on the rectified input image.

15. A computer program product comprising a non-transitory computer readable medium storing a computer readable program, wherein the computer readable program when executed causes a computer to:

receive an input image;

identify a plurality of objects in the input image;

estimate a local transformation matrix for each of the plurality of objects;

calculate rectification parameters for the plurality of objects using the local transformation matrix;

identify a plurality of candidate rectification parameters using a voting procedure on the rectification parameters for the plurality of objects;

estimate final rectification parameters based on the plurality of candidate rectification parameters;

compute a global transformation matrix using the final rectification parameters; and

perform image rectification on the input image using the global transformation matrix.

16. The computer program product of claim 15, wherein to estimate the final rectification parameters based on the plurality of candidate rectification parameters, the computer readable program causes the computer to:

identify, from the plurality of objects, a set of objects corresponding to the plurality of candidate rectification parameters; and

calculate the final rectification parameters based on the set of objects.

17. The computer program product of claim 15, wherein to identify the plurality of candidate rectification parameters using the voting procedure on the rectification parameters for the plurality of objects, the computer readable program causes the computer to identify the candidate rectification parameters with a maximum number of votes.

18. The computer program product of claim 15, wherein to estimate the local transformation matrix for each of the plurality of objects, the computer readable program causes the computer to:

determine a region of interest bordering an object in the input image;

retrieve an indexed image of the object from a database;

retrieve a region of interest bordering the object in the indexed image;

map the region of interest of the object in the input image to the region of interest of the object in the indexed image; and

estimate the local transformation matrix based on the mapping.

19. The computer program product of claim 15, wherein the computer readable program causes the computer to:

perform image rectification on a plurality of input images; and

perform image stitching on the rectified input images.

20. The computer program product of claim 15, wherein the computer readable program causes the computer to perform image recognition on the rectified input image.