Detection of an object in a distorted image

• US 10,255,507 B2
• Filed: 12/19/2016
• Issued: 04/09/2019
• Est. Priority Date: 12/19/2016
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Taylor, Geoffrey Richard
• Primary Examiner(s): Bezuayehu, Solomon G
• Application Number:
  US15/384,090
• Publication Number:
  US 20180173963A1
• Time in Patent Office: 841 Days
• Field of Search: 382170-174
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image
• CPC Class Codes: G06T 3/60 Rotation of whole images or parts thereof
• CPC Class Codes: G06V 10/247 by affine transforms, e.g. correction due to perspective effects; Quadrilaterals, e.g. trapezoids
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

A system and method for determining an object in a distorted image. The method comprises detecting a location of a first part of the object on the distorted image; determining a corresponding location of the first part of the object on an undistorted image plane corresponding to the distorted image based on a plurality of calibration parameters; predicting a location of a second part of the object on the undistorted image plane based on the determined location of the first part on the undistorted image plane; determining a corresponding location of the second part of the object on the distorted image based on the predicted location of the second part of the object on the undistorted image plane and the calibration parameters; and determining the object based on the detected location of the first part of the object and the determined location of the second part in the distorted image.

Claims

1. A method for determining an object in a distorted image, comprising:

detecting a location of a first part of the object on the distorted image on a distorted image plane;

determining a corresponding location of the first part of the object on an undistorted image plane corresponding to the distorted image based on a plurality of calibration parameters;

predicting a location of a second part of the object on the undistorted image plane based on the determined location of the first part of the object on the undistorted image plane;

determining a corresponding location of the second part of the object on the distorted image based on the predicted location of the second part of the object on the undistorted image plane and the calibration parameters;

determining the object based on the detected location of the first part of the object and the determined location of the second part of the object in the distorted image;

generating a rectified image of the object based on a determined bounding box;

determining an appearance descriptor based on the rectified image; and

matching the image of the object to at least one other image of the object based on the determined appearance descriptors.

2. The method of claim 1, wherein the object is a person, the first part of the object is the head of the person and the second part of the object is a foot of the person.

3. The method of claim 1, wherein the object is a vehicle, the first part of the object is a rear wheel of the vehicle and the second part of the object is a front wheel of the vehicle.

4. The method of claim 1, wherein the object is determined by determining the bounding box of the object in the distorted image.

5. The method of claim 4, wherein the bounding box in the distorted image is determined based on determined locations of the first and second parts and a foreground mask.

6. The method of the claim 4, further comprising:

rotating a sub-image in the bounding box to an upright orientation.

7. The method of claim 4, further comprising:

applying a projective rotation of a sub-image in the bounding box to align a centroid of the bounding box to the principal point of a virtual camera with zero roll.

8. The method of claim 1, further comprising:

determining an initial bounding box based on the determined locations of the first object part and second object part on the distorted image;

counting a number of foreground pixels for each image row in the bounding box; and

determining a refined bounding box based on highest and lowest image rows in the initial bounding box with a number of foreground pixels exceeding a threshold.

9. The method of claim 1, further comprising:

generating a rectified image of the object and displaying the rectified image to an operator via a graphical user interface.

10. The method according to claim 1, wherein the calibration parameters comprise at least one of tilt angle, roll angle and focal length of an image capture device used to capture the distorted image, the calibration parameters used to derive a vanishing point of the object.

11. The method according to claim 1, wherein the calibration parameters comprise at least a distortion centre and distortion coefficients of the distorted image.

12. The method according to claim 1, wherein the calibration parameters comprise at least a distortion centre and distortion coefficients of the distorted image, the distortion centre and the distortion coefficients determined from the distorted image.

13. The method according to claim 1, wherein the location of a second part of the object on the undistorted image plane is predicted using a point on a vertical vanishing line of the undistorted image plane.

14. A non-transitory computer readable medium having a program stored thereon for determining an object in a distorted image, the program comprising:

code for detecting a location of a first part of the object on the distorted image on a distorted image plane;

code for determining a corresponding location of the first part of the object on an undistorted image plane corresponding to the distorted image based on a plurality of calibration parameters;

code for predicting a location of a second part of the object on the undistorted image plane based on the determined location of the first part of the object on the undistorted image plane;

code for determining a corresponding location of the second part of the object on the distorted image based on the predicted location of the second part of the object on the undistorted image plane and the calibration parameters;

code for determining the object based on the detected location of the first part of the object and the determined location of the second part of the object in the distorted image;

code for generating a rectified image of the object based on a determined bounding box;

determining an appearance descriptor based on the rectified image; and

matching the image of the object to at least one other image of the object based on the determined appearance descriptors.

17. The non-transitory computer readable medium of claim 14, wherein the object is a person or a vehicle, the first part of the object is the head of the person or a rear wheel of the vehicle and the second part of the object is a foot of the person or a front wheel of the vehicle, respectively.

15. An apparatus for determining an object in a distorted image, the apparatus configured to detect a location of a first part of the object on the distorted image on a distorted image plane;

determine a corresponding location of the first part of the object on an undistorted image plane corresponding to the distorted image based on a plurality of calibration parameters;

predict a location of a second part of the object on the undistorted image plane based on the determined location of the first part of the object on the undistorted image plane;

determine a corresponding location of the second part of the object on the distorted image based on the predicted location of the second part of the object on the undistorted image plane and the calibration parameters;

determine the object based on the detected location of the first part of the object and the determined location of the second part of the object in the distorted image;

generate a rectified image of the object based on a determined bounding box;

determine an appearance descriptor based on the rectified image; and

match the image of the object to at least one other image of the object based on the determined appearance descriptors.

18. The apparatus of claim 15, wherein the object is a person or a vehicle, the first part of the object is the head of the person or a rear wheel of the vehicle and the second part of the object is a foot of the person or a front wheel of the vehicle, respectively.

16. A system comprising:

an image capture device configured to capture a distorted image;

a memory for storing data and a computer readable medium; and

a processor coupled to the memory for executing a computer program, the processor in communication with the image capture device via a network, the program having instructions for;

detecting a location of a first part of an object on a distorted image on a distorted image plane received from the image capture device;

determine a corresponding location of the first part of the object on an undistorted image plane corresponding to the distorted image based on a plurality of calibration parameters;

predict a location of a second part of the object on the undistorted image plane based on the determined location of the first part of the object on the undistorted image plane;

determine a corresponding location of the second part of the object on the distorted image based on the predicted location of the second part of the object on the undistorted image plane and the calibration parameters;

determine the object based on the detected location of the first part of the object and the determined location of the second part of the object in the distorted image;

generate a rectified image of the object based on a determined bounding box;

determine an appearance descriptor based on the rectified image; and

match the image of the object to at least one other image of the object based on the determined appearance descriptors.

19. The system of claim 16, wherein the object is a person or a vehicle, the first part of the object is the head of the person or a rear wheel of the vehicle and the second part of the object is a foot of the person or a front wheel of the vehicle, respectively.