OBJECT DETECTION FOR DISTORTED IMAGES

• US 20200134331A1
• Filed: 09/30/2019
• Published: 04/30/2020
• Est. Priority Date: 10/31/2018
• Status: Active Grant

• Current Assignee: Texas Instruments, Inc.
• Original Assignee: Texas Instruments, Inc.
• Inventors: PODDAR, Deepak, NAGORI, Soyeb, MATHEW, Manu, MAJI, Debapriya
• Granted Patent: US 11,288,525 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 11/203 Drawing of straight lines or curves
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30264 Parking
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 20/586 of parking space

Abstract

Techniques including receiving a distorted image from a camera disposed about a vehicle, detecting, in the distorted image, corner points associated with a target object, mapping the corner points to a distortion corrected domain based on one or more camera parameters, mapping the corner points and lines between the corner points back to a distorted domain based on the camera parameters, interpolating one or more intermediate points to generate lines between the corner points in the distortion corrected domain mapping the corner points and the lines between the corner points back to a distorted domain based on the camera parameters, and adjusting a direction of travel of the vehicle based on the located target object.

Claims

1. A method comprising:

receiving a distorted image from a camera disposed about a vehicle;

detecting, in the distorted image, corner points associated with a target object;

mapping the corner points to a distortion corrected domain based on one or more camera parameters;

interpolating one or more intermediate points to generate lines between the corner points in the distortion corrected domain;

mapping the corner points and the lines between the corner points back to a distorted domain based on the camera parameters;

locating the target object in the distorted image based on pixels within the lines; and

adjusting a direction of travel of the vehicle based on the located target object.

2. The method of claim 1, the method further comprising:

mapping the one or more intermediate points back to the distorted domain based on the camera parameters; and

determining the pixels within the lines based on the corner points and intermediate points.

3. The method of claim 1, wherein the distorted image comprises a fisheye distorted image.

4. The method of claim 1, wherein the target object comprises a parking spot.

5. The method of claim 1, wherein the distorted image is a training image and the method further comprising:

determining a loss criteria between the detected corner points and corner points associated with the training image; and

adjusting one or more weights associated with corner point detectors based on the loss criteria.

6. The method of claim 5, further comprising:

drawing a virtual bounding box based on the corner points; and

comparing the virtual bounding box with a ground truth virtual bounding box associated with the training image.

7. The method of claim 6, wherein the one or more weights are adjusted based on the loss criteria and the comparison between the virtual bounding box and the ground truth virtual bounding box and wherein the loss criteria is prioritized above a result of the comparison between the virtual bounding box with the ground truth virtual bounding box when adjusting the one or more weights associated with the corner point detectors.

8. The method of claim 7, wherein comparing the pixels is based on a pixel based intersection over union comparison.

9. A non-transitory program storage device comprising instructions stored thereon to cause one or more processors to:

receive a distorted image;

detect, in the distorted image, corner points associated with a target object;

map the corner points to a distortion corrected domain based on one or more camera parameters;

interpolate one or more intermediate points to generate lines between the corner points in the distortion corrected domain;

map the corner points and lines between the corner points back to a distorted domain based on the camera parameters;

locate the target object in the distorted image based on pixels within the lines; and

output one or more instructions for adjusting a direction of travel based on the located target object.

10. The non-transitory program storage device of claim 9, wherein the stored instructions further cause one or more processors to:

map the one or more intermediate points back to the distorted domain based on the camera parameters; and

determine the pixels within the lines are further based on the corner points and intermediate points.

11. The non-transitory program storage device of claim 9, wherein the distorted image comprises a fisheye distorted image.

12. The non-transitory program storage device of claim 9, wherein the target object comprises a parking spot.

13. The non-transitory program storage device of claim 9, wherein the image is a training image, and wherein the stored instructions further cause one or more processors to:

determine a loss criteria between the detected corner points and corner points associated with the training image; and

adjust one or more weights associated with corner point detectors based on the loss criteria.

14. The non-transitory program storage device of claim 13, wherein the stored instructions further cause one or more processors to:

draw a virtual bounding box based on the corner points; and

compare the virtual bounding box with a ground truth virtual bounding box associated with the training image.

15. The non-transitory program storage device of claim 14, wherein the one or more weights are adjusted based on the loss criteria and the comparison between the virtual bounding box and the ground truth virtual bounding box and wherein the loss criteria is prioritized above a result of the comparison between the virtual bounding box with the ground truth virtual bounding box when adjusting the one or more weights associated with the corner point detectors.

16. A vehicle capable of detecting a parking spot, the vehicle comprising:

one or more cameras disposed about the vehicle;

a memory; and

one or more processors operatively coupled to the memory, and the one or more cameras, wherein the one or more processors are configured to execute non-transitory instructions causing the one or more processors to;

receive a distorted image from the one or more cameras;

detect, in the distorted image, corner points associated with a target object;

map the corner points to a distortion corrected domain based on one or more camera parameters;

interpolate one or more intermediate points to generate lines between the corner points in the distortion corrected domain;

map the corner points and lines between the corner points back to a distorted domain based on the camera parameters;

locate the target object in the distorted image based on pixels within the lines; and

adjust a direction of travel of the vehicle based on the located target object.

17. The vehicle of claim 16, wherein the non-transitory instructions further cause one or more processors to:

map the one or more intermediate points back to the distorted domain based on the camera parameters; and

determine the pixels within the lines are further based on the corner points and intermediate points.

18. The vehicle of claim 16, wherein the distorted image comprises a fisheye distorted image.

19. The vehicle of claim 16, wherein the target object comprises a parking spot.

20. The vehicle of claim 16, wherein the image is a training image and wherein the non-transitory instructions further cause one or more processors to:

determine a loss criteria between the detected corner points and corner points associated with the training image; and

adjust one or more weights associated with corner point detectors based on the loss criteria.