System and method for lateral vehicle detection

• US 10,685,239 B2
• Filed: 03/18/2018
• Issued: 06/16/2020
• Est. Priority Date: 03/18/2018
• Status: Active Grant

• Current Assignee: TuSimple, Inc. (TuSimple Holdings, Inc.)
• Original Assignee: TuSimple, Inc. (TuSimple Holdings, Inc.)
• Inventors: Yan, Zhipeng, Ge, Lingting, Chen, Pengfei, Wang, Panqu
• Primary Examiner(s): Shin, Soo
• Application Number:
  US15/924,249
• Publication Number:
  US 20190286916A1
• Time in Patent Office: 821 Days
• Field of Search: None
• CPC Class Codes: G05D 1/0055 with safety arrangements
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G06T 2207/30248 Vehicle exterior or interior
• CPC Class Codes: G06T 3/18 Image warping, e.g. rearranging pixels individually
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 10/443 by matching or filtering
• CPC Class Codes: G06V 10/7515 Shifting the patterns to accommodate for positional errors
• CPC Class Codes: G06V 10/757 Matching configurations of points or features
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G08G 1/04 using optical or ultrasonic detectors
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection

Abstract

A system and method for lateral vehicle detection is disclosed. A particular embodiment can be configured to: receive lateral image data from at least one laterally-facing camera associated with an autonomous vehicle; warp the lateral image data based on a line parallel to a side of the autonomous vehicle; perform object extraction on the warped lateral image data to identify extracted objects in the warped lateral image data; and apply bounding boxes around the extracted objects.

Claims

1. A system comprising:

a data processor; and

an autonomous lateral vehicle detection system, executable by the data processor, the autonomous lateral vehicle detection system being configured to perform an autonomous lateral vehicle detection operation for autonomous vehicles, the autonomous lateral vehicle detection operation being configured to;

receive lateral image data from at least one laterally-facing camera associated with an autonomous vehicle;

warp the lateral image data based on a line parallel to a side of the autonomous vehicle defined with configuration parameters corresponding to an installation orientation of the at least one laterally-facing camera on the autonomous vehicle;

perform object extraction on the warped lateral image data to identify extracted objects in the warped lateral image data; and

apply bounding boxes around the extracted objects.

2. The system of claim 1 wherein the at least one laterally-facing camera comprises a plurality of laterally-facing cameras, wherein the system is further configured to receive the lateral image data from the plurality of laterally-facing cameras of the autonomous vehicle, the autonomous lateral vehicle detection operation being further configured to:

identify matching portions of extracted features from the warped lateral image data from different ones of the plurality of laterally-facing cameras;

stitch together the lateral image data based on the matching portions of the extracted features; and

stitch together bounding boxes based on the matching portions of the extracted objects.

3. The system of claim 1 wherein the at least one laterally-facing camera is from the group consisting of:

a forward laterally-facing camera and a backward/rear-facing camera.

4. The system of claim 1 wherein the line parallel to a side of the autonomous vehicle is configured based on an installation of the laterally-facing camera in the autonomous vehicle and the configuration parameters further correspond to an orientation of the bottom edge of the lateral image data.

5. The system of claim 1 wherein the warped lateral image data creates a trapezoidal image.

6. The system of claim 1 wherein the bounding boxes are rectangular.

7. The system of claim 2 wherein the stitched lateral image data combines to form a single contiguous image representing a combined image from multiple laterally-facing cameras.

8. A method comprising:

receiving lateral image data from at least one laterally-facing camera associated with an autonomous vehicle;

warping the lateral image data based on a line parallel to a side of the autonomous vehicle defined with configuration parameters corresponding to an installation orientation of the at least one laterally-facing camera on the autonomous vehicle;

performing object extraction on the warped lateral image data to identify extracted objects in the warped lateral image data; and

applying bounding boxes around the extracted objects.

9. The method of claim 8 wherein the at least one laterally-facing camera comprises a plurality of laterally-facing cameras, wherein the method including receiving the lateral image data from the plurality of laterally-facing cameras of the autonomous vehicle, the method further including:

identifying matching portions of extracted features from the warped lateral image data from different ones of the plurality of laterally-facing cameras; and

stitching together the lateral image data based on the matching portions of the extracted features.

10. The method of claim 8 wherein the at least one laterally-facing camera is from the group consisting of:

a forward laterally-facing camera and a backward/rear-facing camera.

11. The method of claim 8 wherein the configuration parameters further correspond to an orientation of the bottom edge of the lateral image data.

12. The method of claim 8 wherein the warped lateral image data creates a trapezoidal image.

13. The method of claim 8 wherein the bounding boxes are rectangular.

14. The method of claim 9 wherein the stitched lateral image data represents a combined image from multiple laterally-facing cameras.

15. A non-transitory machine-useable storage medium embodying instructions which, when executed by a machine, cause the machine to:

receive lateral image data from at least one laterally-facing camera associated with an autonomous vehicle;

warp the lateral image data based on a line parallel to a side of the autonomous vehicle defined with configuration parameters corresponding to an installation orientation of the at least one laterally-facing camera on the autonomous vehicle;

perform object extraction on the warped lateral image data to identify extracted objects in the warped lateral image data; and

apply bounding boxes around the extracted objects.

16. The non-transitory machine-useable storage medium of claim 15 wherein the at least one laterally-facing camera comprises a plurality of laterally-facing cameras, wherein the non-transitory machine-useable storage medium is further configured to receive the lateral image data from the plurality of laterally-facing cameras of the autonomous vehicle, the autonomous lateral vehicle detection operation being further configured to:

identify matching portions of extracted features from the warped lateral image data from different ones of the plurality of laterally-facing cameras; and

stitch together bounding boxes based on the matching portions of the extracted objects.

17. The non-transitory machine-useable storage medium of claim 15 wherein the at least one laterally-facing camera is from the group consisting of:

a forward laterally-facing camera and a backward/rear-facing camera.

18. The non-transitory machine-useable storage medium of claim 15 wherein the line parallel to a side of the autonomous vehicle is configured based on an installation of the laterally-facing camera in the autonomous vehicle and the configuration parameters further correspond to an orientation of an edge of the lateral image data.

19. The non-transitory machine-useable storage medium of claim 15 wherein the warped lateral image data creates a trapezoidal image.

20. The non-transitory machine-useable storage medium of claim 16 wherein the stitched lateral image data combines to form a single contiguous image.