Image and map-based detection of vehicles at intersections

• US 9,495,602 B2
• Filed: 10/23/2013
• Issued: 11/15/2016
• Est. Priority Date: 10/23/2013
• Status: Active Grant

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Inventors: Okumura, Bunyo
• Primary Examiner(s): Harandi, Siamak
• Assistant Examiner(s): TRAN, MAI H
• Application Number:
  US14/061,166
• Publication Number:
  US 20150110344A1
• Time in Patent Office: 1,119 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

A system, device, and methods for image and map-based detection of vehicles at intersections. Once example computer-implemented method for detecting objects includes receiving, from the one or more sensors disposed on a vehicle, image data representative of an image and detecting an object on the image. The method further includes identifying a path extending from the vehicle to the detected object on the image and retrieving map data including lane information. The method further includes comparing the path to a representation of the lane information and determining the position of the detected object based on a comparison of the path, representation of the lane information, and the image.

Claims

1. An object detection system, comprising:

one or more sensors disposed on a vehicle; and

a computing device in communication with the one or more sensors, comprising;

one or more processors for controlling operations of the computing device; and

a memory for storing data and program instructions used by the one or more processors, wherein the one or more processors are configured to execute instructions stored in the memory to;

receive, from the one or more sensors, image data of an external environment of the vehicle;

detect an object in the external environment based on the image data;

generate a bounding box surrounding the detected object;

generate a path extending from the vehicle to the detected object, the path including the bounding box, the path being independent from an intended travel path of the vehicle;

retrieve map data including lane information;

compare the path to a representation of the lane information;

determine a plurality of candidate positions for the detected object located between intersecting areas of the lane information and the path;

project the plurality of candidate positions onto the image data; and

determine a position of the detected object based on a comparison of the one or more candidate positions and the detected object in the image data.

2. The system of claim 1 wherein the size of each candidate position indicates the distance to each projected candidate position from the vehicle.

3. The system of claim 1 wherein the processors are further configured to determine a motion direction for the detected object based on at least one of visual attributes of the detected object on the image and a comparison of the position of the detected object in two or more time-sequenced images.

4. The system of claim 3 wherein determining the position of the detected object is further based on the motion direction of the detected object.

5. The system of claim 1 wherein the processors are further configured to determine a velocity for the detected object based on a comparison of the position of the detected object in two or more time-sequenced images.

6. The system of claim 1 wherein detecting the object includes comparing the map data to the image data to define a region of interest on the image and detection of the object occurs within the region of interest.

7. The system of claim 1, wherein detecting the object includes processing the image data using a cascaded detector.

8. The system of claim 1 wherein the lane information includes the number of lanes and the direction of each lane.

9. A computer-implemented method for detecting objects, comprising:

receiving, from one or more sensors disposed on a vehicle, image data of an external environment of the vehicle;

detecting an object in the external environment based on the image data;

generating a path extending from the vehicle to the detected object, the path including the detected object, the path being independent from an intended travel path of the vehicle;

retrieving map data including lane information;

comparing the path to a representation of the lane information; and

determining a position of the detected object based the comparison of the path to the representation of the lane information.

10. The method of claim 9 wherein the comparing of the path to the representation of the lane information includes determining one or more candidate positions for the detected object on the image.

11. The method of claim 10 wherein the one or more candidate positions are located between intersecting areas of the lane information and the path.

12. The method of claim 10 wherein the size of each candidate position indicates the distance to each projected candidate position from the vehicle.

13. The method of claim 9 wherein the processors are further configured to:

determine a motion direction for the detected object;

wherein the determination of the motion direction is based on at least one of visual attributes of the detected object on the image and a comparison of the position of the detected object in two or more time-sequenced images; and

wherein determining the position of the detected object is further based on the motion direction of the detected object.

14. A computing device, comprising:

one or more processors for controlling the operations of the computing device; and

a memory for storing data and program instructions used by the one or more processors, wherein the one or more processors are configured to execute instructions stored in the memory to;

receive, from one or more sensors disposed on a vehicle, image data of an external environment of the vehicle;

detect an object in the external environment based on the image data;

generate a path extending from the vehicle to the detected object, the path including the detected object, the path being independent from an intended travel path of the vehicle;

retrieve map data including lane information;

compare the path to a representation of the lane information; and

determine a position of the detected object based on the comparison of the path to the representation of the lane information.

15. The computing device of claim 14 wherein the comparing of the path to the representation of the lane information includes determining one or more candidate positions for the detected object on the image.

16. The computing device of claim 15 wherein the one or more candidate positions are located between intersecting areas of the lane information and the path.

17. The computing device of claim 15 wherein the size of each candidate position indicates the distance to each projected candidate position from the vehicle.

18. The computing device of claim 14 wherein the processors are further configured to:

determine a motion direction for the detected object;

wherein the determination of the motion direction is based on at least one of visual attributes of the detected object on the image and a comparison of the position of the detected object in two or more time-sequenced images; and

wherein determining the position of the detected object is further based on the motion direction of the detected object.