SYSTEMS AND METHODS FOR VEHICLE SIGNAL LIGHT DETECTION

• US 20170364758A1
• Filed: 09/01/2017
• Published: 12/21/2017
• Est. Priority Date: 09/01/2017
• Status: Active Grant

• Current Assignee: GM Global Technology Operations LLC (General Motors Company)
• Original Assignee: GM Global Technology Operations LLC (General Motors Company)
• Inventors: Minster, Gautier
• Granted Patent: US 10,163,017 B2
• Time in Patent Office: Days
• CPC Class Codes: G01S 17/04 Systems determining the presence of a target
• CPC Class Codes: G01S 17/06 Systems determining position data of a target
• CPC Class Codes: G01S 17/66 Tracking systems using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G05D 1/0248 in combination with a laser lasers per se H01S
• CPC Class Codes: G06F 18/2113 by ranking or filtering the set of features, e.g. using a measure of variance or of feature cross-correlation
• CPC Class Codes: G06F 18/2413 based on distances to training or reference patterns
• CPC Class Codes: G06F 18/25 Fusion techniques
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G08G 1/166 : for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

Systems and methods are provided for analyzing vehicle signal lights in order to operate an autonomous vehicle. A method includes receiving an image from a camera regarding a vehicle proximate to the autonomous vehicle. Data from a lidar sensor regarding the proximate vehicle is used to determine object information for identifying a subsection within the camera image. The identified subsection corresponds to an area of the proximate vehicle containing one or more vehicle signals. One or more vehicle signal lights of the proximate vehicle is located by using the identified camera image subsection as an area of focus.

Claims

1. A method for analyzing vehicle signal lights in order to operate an autonomous vehicle, comprising:

receiving, by one or more processors, an image from a camera regarding a vehicle proximate to the autonomous vehicle;

using, by the one or more processors, data from a lidar sensor regarding the proximate vehicle to determine object geometry and position information for identifying a subsection within the camera image, wherein the identified subsection corresponds to an area of the proximate vehicle containing one or more vehicle signals; and

locating, by the one or more processors, one or more vehicle signal lights of the proximate vehicle by using the identified camera image subsection as an area of focus;

wherein tracking of the proximate vehicle is based upon the located one or more vehicle signal lights of the proximate vehicle.

2. The method of claim 1, wherein the data from the lidar sensor provides object depth and distance information;

wherein the lidar sensor and camera are calibrated so that depth can be assigned to the camera image.

3. The method of claim 1, wherein behavior of the proximate vehicle is predicted based upon the located proximate vehicle signal lights.

4. The method of claim 1, wherein the locating of the one or more vehicle signal lights by using the identified camera image subsection results in filtering out false positives.

5. The method of claim 1, wherein the locating of the one or more vehicle signal lights includes using lane information and velocity of the proximate vehicle to differentiate between front of the proximate vehicle and back of the proximate vehicle.

6. The method of claim 1, wherein the camera image is from a fixed exposure camera.

7. The method of claim 1, wherein the one or more vehicle signal lights include turn signals, hazard lights, and brake lights.

8. The method of claim 1, wherein the identifying of the subsection of the camera image includes determining whether the camera image includes rear of the proximate vehicle or front of the proximate vehicle, wherein the identifying of the subsection of the camera image is based upon sizes of signal lights expected to be located within the camera image.

9. The method of claim 1, wherein the identifying of the subsection of the camera image eliminates other lights that are not vehicle signal lights, thereby reducing processing cycles by avoiding analysis of non-relevant portions of the camera image.

10. The method of claim 1, wherein the proximate vehicle is in front of the autonomous vehicle or to the side of the autonomous vehicle.

11. A system for analyzing vehicle signal lights in order to operate an autonomous vehicle, the system comprising:

a storage device for storing instructions for performing safety analysis of autonomous vehicle operation; and

one or more data processors configured to execute the instructions to;

receive an image from a camera regarding a vehicle proximate to the autonomous vehicle;

use data from a lidar sensor regarding the proximate vehicle to determine object geometry and position information for identifying a subsection within the camera image, wherein the identified subsection corresponds to an area of the proximate vehicle containing one or more vehicle signals; and

locate one or more vehicle signal lights of the proximate vehicle by using the identified camera image subsection as an area of focus;

wherein tracking of the proximate vehicle is based upon the located one or more vehicle signal lights of the proximate vehicle.

12. The system of claim 11, wherein the data from the lidar sensor provides object depth and distance information;

wherein the lidar sensor and camera are calibrated so that depth can be assigned to the camera image.

13. The system of claim 11, wherein behavior of the proximate vehicle is predicted based upon the located proximate vehicle signal lights.

14. The system of claim 11, wherein the locating of the one or more vehicle signal lights by using the identified camera image subsection results in filtering out false positives.

15. The system of claim 11, wherein the locating of the one or more vehicle signal lights includes using lane information and velocity of the proximate vehicle to differentiate between front of the proximate vehicle and back of the proximate vehicle.

16. The system of claim 11, wherein the camera image is from a fixed exposure camera.

17. The system of claim 11, wherein the one or more vehicle signal lights include turn signals, hazard lights, and brake lights.

18. The system of claim 11, wherein the identifying of the subsection of the camera image includes determining whether the camera image includes rear of the proximate vehicle or front of the proximate vehicle, wherein the identifying of the subsection of the camera image is based upon sizes of signal lights expected to be located within the camera image.

19. The system of claim 11, wherein the identifying of the subsection of the camera image eliminates other lights that are not vehicle signal lights, thereby reducing processing cycles by avoiding analysis of non-relevant portions of the camera image, wherein the proximate vehicle is in front of the autonomous vehicle or to the side of the autonomous vehicle.

20. An autonomous vehicle comprising:

at least a camera and lidar sensor that provides sensor data; and

a controller that, by a processor and based on the sensor data;

receives an image from the camera regarding a vehicle proximate to the autonomous vehicle;

uses data from the lidar sensor regarding the proximate vehicle to determine object geometry and position information for identifying a subsection within the camera image, wherein the identified subsection corresponds to an area of the proximate vehicle containing one or more vehicle signals; and

locates one or more vehicle signal lights of the proximate vehicle by using the identified camera image subsection as an area of focus;

wherein tracking of the proximate vehicle is based upon the located one or more vehicle signal lights of the proximate vehicle.