Brake light detection

• US 10,248,874 B2
• Filed: 11/22/2016
• Issued: 04/02/2019
• Est. Priority Date: 11/22/2016
• Status: Active Grant

• Current Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Original Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Inventors: Moosaei, Maryam, Hotson, Guy, Mahmoudieh, Parsa, Nariyambut Murali, Vidya
• Primary Examiner(s): Misleh, Justin P.
• Application Number:
  US15/359,158
• Publication Number:
  US 20180144202A1
• Time in Patent Office: 861 Days
• Field of Search: 382156
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30248 Vehicle exterior or interior
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights

Abstract

Systems, methods, and devices for detecting brake lights are disclosed herein. A system includes a mode component, a vehicle region component, and a classification component. The mode component is configured to select a night mode or day mode based on a pixel brightness in an image frame. The vehicle region component is configured to detect a region corresponding to a vehicle based on data from a range sensor when in a night mode or based on camera image data when in the day mode. The classification component is configured to classify a brake light of the vehicle as on or off based on image data in the region corresponding to the vehicle.

Claims

1. A method comprising:

selecting a night mode or a day mode based on a pixel brightness in an image frame;

detecting an image region corresponding to a vehicle based on range sensor data when in the night mode or based on camera image data when in the day mode;

converting the range sensor data into luminosity and two-color channel (LAB) color space when in the night mode;

converting the camera image data into hue, saturation, and value (HSV) color space when in the day mode;

determining at least one region of interest within the image region corresponding to a brake light of the vehicle based on the converted range sensor data or the converted camera image data;

classifying the brake light of the vehicle as on or off based on the at least one region of interest to generate a brake light classification; and

executing a driving maneuver based on the brake light classification.

2. The method of claim 1, further comprising determining one or more bounding boxes corresponding to brake lights of the vehicle in the image region corresponding to the vehicle.

3. The method of claim 2, wherein determining the one or more bounding boxes comprises:

extracting contours in the image region corresponding to the vehicle; and

selecting contours having a shape or size corresponding to a brake light shape or size.

4. The method of claim 2, wherein classifying the brake light comprises classifying based on image data corresponding to the one or more bounding boxes.

5. The method of claim 2, wherein determining the one or more bounding boxes corresponding to brake lights comprises determining based on the HSV color space when in the day mode and based on the LAB color space when in the night mode.

6. The method of claim 1, wherein classifying the brake light of the vehicle as on or off comprises classifying using a neural network.

7. The method of claim 1, wherein detecting the image region corresponding to the vehicle based on the range sensor data when in the night mode comprises:

identifying a cluster of points in the range sensor data corresponding to the vehicle; and

mapping a bounding box of the cluster of points to a corresponding image frame.

8. The method of claim 1, further comprising:

generating filtered frames by filtering at least one of the LAB color space or the HSV color space with different sizes based on different lighting conditions;

wherein classifying the brake light of the vehicle as on or off comprises classifying based on the filtered frames.

9. A system comprising a processor that is programmable to execute instructions stored in non-transitory computer readable storage media, the instructions comprising:

selecting a night mode or a day mode based on a pixel brightness in an image frame;

detecting an image region corresponding to a vehicle based on range sensor data when in the night mode or based on camera image data when in the day mode;

converting the range sensor data into luminosity and two-color channel (LAB) color space when in the night mode;

converting the camera image data into hue, saturation, and value (HSV) color space when in the day mode;

determining at least one region of interest within the image region corresponding to a brake light of the vehicle based on the converted range sensor data or the converted camera image data;

classifying the brake light of the vehicle as on or off based on the at least one region of interest to generate a brake light classification; and

executing a driving maneuver based on the brake light classification.

10. The system of claim 9, wherein the instructions further comprise determining one or more bounding boxes corresponding to brake lights of the vehicle in the image region corresponding to the vehicle.

11. The system of claim 10, wherein determining the one or more bounding boxes comprises:

extracting contours in the region corresponding to the vehicle; and

selecting contours having a shape or size corresponding to a brake light shape or size.

12. The system of claim 10, wherein classifying the brake light as on or off comprises classifying based on image data corresponding to the one or more bounding boxes.

13. The system of claim 10, wherein determining the one or more bounding boxes corresponding to brake lights comprises determining based on the HSV color space when in the day mode and based on the LAB color space when in the night mode.

14. The system of claim 9, wherein classifying the brake light of the vehicle as on or off comprises classifying using a neural network.

15. The system of claim 9, wherein detecting the image region corresponding to the vehicle based on the range sensor data when in the night mode comprises:

identifying a cluster of points in the range sensor data corresponding to the vehicle; and

mapping a bounding box of the cluster of points to a corresponding image frame.

16. Non-transitory computer readable storage media storing instructions that, when executed by one or more processors, cause the one or more processors to:

select a night mode or a day mode based on a pixel brightness in an image frame;

detect an image region corresponding to a vehicle based on range sensor data when in the night mode or based on camera image data when in the day mode;

convert the range sensor data into luminosity and two-color channel (LAB) color space when in the night mode;

convert the camera image data into hue, saturation, and value (HSV) color space when in the day mode;

determine at least one region of interest within the image region corresponding to a brake light of the vehicle based on the converted range sensor data or the converted camera image data;

classify the brake light of the vehicle as on or off based on the at least one region of interest to generate a brake light classification; and

execute a driving maneuver based on the brake light classification.

17. The non-transitory computer readable storage media of claim 16, further comprising instructions that cause the one or more processors to determine one or more bounding boxes corresponding to brake lights of the vehicle in the image region corresponding to the vehicle.

18. The non-transitory computer readable storage media of claim 17, wherein the instructions cause the one or more processors to determine the one or more bounding boxes by:

extracting contours in the image region corresponding to the vehicle; and

selecting contours having a shape or size corresponding to a brake light shape or size;

wherein the instructions cause the one or more processors to classify the brake light based on image data corresponding to the one or more bounding boxes.

19. The non-transitory computer readable storage media of claim 16, wherein the instructions cause the one or more processors to classify the brake light of the vehicle as on or off using a neural network.

20. The non-transitory computer readable storage media of claim 16, wherein the instructions cause the one or more processors to detect the image region corresponding to the vehicle based on the range sensor data when in the night mode by:

identifying a cluster of points in the range sensor data corresponding to the vehicle; and

mapping a bounding box of the cluster of points to a corresponding image frame.