Traffic light detection

• US 9,953,229 B2
• Filed: 08/20/2013
• Issued: 04/24/2018
• Est. Priority Date: 08/20/2013
• Status: Active Grant

• Current Assignee: Harman International Industries Incorporated (Samsung Electronics Co. Ltd.)
• Original Assignee: Harman International Industries Incorporated (Samsung Electronics Co. Ltd.)
• Inventors: Hong, Chuyang, Zhang, Yankun, Jiang, Ruyi, Zhou, Yun
• Primary Examiner(s): Bali, Vikkram
• Assistant Examiner(s): Shin, Soo
• Application Number:
  US14/901,651
• Publication Number:
  US 20160148063A1
• Time in Patent Office: 1,708 Days
• Field of Search: None
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 20/582 of traffic signs
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights
• CPC Class Codes: G08G 1/09623 Systems involving the acquisition of information from passive traffic signs by means mounted on the vehicle G08G1/0967 takes precedence

Abstract

A method and a system for traffic light detection are provided. The method may include: obtaining a color image; calculating pixel response values for pixels of the color image, respectively, where each of the pixel response values may be calculated using R, G, and B values of a corresponding pixel directly, such that pixel response values of red traffic light pixels are substantially distributed on a first side of a predetermined range and pixel response values of green traffic light pixels are substantially distributed on a second side of the predetermined range which is opposite to the first side; identifying pixels whose pixel response values are distributed on the first side or the second side as candidate pixels; identifying candidate blobs based on the candidate pixels; and verifying whether the candidate blobs are traffic lights. Efficiency and reliability may be improved.

Claims

1. A method for traffic light detection, comprising:

obtaining, with a camera, a color image;

calculating, with a processor, pixel response values for pixels of the color image, respectively, where each of the pixel response values is calculated using R, G, and B values of a corresponding pixel directly, such that pixel response values of red traffic light pixels and pixel response values of green traffic light pixels are substantially distributed outside of a predetermined range, where the pixel response values of red traffic light pixels are substantially distributed on a first side of the predetermined range and the pixel response values of green traffic light pixels are substantially distributed on a second side of the predetermined range which is opposite to the first side, each of the pixel response values being calculated based on multiplication of a first component and a second component of a corresponding pixel, the second component being calculated based on a summation of a sixth component and a seventh component which are respectively based on R and G values of the corresponding pixel, where the sixth and seventh components have a same sign;

identifying, with the processor, pixels whose pixel response values are distributed on the first side or the second side as candidate pixels;

identifying, with the processor, candidate blobs based on the candidate pixels; and

verifying, with the processor, whether the candidate blobs are traffic lights,

wherein the processor is configured to use a classifier to verify whether the candidate blobs are traffic lights, and

wherein the classifier is a neural network, a support vector machine, or a cascade detector.

2. The method according to claim 1, wherein pixel response values of pixels other than red traffic light pixels and green traffic light pixels are substantially distributed within the predetermined range.

3. The method according to claim 1, wherein the first component indicates a tendency of the pixel response value, and the second component is able to amplify the tendency if the tendency is towards the first side or the second side.

4. The method according to claim 3, wherein the first component is calculated based on a summation of a third, a fourth, and a fifth component which are respectively based on R, G, and B values of the corresponding pixel, where the third component has a first sign, and the fourth and fifth components have a second sign which is opposite to the first sign.

5. The method according to claim 4, wherein the third component is k1*R, the fourth component is −

k2*G, and the fifth component is −

k3*B, where k1, k2, and k3 are set such that the tendency is substantially towards the first side if the corresponding pixel is a red traffic light pixel and the tendency is substantially towards the second side if the corresponding pixel is a green traffic light pixel.

6. The method according to claim 1, wherein the sixth component is k4*R and the seventh component is k5*G, where k4 and k5 are set such that the pixel response value is substantially on the first side if the corresponding pixel is a red traffic light pixel, and the pixel response value is substantially on the second side if the corresponding pixel is a green traffic light pixel.

7. The method according to claim 1, wherein identifying the candidate blobs based on the candidate pixels comprises:

identifying a first set of blobs based on the candidate pixels; and

identifying the candidate blobs from the first set of blobs by determining, for each blob of the first set of blobs, whether or not the blob is a member of the candidate blobs based on pixel response values of pixels of the blob and surrounding pixels of the blob.

8. A system for traffic light detection, comprising a processing device configured to use a classifier, wherein the classifier is a neural network, a support vector machine, or a cascade detector, the system configured to:

obtain, from a camera, a color image;

calculate pixel response values for pixels of the color image, respectively, where each of the pixel response values is calculated using R, G, and B values of a corresponding pixel directly, such that pixel response values of red traffic light pixels and pixel response values of green traffic light pixels are substantially distributed outside of a predetermined range, where the pixel response values of red traffic light pixels are substantially distributed on a first side of the predetermined range and pixel response values of green traffic light pixels are substantially distributed on a second side of the predetermined range which is opposite to the first side, each of the pixel response values being calculated based on multiplication of a first component and a second component of a corresponding pixel, the second component being calculated based on a summation of a sixth component and a seventh component which are respectively based on R and G values of the corresponding pixel, where the sixth and seventh components have a same sign;

identify pixels whose pixel response values are distributed on the first side or the second side as candidate pixels;

identify candidate blobs based on the candidate pixels; and

verify, with the classifier, whether the candidate blobs are traffic lights.

9. The system according to claim 8, wherein pixel response values of pixels other than red traffic light pixels and green traffic light pixels are substantially distributed within the predetermined range.

10. The system according to claim 8, wherein the first component indicates a tendency of the pixel response value, and the second component is able to amplify the tendency if the tendency is towards the first side or the second side.

11. The system according to claim 10, wherein the first component is calculated based on a summation of a third, a fourth, and a fifth component which are respectively based on R, G, and B values of the corresponding pixel, where the third component has a first sign, and the fourth and fifth components have a second sign which is opposite to the first sign.

12. The system according to claim 11, wherein the third component is k1*R, the fourth component is −

k2*G, and the fifth component is −

k3*B, where k1, k2, and k3 are set such that the tendency is substantially towards the first side if the corresponding pixel is a red traffic light pixel and the tendency is substantially towards the second side if the corresponding pixel is a green traffic light pixel.

13. The system according to claim 8, wherein the sixth component is k4*R and the seventh component is k5*G, where k4 and k5 are set such that the pixel response value is substantially on the first side if the corresponding pixel is a red traffic light pixel, and the pixel response value is substantially on the second side if the corresponding pixel is a green traffic light pixel.

14. The system according to claim 8, wherein the processing device is configured to identify the candidate blobs by:

identifying a first set of blobs based on the candidate pixels; and

identifying the candidate blobs from the first set of blobs by determining, for each blob of the first set of blobs, whether or not the blob is a member of the candidate blobs based on pixel response values of pixels of the blob and surrounding pixels of the blob.

15. A system for traffic light detection, comprising:

a processor configured to use a classifier, wherein the classifier is a neural network, a support vector machine, or a cascade detector; and

a non-transitory computer readable medium storing instructions executable by the processor to;

calculate pixel response values for pixels of a color image, respectively, where each of the pixel response values is calculated using R, G, and B values of a corresponding pixel directly, such that pixel response values of red traffic light pixels and pixel response values of green traffic light pixels are substantially distributed outside of a predetermined range, where the pixel response values of red traffic light pixels are substantially distributed on a first side of the predetermined range and pixel response values of green traffic light pixels are substantially distributed on a second side of the predetermined range which is opposite to the first side, each of the pixel response values being calculated based on multiplication of a first component and a second component of a corresponding pixel, the second component being calculated based on a summation of a sixth component and a seventh component which are respectively based on R and G values of the corresponding pixel, where the sixth and seventh components have a same sign;

identify pixels whose pixel response values are distributed on the first side or the second side as candidate pixels;

identify candidate blobs based on the candidate pixels; and

verify, using the classifier, whether the candidate blobs are traffic lights.

16. The system according to claim 15, wherein the color image is captured by a camera, the camera mounted on a vehicle.

17. The system according to claim 15, wherein verifying whether the candidate blobs are traffic lights includes identifying a set of blobs from the candidate blobs based on one or more geometric parameters of each blob of the set of blobs, the geometric parameters including a pixel number based on an image resolution, a height, a width, and a width to height ratio.

18. The system according to claim 17, wherein verifying whether the candidate blobs are traffic lights further comprises calculating a probability of each blob of the set of blobs based on a first pixel response value for pixels within an ego segment and a second pixel response value for pixels within an ambient segment, the ego segment enclosing each blob of the set of blobs and the ambient segment partially overlapping with the ego segment.