Method and device for recognising pulsing light sources

• US 9,264,626 B2
• Filed: 02/15/2013
• Issued: 02/16/2016
• Est. Priority Date: 07/19/2010
• Status: Active Grant

• Current Assignee: CARIAD SE (Volkswagen AG)
• Original Assignee: Hella KGaA Hueck & Company
• Inventors: Hoffmann, Ingo, Stefanovic, Marko, Siecke, Jona, Debski, Andreas, Lorenz, Tara, Mohr, Marcel
• Primary Examiner(s): Bella, Matthew
• Assistant Examiner(s): Heidemann, Jason
• Application Number:
  US13/768,443
• Publication Number:
  US 20130229521A1
• Time in Patent Office: 1,096 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06V 10/12 Details of acquisition arrangements; Constructional details thereof
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights
• CPC Class Codes: H04N 23/71 Circuitry for evaluating the brightness variation

Abstract

The invention relates to a method and a device for recognizing a pulsing light source, in which a sequence of individual images is acquired with a light sensitive sensor. In an evaluation unit, all or at least some of the picture points of each individual image are examined with regard to at least one predefined feature. A gray value of any picture point depends on two or more charge levels of at least one of the photodetectors of the sensor corresponding to the picture point at the ends of at least two integration intervals for the picture point. The at least two integration intervals for said picture point have different lengths. The examination comprises an examination of the picture points as to whether the picture points have gray values in a defined vicinity of one of at least one predefined gray value.

Claims

1. A method for recognising a pulsing light source (7), in which a sequence of individual images is acquired with a camera (2), and gray values of picture points of the individual images are measured with a light sensitive sensor (5) of the camera (2), wherein the sensor (5) comprises an array of photodetectors, wherein further, in an evaluation unit (6), all or at least some of the picture points of each individual image, on account of their gray values, are examined with regard to at least one predefined feature, wherein a given image point is classified as a light source or as a part of a light source depending on whether it displays the at least one feature,

wherein for each of the picture points of each of the individual images the gray value of the picture point depends on two or more charge levels (c_i, i=1, 2, . . . ) of at least one of the photodetectors corresponding to the picture point at the ends of at least two integration intervals (i=1, 2 . . . ) for the picture point, wherein the at least two integration intervals for said picture point have different lengths,

wherein the charge level of any of the photodetectors at the end of any of the integration intervals is indicative of the amount of illumination incident on the photodetector during the integration interval, a zero value (c_i⁰) of the charge level at the end of any of the integration intervals indicating no illumination during the integration interval and a saturation value (c_i^sat) of the charge level at the end of any of the integration intervals indicating saturation of the photodetector during the integration interval, characterised in that

the examination with regard to the at least one predefined features comprises an examination of the picture points as to whether the picture points have gray values in a defined vicinity of one of at least one predefined gray value, the at least one predefined gray value being defined as the gray value that any of the picture points attains if the charge levels of the at least one photodetector corresponding to the picture point at the ends of the at least two integration intervals (i=1, 2 . . . ) for the picture point have either the corresponding zero value or the corresponding saturation value.

2. A method according to claim 1, characterised in that a sensitivity of the photodetectors is such that if any one of the photodetectors is illuminated with one of the light pulses during one of the integration intervals, the photodetector saturates during the illumination with the light pulse and the charge level of the photodetector at the end of the integration interval has the corresponding saturation value.

3. A method according to claim 1, characterised in that a sensitivity of the photodetectors is such that the charge level of each photodetector of the sensor located in an area of the sensor associated with the pulsing light source does not change during any of said integration intervals of the photodetector if said photodetector is not illuminated by any of the light pulses during the integration interval and the charge level of the photodetector at the end of the integration interval has the corresponding zero value.

4. A method according to claim 1, characterised in that an image rate of the sensor is less than a pulse rate of the light pulses and/or that the lengths of said at least two integration intervals for any of the image points is greater than a pulse duration of the light pulses.

5. A method according to claim 1, characterised in that for each picture point of each of the individual images the charge levels are read out from the one or more photodetectors corresponding to the picture point at the ends of each of the integration intervals for said picture point.

6. A method according to claim 1, characterised in that for each picture point of each of the individual images the at least two integration intervals for the picture point follow each other, wherein the charge level of the at least one photodetector corresponding to the picture point is reset to one of at least one predefined adjustment value of the sensor at the end of one of the integration intervals for said picture point if the charge level of the at least one photodetector has passed the adjustment value at the end of said integration interval such that said adjustment value is the charge level of said photodetector at the beginning of the following integration interval for said picture point of said individual image, wherein the charge level of the at least one photodetector corresponding to the picture point is read out from the at least one photodetector corresponding to the picture point at the end of the last integration interval for said picture point of said individual image.

7. A method according to claim 6, characterised in that the at least one predefined gray value is defined as the at least one predefined adjustment value of the sensor or that the at least one predefined gray value includes the at least one predefined adjustment value of the sensor.

8. A method according to claim 1, characterised in that for each of the picture points of each of the individual images the gray value of the picture point corresponds to the charge level of the at least one photodetector corresponding to the picture point at the end of the longest of the at least two integration intervals for that picture point at the end of which the at least one photodetector is not saturated.

9. A method according to claim 1, wherein the picture points are, by way of their gray values, combined into image regions.

10. A method according to claim 9, characterised in that the given image region is classified as a pulsing light source or as part of a pulsing light source depending on whether a relative share of picture points of the given image region, whose gray values correspond exactly to one of the at least one predefined gray value, exceeds a predefined threshold value.

11. A method according to claim 9, characterised in that the given image region is classified as a pulsing light source or as part of a pulsing light source depending on whether a variance of the gray values of the picture points of the given image region falls short of a threshold value.

12. A method according to claim 9, characterised in that the given image region is classified as a pulsing light source or as part of a pulsing light source depending on whether a ratio which is formed by dividing a variance of the gray values of the picture points of the given image region by a variance of the gray values of all picture points of the given image region, falls short of a threshold value.

13. A method according to claim 12, characterised in that the given image region is classified as a pulsing light source or as part of a pulsing light sources, depending on whether in the individual images which are subsequent to the individual image, an average value of the gray values of the picture points of the given image region changes by more than a predefined threshold value.

14. A method according to claim 13, characterised in that the periodicity is determined by way of the Hamming distance between a first vector and a second vector, wherein in an nth entry of the first vector, it is specified whether the gray values of the given image region in an individual image preceding the given individual image in the sequence of individual images at the nth location, lie within the defined vicinity of one of the at least one predefined gray value or outside the defined vicinity of one of the at least one predefined gray value, and the second vector is formed from the first vector by way of a bitwise shifting of the first vector by p bits, wherein the computation of the Hamming distance is repeated for different values of p, and that value of p, at which the Hamming distance is minimal, is determined as a period of the image region, and the periodicity of the image region is computed from the Hamming distance belonging to this period.

15. A method according to claim 12, characterised in that the given picture point or image region is classified as a pulsing light source or as part of a pulsing light source, depending on whether in the subsequent individual images, the gray values of the picture point or picture points of the given image region, lie in the defined vicinity of one of the at least one predefined gray value in a manner recurring with a periodicity, wherein the periodicity lies above a threshold value.

16. A method according to claim 1, characterised in that the given picture point or image region is classified as a pulsing light source or as part of a pulsing light source, depending on whether in individual images which in the sequence are subsequent to the given individual image, the given picture point or image region is re-recognised.

17. A device for recognising a pulsing light source (7), comprising a camera (2) for acquiring a sequence of individual images, wherein the camera (2) comprises a light-sensitive sensor (5) with an array of photodetectors for measuring gray values of picture points of the individual images, and an evaluation unit (6) which is configured to examine all or at least some of the picture points of each individual image, on account of their gray values, are examined with regard to at least one predefined feature, wherein a given image point is classified as a light source or as a part of a light source depending on whether it displays the at least one feature,

wherein for each of the picture points of each of the individual images the gray value of the picture point depends on two or more charge levels (c_i, i=1, 2, . . . ) of at least one of the photodetectors corresponding to the picture point at the ends of at least two integration intervals (i=1, 2 . . . ) for the picture point, wherein the at least two integration intervals for said picture point have different lengths,

wherein the charge level of any of the photodetectors at the end of any of the integration intervals is indicative of the amount of illumination incident on the photodetector during the integration interval, a zero value (c_i⁰) of the charge level at the at the end of any of the integration intervals indicating no illumination during the integration interval and a saturation value (c_i^sat) of the charge level at the end of any of the integration intervals indicating saturation of the photodetector during the integration interval, characterised in that

at least one predefined gray value is defined as the gray value that any of the picture points attains if the charge levels of the at least one photodetector corresponding to this picture point at the ends of the at least two integration intervals (i=1, 2 . . . ) for the picture point have either the corresponding zero value or the corresponding saturation value,

wherein the examination with regard to the at least one predefined feature comprises an examination of the image regions as to whether the picture points of these image regions comprise gray values in a defined vicinity of one of the at least one predefined gray value of the sensor.

18. A device according to claim 17, characterised in that a sensitivity of the photodetectors is such that if any one of the photodetectors is illuminated with one of the light pulses during one of the integration intervals, the photodetector saturates during the illumination with the light pulse and the charge level of the photodetector at the end of the integration interval has the corresponding saturation value (c_i=c_i^sat).

19. A device according to claim 17, characterised in that a sensitivity of the photodetectors is such that the charge level of each photodetector of the sensor located in an area of the sensor associated with the pulsing light source does not change during any of said integration intervals of the photodetector if said photodetector is not illuminated by any of the light pulses during the integration interval and the charge level of the photodetector at the end of the integration interval has the corresponding zero value (c_i=c_i⁰).

20. A device according to claim 17, characterised in that an image rate of the sensor is less than a pulse rate of the light pulses and/or that the duration of said at least two integration intervals for any of the image points is greater than a pulse duration of the light pulses.

21. A device according to claim 17, characterised in that the sensor comprises a read-out circuitry configured to read out from any of the photodetectors the charge levels at the ends of any of the integration intervals.

22. A device according to claim 17, characterised in that the sensor is configured such that for each picture point of each of the individual images the at least two integration intervals for the picture point follow each other, wherein the sensor is configured to reset the charge level of the at least one photodetector corresponding to the picture point to one of at least one predefined adjustment value of the sensor at the end of one of the integration intervals for said picture point if the charge level of the at least one photodetector has passed the adjustment value at the end of said integration interval such that said adjustment value is the charge level of said photodetector at the beginning of the following integration interval for said picture point of said individual image, wherein the sensor further comprises a read-out circuitry configured to read the charge level of the at least one photodetector corresponding to said picture point out from said at least one photodetector at the end of the last integration interval for said picture point.

23. A device according to claim 22, characterised in that the at least one predefined gray value is defined as the at least one predefined adjustment value of the sensor or that the at least one predefined gray value includes the at least one predefined adjustment value of the sensor.

24. A device according to claim 17, characterised in that for each of the picture points of each of the individual images the gray value of the picture point corresponds to the charge level of the at least one photodetector corresponding to said picture point at the end of the longest of the at least two integration intervals for that picture point at the end of which the at least one photodetector is not saturated.

25. A vehicle (10) with a driver assistance system which comprises a device according to claim 17 and which is configured to recognise headlights (7) and/or rear lights (7) of other vehicles (4), in particular pulsing light sources, such as LED headlights and/or LED rear lights, wherein the camera (2) of the device is preferably aligned in the direction of a front side (3) of the vehicle (1).