Object recognition in low-lux and high-lux conditions

• US 8,781,171 B2
• Filed: 10/24/2012
• Issued: 07/15/2014
• Est. Priority Date: 10/24/2012
• Status: Active Grant

• Current Assignee: Edge Technologies, Inc, Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Edge 3 Technologies, LLC, Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: King, Josh Tyler, El Dokor, Tarek, Vaghefinazari, Pedram, Yamamoto, Stuart Masakazu, Huang, Ritchie Winson
• Primary Examiner(s): Liew, Alex
• Application Number:
  US13/659,826
• Publication Number:
  US 20140112528A1
• Time in Patent Office: 629 Days
• Field of Search: 382/100, 382/103
• US Class Current: 382/104
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/106 using night vision cameras
• CPC Class Codes: B60R 2300/8006 for monitoring and displaying scenes of vehicle interior, e.g. for monitoring passengers or cargo
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 20/59 inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/71 Circuitry for evaluating the brightness variation
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

A system for capturing image data for gestures from a passenger or a driver in a vehicle with a dynamic illumination level comprises a low-lux sensor equipped to capture image data in an environment with an illumination level below an illumination threshold, a high-lux sensor equipped to capture image data in the environment with the illumination level above the illumination threshold, and an object recognition module for activating the sensors. The object recognition module determines the illumination level of the environment and activates the low-lux sensor if the illumination level is below the illumination threshold. If the illumination level is above the threshold, the object recognition module activates the high-lux sensor.

Claims

1. A computer-implemented method for capturing image data for gestures from a passenger or a driver in a vehicle with a dynamic illumination level, the method comprising:

determining an illumination level for the vehicle, the vehicle comprising a low-lux sensor and a high-lux sensor located in an overhead console of the vehicle, the low-lux sensor assigned to capture image data indicating gestures from the driver or the passenger in the vehicle with illumination levels below a threshold illumination level, the high-lux sensor assigned to capture image data indicating gestures from the driver or the passenger in the vehicle with illumination levels above a threshold illumination level;

determining, by an illumination threshold module, whether the determined illumination level is below a threshold illumination level;

processing image data captured by the low-lux sensor when the illumination level is below the threshold illumination level; and

processing image data captured by the high-lux sensor when the illumination level is above the threshold illumination level.

2. The computer-implemented method of claim 1, further comprising:

determining that the illumination level is below the threshold illumination level;

processing image data captured by the low-lux sensor;

determining that the illumination level is above the threshold illumination level; and

processing image data captured by the high-lux sensor.

3. The computer-implemented method of claim 1, further comprising activating an illumination source responsive to determining that the illumination level is below the threshold illumination level.

4. The computer-implemented method of claim 1, wherein the low-lux sensor and the high-lux sensor have different luminance ranges, wherein luminance range of a sensor is a range of scene luminance under which the sensor is equipped to capture image data.

5. The computer-implemented method of claim 4, wherein the luminance ranges of the low-lux sensor and the high-lux sensor overlap, and the threshold illumination level is within the overlap of the luminance ranges.

6. The computer-implemented method of claim 1, wherein the high-lux sensor comprises an infrared filter to reduce an amount of infrared light reaching the high-lux sensor.

7. The computer-implemented method of claim 3, wherein the illumination source emits a light with a spectrum that lies within a range at a peak of a response curve of the low-lux sensor.

8. The computer-implemented method of claim 1, further comprising:

responsive to determining that the illumination level is below the threshold illumination level, determining whether the high-lux sensor is active; and

responsive to determining that the high-lux sensor is active, deactivating the active high-lux sensor.

9. The computer-implemented method of claim 1, wherein the vehicle comprises four sensors for capturing image data indicating gestures from the driver or the passenger, the four sensors located in the overhead console of the vehicle, wherein the four sensors include two RGB sensors and two infrared sensors.

10. A computer program product for capturing image data for gestures from a passenger or a driver in a vehicle with a dynamic illumination level, the computer program product comprising a non-transitory computer-readable storage medium including computer program code for:

determining an illumination level for the vehicle, the vehicle comprising a low-lux sensor and a high-lux sensor located in an overhead console of the vehicle, the low-lux sensor assigned to capture image data indicating gestures from the driver or the passenger in the vehicle with illumination levels below a threshold illumination level, the high-lux sensor assigned to capture image data indicating gestures from the driver or the passenger in the vehicle with illumination levels above a threshold illumination level;

determining, by an illumination threshold module, whether the determined illumination level is below a threshold illumination level;

processing image data captured by the low-lux sensor when the illumination level is below the threshold illumination level; and

processing image data captured by the high-lux sensor when the illumination level is above the threshold illumination level.

11. The computer program product of claim 10, further comprising computer program code for:

determining that the illumination level is below the threshold illumination level;

processing image data captured by the low-lux sensor;

determining that the illumination level is above the threshold illumination level; and

processing image data captured by the high-lux sensor.

12. The computer program product of claim 10, further comprising computer program code for activating an illumination source responsive to determining that the illumination level is below the threshold illumination level.

13. The computer program product of claim 10, wherein the low-lux sensor and the high-lux sensor have different luminance ranges, wherein luminance range of a sensor is a range of scene luminance under which the sensor is equipped to capture image data.

14. The computer program product of claim 13, wherein the luminance ranges of the low-lux sensor and the high-lux sensor overlap, and the threshold illumination level is within the overlap of the luminance ranges.

15. The computer program product of claim 10, wherein the high-lux sensor comprises an infrared filter to reduce an amount of infrared light reaching the high-lux sensor.

16. The computer program product of claim 12, wherein the illumination source emits a light with a spectrum that lies within a range at a peak of a response curve of the low-lux sensor.

17. The computer program product of claim 10, further comprising computer program code for:

responsive to determining that the illumination level is below the threshold illumination level, determining whether the high-lux sensor is active; and

responsive to determining that the high-lux sensor is active, deactivating the active high-lux sensor.

18. The computer program product of claim 10, wherein the vehicle comprises four sensors for capturing image data indicating gestures from the driver or the passenger, the four sensors located in the overhead console of the vehicle, wherein the four sensors include two RGB sensors and two infrared sensors.

19. A computer system for capturing image data for gestures from a passenger or a driver in a vehicle with a dynamic illumination level, the computer system comprising a processor and a non-transitory computer readable medium, the computer readable medium including computer program code for:

determining an illumination level for the vehicle, the vehicle comprising a low-lux sensor and a high-lux sensor located in an overhead console of the vehicle, the low-lux sensor assigned to capture image data indicating gestures from the driver or the passenger in the vehicle with illumination levels below a threshold illumination level, the high-lux sensor assigned to capture image data indicating gestures from the driver or the passenger in the vehicle with illumination levels above a threshold illumination level;

determining, by an illumination threshold module, whether the determined illumination level is below a threshold illumination level;

processing image data captured by the low-lux sensor when the illumination level is below the threshold illumination level; and

processing image data captured by the high-lux sensor when the illumination level is above the threshold illumination level.

20. The computer system of claim 19, further comprising computer program code for:

determining that the illumination level is below the threshold illumination level;

processing image data captured by the low-lux sensor;

determining that the illumination level is above the threshold illumination level; and

processing image data captured by the high-lux sensor.

21. The computer system of claim 19, further comprising computer program code for activating an illumination source responsive to determining that the illumination level is below the threshold illumination level.

22. The computer system of claim 19, wherein the low-lux sensor and the high-lux sensor have different luminance ranges, wherein luminance range of a sensor is a range of scene luminance under which the sensor is equipped to capture image data.

23. The computer system of claim 22, wherein the luminance ranges of the low-lux sensor and the high-lux sensor overlap, and the threshold illumination level is within the overlap of the luminance ranges.

24. The computer system of claim 19, wherein the high-lux sensor comprises an infrared filter to reduce an amount of infrared light reaching the high-lux sensor.

25. The computer system of claim 21, wherein the illumination source emits a light with a spectrum that lies within a range at a peak of a response curve of the low-lux sensor.

26. The computer system of claim 19, wherein the vehicle comprises four sensors for capturing image data indicating gestures from the driver or the passenger, the four sensors located in the overhead console of the vehicle, wherein the four sensors include two RGB sensors and two infrared sensors.

27. A computer system for capturing image data for gestures from a passenger or a driver in a vehicle with a dynamic illumination level, the computer system comprising:

an illumination level module for determining an illumination level for the vehicle, the vehicle comprising a low-lux sensor and a high-lux sensor located in an overhead console of the vehicle, the low-lux sensor assigned to capture image data indicating gestures from the driver or the passenger in the vehicle with illumination levels below a threshold illumination level, the high-lux sensor assigned to capture image data indicating gestures from the driver or the passenger in the vehicle with illumination levels above a threshold illumination level;

an illumination threshold module for determining whether the determined illumination level is below a threshold illumination level; and

a capture source module for

activating the low-lux sensor when the illumination level is below the threshold illumination level, and

activating the high-lux sensor when the illumination level is above the threshold illumination level.

28. The computer system of claim 27, wherein the low-lux sensor and the high-lux sensor have different luminance ranges, wherein luminance range of a sensor is a range of scene luminance under which the sensor is equipped to capture image data.

29. The computer system of claim 28, wherein the luminance ranges of the low-lux sensor and the high-lux sensor overlap, and the threshold illumination level is within the overlap of the luminance ranges.

30. The computer system of claim 27, wherein the vehicle comprises four sensors for capturing image data indicating gestures from the driver or the passenger, the four sensors located in the overhead console of the vehicle, wherein the four sensors include two RGB sensors and two infrared sensors.