Systems and methods for spatially controlled scene illumination

• US 8,890,946 B2
• Filed: 03/01/2010
• Issued: 11/18/2014
• Est. Priority Date: 03/01/2010
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Eyefluence, Inc. (Alphabet Inc.)
• Inventors: Publicover, Nelson G., Heffernan, Jason
• Primary Examiner(s): SHAW, PELING ANDREW
• Application Number:
  US12/715,177
• Publication Number:
  US 20110211056A1
• Time in Patent Office: 1,723 Days
• US Class Current: 348/78
• CPC Class Codes: G02C 11/10 Electronic devices other than hearing aids
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/71 Circuitry for evaluating the brightness variation
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

A scene illumination system is provided that produces spatially uniform or controlled brightness levels for machine vision applications. The system includes a camera, multiple light sources that preferentially illuminate different regions within the camera'"'"'s field-of-view, and a processing unit coupled to the camera and light sources. Focal regions of the light sources within the camera'"'"'s field-of-view are sampled to determine average regional brightness and compared to target brightness levels. The processing unit controls the light sources to increase or decrease illumination levels to converge toward the target brightness levels within the field-of-view. This modulation of the light sources may be repeated with successive video images until target brightness levels are achieved. Once achieved, the iterative feedback control may be locked-in for some applications, while for others, the iterative process may continue periodically or continuously to account for different scenes or changes in lighting conditions.

Claims

1. A system for monitoring an eye of a person, comprising:

a device configured to be worn on a person'"'"'s head;

a camera mounted on the device and positioned for viewing a first eye of the person wearing the device;

a plurality of light sources on the device that simultaneously and preferentially illuminate respective separate focal regions of the person'"'"'s face around the first eye and within the camera'"'"'s field-of-view, the focal regions only partially overlapping one another; and

a controller coupled to the camera and the light sources, the controller configured for sampling brightness in the respective focal regions of the light sources using the camera and modulating the light sources based at least in part on the sampled brightness to increase or decrease intensity of light from the light sources to provide desired brightness levels within the respective focal regions.

2. The system of claim 1, wherein the controller is mounted on the device.

3. The system of claim 1, wherein the controller is located remote from the device.

4. The system of claim 1, wherein the controller is configured for sampling brightness from multiple pixels that are a first subset less than all of the pixels of the camera that correspond to a first focal region of a first light source within the camera'"'"'s field-of-view that is illuminated by the first light source, the controller combining the sampled brightness to determine an average brightness provided by the first light source, the controller modulating the first light source to provide a desired brightness level within the first focal region.

5. The system of claim 4, wherein the controller is configured for amplitude modulation of at least one of the current and the voltage to the first light source to provide the desired brightness level.

6. The system of claim 4, wherein the controller is configured for pulse-width modulation of at least one of the current and the voltage to the first light source to provide the desired brightness level.

7. The system of claim 1, wherein the controller is configured for activating the light sources only during periods when the camera is activated to acquire images of the first eye.

8. A feedback controlled system for producing spatially controlled illumination comprising:

a camera that measures scene brightness in two or more spatial regions of a field-of-view;

light sources that simultaneously and preferentially illuminate corresponding separate or overlapping focal regions within the camera'"'"'s field-of-view, the focal regions having different centers than one another; and

a processor that computes average brightness within each region of the camera'"'"'s field-of-view and modulates corresponding light sources to one or more target brightness levels to provide desired brightness levels within the field-of-view.

9. The system of claim 8, wherein the camera and the light sources are mounted on a device configured to be worn on a person'"'"'s head, the camera positioned on the device for viewing a first eye of the person wearing the device, the light sources located on the device for preferentially illuminating respective focal regions of the person'"'"'s face around the first eye and within the camera'"'"'s field-of-view.

10. The system of claim 9, wherein the processor is located remote from the device.

11. The system of claim 8, wherein the processor is configured for sampling brightness from multiple pixels that are a first subset less than all of the pixels of the camera that correspond to a first focal region of a first light source within the camera'"'"'s field-of-view that is illuminated by the first light source, the processor combining the sampled brightness to determine an average brightness provided by the first light source, the processor modulating the first light source to provide a desired brightness level within the first focal region.

12. The system of claim 8, wherein the processor is configured for amplitude modulation of at least one of the current and the voltage to the light sources to provide the desired brightness levels.

13. The system of claim 8, wherein the processor is configured for pulse-width modulation of at least one of the current and the voltage to the light sources to provide the desired brightness levels.

14. The system of claim 8, wherein the processor is coupled to the light sources for activating the light sources only during periods when the camera is activated to acquire images of the first eye.

15. The system of claim 14, wherein the processor is configured for pulse-width modulation of at least one of the current and the voltage to the light sources during the periods when the camera is activated to provide the desired brightness levels.

16. The system of claim 8, wherein the processor is coupled to the light sources for deactivating the light sources when the camera is inactive.

17. The system of claim 8, further comprising an electronic display, the camera and light sources mounted relative to the display such that the camera and light sources are oriented towards a face of a person viewing the electronic display.

18. The system of claim 17, wherein the camera and light sources are mounted to the display.

19. The system of claim 8, wherein the camera is mounted to a dashboard of a vehicle such that the camera is oriented towards the face of an operator of the vehicle.

20. The system of claim 19, wherein the light sources are mounted to the vehicle such that the light sources are oriented towards the face of the operator for preferentially illuminating respective focal regions of the operator'"'"'s face within the camera'"'"'s field-of-view.

21. The system of claim 8, wherein the camera is mounted to a structure adjacent an assembly line or conveyor belt for obtaining images of objects being directed along the assembly line or conveyor belt.

22. The system of claim 8, wherein the camera is mounted to a structure adjacent a plurality of storage areas for obtaining images of objects located in the storage areas.

23. The system of claim 22, wherein the light sources are mounted adjacent the storage areas to preferentially illuminate respective focal regions of the storage areas within the camera'"'"'s field-of-view.

24. A method for controlling illumination of a first eye of a person, comprising:

positioning a camera towards a person'"'"'s face such that a first eye of the person lies within a field-of-view of the camera;

illuminating at least a portion of the person'"'"'s face with a plurality of light sources that simultaneously and preferentially illuminate respective separate focal regions of the person'"'"'s face around the first eye and within the camera'"'"'s field-of-view, the focal regions only partially overlapping one another;

sampling brightness in the respective focal regions of the light sources using the camera; and

modulating the light sources based at least in part on the sampled brightness to provide desired brightness levels within the respective focal regions.

25. The method of claim 24, wherein the camera is mounted on a device configured to be worn on a person'"'"'s head, and wherein the camera is positioned towards the person'"'"'s face by placing the device on the person'"'"'s head.

26. The method of claim 24, wherein brightness in the respective focal regions is sampled using a controller coupled to the camera, the controller coupled to the light sources for modulating the light sources based at least in part on the sampled brightness.

27. The method of claim 26, wherein the camera is mounted on a device configured to be worn on a person'"'"'s head, and wherein the controller is located remote from the device.

28. The method of claim 24, wherein sampling brightness comprises:

sampling brightness from multiple pixels that are a subset less than all of the pixels of the camera that correspond to a first focal region of a first light source within the camera'"'"'s field-of-view that is illuminated by the first light source; and

determining an average brightness provided by the first light source, and

wherein modulating the light sources comprises modulating the first light source to provide a desired brightness level within the first focal region based at least in part on the determined average brightness.

29. The method of claim 24, wherein modulating the light sources comprises modifying amplitudes of at least one of the current and the voltage to the light sources to provide the desired brightness levels within the respective focal regions.

30. The method of claim 24, wherein modulating the light sources comprises modifying pulse widths of at least one of the current and the voltage to the light sources to provide the desired brightness levels within the respective focal regions.

31. A method for controlling illumination of a scene, comprising:

directing a camera towards a scene such that one or more objects in the scene are within a field-of-view of the camera;

illuminating the scene with a plurality of light sources that simultaneously and preferentially illuminate respective separate or overlapping focal regions within the camera'"'"'s field-of-view;

sampling brightness in the respective focal regions of the light sources using the camera; and

modulating the light sources based at least in part on the sampled brightness to provide desired brightness levels within the respective focal regions.

32. The method of claim 31, wherein the scene includes at least a portion of a person'"'"'s face, and wherein the one or more objects comprise at least one eye of the person.

33. The method of claim 31, wherein the camera is mounted on a device on the person'"'"'s head.

34. The method of claim 31, wherein the plurality of light sources are mounted on a device on the person'"'"'s head.

35. The method of claim 31, wherein the camera is mounted at a location remotely from the person'"'"'s head.

36. The method of claim 31, wherein the plurality of light sources are mounted remotely from the person'"'"'s head.

37. The method of claim 31, wherein brightness in the respective focal regions is sampled using a controller coupled to the camera, the controller coupled to the light sources for modulating the light sources based at least in part on the sampled brightness.

38. The method of claim 31, wherein sampling brightness comprises:

sampling brightness from multiple pixels that are a first subset less than all of the pixels of the camera that correspond to a first focal region of a first light source within the camera'"'"'s field-of-view that is illuminated by the first light source; and

determining an average brightness provided by the first light source from the brightness sampled from the multiple pixels, and

wherein modulating the light sources comprises modulating the first light source to provide a desired brightness level within the first focal region based at least in part on the determined average brightness.

39. The method of claim 38, wherein sampling brightness further comprises:

sampling brightness from multiple pixels that are a second subset less than all of the pixels of the camera that correspond to a second focal region of a second light source within the camera'"'"'s field-of-view that is illuminated by the second light source; and

determining an average brightness provided by the second light source, and

wherein modulating the light sources comprises modulating the second light source to provide a desired brightness level within the second focal region based at least in part on the determined average brightness.

40. The method of claim 31, wherein modulating the light sources comprises modifying amplitudes of at least one of the current and the voltage to the light sources to provide the desired brightness levels within the respective focal regions.

41. The method of claim 31, wherein modulating the light sources comprises modifying pulse widths of at least one of the current and the voltage to the light sources to provide the desired brightness levels within the respective focal regions.

42. The method of claim 31, wherein the light sources are modulated to provide a substantially uniform brightness within the camera'"'"'s field-of-view.

43. The method of claim 31, wherein the light sources are modulated to provide a predetermined brightness that varies within different portions of the camera'"'"'s field-of-view.

44. The method of claim 31, wherein the camera and light sources are mounted relative to an electronic display such that the camera and light sources are oriented towards a face of a person viewing the electronic display, and wherein the light sources are modulated to provide desired brightness levels to respective focal regions of the person'"'"'s face within the camera'"'"'s field-of-view.

45. The method of claim 31, wherein the camera is mounted to a dashboard of a vehicle such that the camera is oriented towards the face of an operator of the vehicle.

46. The method of claim 45, wherein the light sources are mounted to the vehicle such that the light sources are oriented towards the face of the operator for preferentially illuminating respective focal regions of the operator'"'"'s face within the camera'"'"'s field-of-view.

47. The method of claim 31, wherein the camera is directed towards an assembly line or conveyor belt for obtaining images of objects being directed along the assembly line or conveyor belt.

48. The method of claim 31, wherein the camera is directed towards a plurality of storage areas for obtaining images of objects located in the storage areas.

49. The method of claim 31, further comprising acquiring images of the camera'"'"'s field-of-view using the camera, and wherein the brightness in the respective focal regions of the light sources is sampled from the images acquired using the camera.

50. The method of claim 49, wherein light sources are deactivated when the camera is not acquiring images.

51. The method of claim 31, further comprising acquiring images of the camera'"'"'s field-of-view using the camera, and wherein the brightness in the respective focal regions of the light sources is sampled separately from the images acquired using the camera.

52. The method of claim 51, wherein light sources are activated only during periods when the camera is acquiring images and during periods when the brightness in the respective focal regions of the light sources is sampled.