Apparatus and method of monitoring a subject's eyes using two different wavelengths of light

• US 6,082,858 A
• Filed: 04/29/1999
• Issued: 07/04/2000
• Est. Priority Date: 04/29/1998
• Status: Active Grant

• Current Assignee: Carnegie Mellon University
• Original Assignee: Carnegie Mellon University
• Inventors: Davis, Robert K., Grace, Richard
• Primary Examiner(s): MANUEL, GEORGE C
• Application Number:
  US09/301,996
• Time in Patent Office: 432 Days
• Field of Search: 351/200, 351/206, 351/207, 351/213, 351/215, 351/221, 250/559.3, 257/440, 257/184, 356/73, 356/402, 356/407, 356/425
• US Class Current: 351/200
• CPC Class Codes: A61B 2503/22 Motor vehicles operators, e.g. drivers, pilots, captains
• CPC Class Codes: A61B 2562/0233 Special features of optical sensors or probes classified in A61B5/00
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: A61B 5/1103 Detecting eye twinkling
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/18 for vehicle drivers or machine operators
• CPC Class Codes: A61B 5/6821 Eye
• CPC Class Codes: B60K 28/06 responsive to incapacity of driver
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/19 Sensors therefor

Abstract

A device and method for monitoring a subject'"'"'s eyes. The device includes a light source producing light of a first wavelength and producing light of a second wavelength, and the device includes at least one image sensor positioned to receive light reflected from the subject'"'"'s eyes. The at least one image sensor produces a first signal indicative of light of the first wavelength and produces a second signal indicative of light of the second wavelength. The device may include a controller connected to the at least one image sensor and receiving the first and second signals. The controller may subtract the first signal from the second signal and produce a third signal indicative thereof. The controller may also provide a synchronization signal to the at least one image sensor and may also provide the signal to other components, such as the light source. One or more filters may be provided in the path of the reflected light so as to filter the light before it reaches the at least one image sensor. One or more lens may also be provided to focus the reflected light before the reflected light reaches the at least one image sensor.

Claims

1. A device for monitoring a subject, comprising:

a source of light having first and second wavelengths wherein the first wavelength does not equal the second wavelength;

at least one image sensor including a plurality of pixels, the image sensor producing a first signal indicative of the plurality of pixels representing reflected light having the first wavelength and producing a second signal indicative of the plurality of pixels representing reflected light having the second wavelength; and

a controller receiving the first and second signals and producing a third signal indicative of the first signal subtracted from the second signal.

2. The device of claim 1, wherein the first wavelength is 950 nm.

3. The device of claim 2, wherein the second wavelength is 850 nm.

4. The device of claim 1, wherein a ratio of light of the first wavelength reflected by the subject to light of the first wavelength absorbed by the subject is not equal to a ratio of light of the second wavelength reflected by the subject to light of the second wavelength absorbed by the subject.

5. The device of claim 1, wherein the source of light is oriented to direct light toward the subject and the at least one image sensor is oriented to receive light reflected from the subject.

6. The device of claim 5, wherein:

the source of light includes a plurality of light sources defining an opening between the plurality of light sources; and

the at least one image sensor is oriented to receive reflected light passing through the opening.

7. The device of claim 1, wherein the at least one image sensor includes a first image sensor producing the first and second signals.

8. The device of claim 1, wherein the at least one image sensor includes a first image sensor producing the first signal and a second image sensor producing the second signal.

9. The device of claim 8, wherein the first image sensor receives reflected light having the first wavelength to the exclusion of the second wavelength and the second image sensor receives light having the second wavelength to the exclusion of the first wavelength.

10. The device of claim 8, further comprising a beamsplitter oriented to split light reflected from the subject to the first and second image sensors.

11. The device of claim 10, further comprising:

a first filter between the beamsplitter and the first image sensor; and

a second filter between the beamsplitter and the second image sensor.

12. The device of claim 11, further comprising:

a first lens between the beamsplitter and the first image sensor; and

a second lens between the beamsplitter and the second image sensor.

13. The device of claim 10, further comprising a first filter, wherein the beamsplitter is oriented between the first filter and the first and second image sensors.

14. The device of claim 13, further comprising:

a first lens between the beamsplitter and the first image sensor; and

a second lens between the beamsplitter and the second image sensor.

15. The device of claim 10, further comprising a mirror oriented to reflect light from the beamsplitter to the first image sensor.

16. The device of claim 15, further comprising:

a first lens between the mirror and the first image sensor; and

a second lens between the beamsplitter and the second image sensor.

17. The device of claim 8, wherein the first and second image sensors simultaneously capture an image of the subject.

18. The device of claim 17, wherein the controller provides a synchronization signal to each of the first and second image sensors.

19. The device of claim 1, wherein the controller includes:

a converter having an input terminal coupled to the at least one image sensor; and

a processor coupled to the converter.

20. The device of claim 1, further comprising a video monitor in communication with the controller.

21. The device of claim 1, further comprising a feedback device in communication with the controller.

22. The device of claim 1, wherein the image sensor is selected from the group consisting of a CCD image sensor and a CMOS-type image sensor.

23. A device for monitoring a subject'"'"'s eyes, comprising:

a source of light oriented to direct light toward the subject'"'"'s eyes, the light having first and second wavelengths, wherein the first wavelength does not equal the second wavelength;

a first image sensor;

a second image sensor;

a beamsplitter oriented to split light reflected from the subject'"'"'s eyes to the first and second image sensor;

a first filter between the beamsplitter and the first image sensor;

a second filter between the beamsplitter and the second image sensor; and

a controller in communication with the first and second image sensors.

24. The device of claim 23, wherein the first image sensor produces a first signal indicative of reflected light having the first wavelength, the second image sensor produces a second signal indicative of reflected light having the second wavelength, and the controller produces a third signal indicative of the first signal subtracted from the second signal.

25. A device for monitoring a subject'"'"'s eyes, comprising:

a source of light oriented to transmit light toward the subject'"'"'s eyes, the light having first and second wavelengths, wherein the first wavelength does not equal the second wavelength;

a first image sensor;

a second image sensor;

a beamsplitter oriented to split light reflected from the subject'"'"'s eyes to the first and second image sensors;

a first filter oriented to filter light reflected by the subject'"'"'s eyes before being split by the beamsplitter; and

a controller in communication with the first and second image sensors.

26. The device of claim 25, wherein the first filter is an infrared high pass filter and wherein the beamsplitter is a dichroic beamsplitter.

27. The device of claim 25, wherein the first image sensor produces a first signal indicative of reflected light having the first wavelength, the second image sensor produces a second signal indicative of reflected light having the second wavelength, and the controller produces a third signal indicative of the first signal subtracted from the second signal.

28. A device for monitoring a subject'"'"'s eyes, comprising:

a source of light oriented to transmit light toward the subject'"'"'s eyes, the light having first and second wavelengths, wherein the first wavelength does not equal the second wavelength;

a first image sensor;

a second image sensor;

a beamsplitter oriented to split light reflected from the subject'"'"'s eyes to the first and second image sensors;

a mirror oriented to reflect light from the beamsplitter to the first image sensor; and

a controller in communication with the first and second image sensors.

29. The device of claim 28, wherein the first image sensor produces a first signal indicative of reflected light having the first wavelength, the second image sensor produces a second signal indicative of reflected light having the second wavelength, and the controller produces a third signal indicative of the first signal subtracted from the second signal.

30. A device, comprising:

a source of light oriented to transmit light toward a subject, the light having first and second wavelengths wherein the first wavelength does not equal the second wavelength;

means for producing a first signal indicative of a plurality of pixels representing light reflected by the subject having the first wavelength;

means for producing a second signal indicative of a plurality of pixels representing light reflected by the subject having the second wavelength; and

means for producing a third signal indicative of the first signal subtracted from the second wavelength.

31. The device of claim 30, further comprising means for separating reflected having the first wavelength from reflected light having the second wavelength.

32. The device of claim 30, wherein the means for producing a first signal include means for capturing a first image of the subject and the means for producing a second signal includes means for capturing a second image of the subject.

33. The device of claim 32, further comprising means for simultaneously capturing the first and second images.

34. A method, comprising:

directing light having a first wavelength toward a subject;

directing light having a second wavelength toward the subject, wherein the first wavelength does not equal the second wavelength;

capturing a first image of light reflected by the subject and having the first wavelength, wherein the first image is represented as a plurality of pixels;

capturing a second image of light reflected by the subject and having the second wavelength, wherein the second image is represented as a plurality of pixels; and

subtracting a signal indicative of the first image from a signal indicative of the second image.

35. The method of claim 34, wherein directing light having the first wavelength and directing light having the second wavelength includes simultaneously directing light having the first and second wavelengths.

36. The method of claim 35, wherein capturing the first image and capturing the second image includes simultaneously capturing the first and second images.

37. The method of claim 34, wherein directing light having the first wavelength and directing light having the second wavelength includes consecutively directing light having the first wavelength and light having the second wavelength.

38. The method of claim 37, wherein capturing the first image and capturing the second image include consecutively capturing the first and second images.

39. The method of claim 34, further comprising separating light reflected from the subject having the first wavelength from light reflected from the subject having the second wavelength.

40. A method of for monitoring a subject'"'"'s eyes, comprising:

illuminating the subject'"'"'s eyes with light having a first wavelength;

illuminating the subject'"'"'s eyes with light having a second wavelength, wherein the first wavelength does not equal the second wavelength;

sensing light of the first wavelength reflected by the subject'"'"'s eyes;

sensing light of the second wavelength reflected by the subject'"'"'s eyes;

providing a first signal indicative of the sensed light having the first wavelength;

providing a second signal indicative of the sensed light having the second wavelength; and

providing a third signal indicative of the first signal subtracted from the second signal.

41. The method of claim 40, wherein illuminating with light having the first wavelength and illuminating with light having the second wavelength includes simultaneously illuminating the subject'"'"'s eyes with light having the first and second wavelengths.

42. The method of claim 41, wherein sensing light of the first wavelength and sensing light of the second wavelength includes simultaneously sensing light having the first and second wavelengths.

43. The method of claim 40, wherein illuminating the subject'"'"'s eyes with light having the first wavelength includes illuminating the subject'"'"'s eyes with light having a wavelength of 850 nm, and illuminating the subject'"'"'s eyes with light having the second wavelength includes illuminating the subject'"'"'s eyes with light having a wavelength of 950 nm.

44. The method of claim 40, wherein illuminating with light having the first wavelength and illuminating with light having the second wavelength includes consecutively illuminating the subject'"'"'s eyes with light having the first wavelength and illuminating the subject'"'"'s eyes with light having the second wavelength.

45. The method of claim 44, wherein sensing light of the first wavelength and sensing light of the second wavelength includes consecutively sensing light having the first wavelength and sensing light having the second wavelength.

46. A device for monitoring a subject, comprising:

a source of light having first and second wavelengths wherein the first wavelength does not equal the second wavelength;

at least one image sensor producing a first signal indicative of a reflected image of the subject in the form of light having the first wavelength and producing a second signal indicative of a reflected image of the subject in the form of light having the second wavelength; and

a controller receiving the first and second signals and producing a third signal indicative of the first signal subtracted from the second signal.

47. The device of claim 46, wherein the image sensor is selected from the group consisting of a CCD image sensor and a CMOS-type image sensor.

48. The device of claim 46, wherein the first wavelength is 950 nm.

49. The device of claim 48, wherein the second wavelength is 850 nm.

50. The device of claim 46, wherein a ratio of light of the first wavelength reflected by the subject to light of the first wavelength absorbed by the subject is not equal to a ratio of light of the second wavelength reflected by the subject to light of the second wavelength absorbed by the subject.

51. The device of claim 46, wherein the source of light is oriented to direct light toward the subject and the at least one image sensor is oriented to receive light reflected from the subject.

52. The device of claim 51, wherein:

the source of light includes a plurality of light sources defining an opening between the plurality of light sources; and

the at least one image sensor is oriented to receive reflected light passing through the opening.

53. The device of claim 46, wherein the controller includes:

a converter having an input terminal coupled to the at least one image sensor; and

a processor coupled to the converter.

54. A device, comprising:

a source of light oriented to transmit light toward a subject, the light having first and second wavelengths wherein the first wavelength does not equal the second wavelength;

means for receiving a first image of the subject from light reflected by the subject having the first wavelength;

means for receiving a second image of the subject from light reflected by the subject having the second wavelength;

means for producing a first signal indicative of the first image;

means for producing a second signal indicative of second image; and

means for producing a third signal indicative of the first signal subtracted from the second signal.

55. A method, comprising:

directing light having a first wavelength toward a subject;

directing light having a second wavelength toward the subject, wherein the first wavelength does not equal the second wavelength;

capturing a first image in the form of light reflected by the subject and having the first wavelength;

capturing a second image in the form of light reflected by the subject and having the second wavelength; and

producing a signal indicative of the first image subtracted from the second image.

56. The method of claim 55, wherein directing light having the first wavelength and directing light having the second wavelength includes simultaneously directing light having the first and second wavelengths.

57. The method of claim 56, wherein capturing the first image and capturing the second image includes simultaneously capturing the first and second images.

58. The method of claim 55, wherein directing light having the first wavelength and directing light having the second wavelength includes consecutively directing light having the first wavelength and light having the second wavelength.

59. The method of claim 58, wherein capturing the first image and capturing the second image include consecutively capturing the first and second images.

60. The method of claim 55, further comprising separating light reflected from the subject having the first wavelength from light reflected from the subject having the second wavelength.

61. A method, comprising:

directing light having a first wavelength toward a subject;

directing light having a second wavelength toward the subject, wherein the first wavelength does not equal the second wavelength;

capturing a first image of light reflected by the subject and having the first wavelength, wherein the first image is represented as a plurality of pixels;

capturing a second image of light reflected by the subject and having the second wavelength, wherein the second image is represented as a plurality of pixels; and

producing a signal indicative of the first image subtracted from the second image.