Motion clutter suppression for image-subtracting cameras

• US 7,315,324 B2
• Filed: 08/14/2003
• Issued: 01/01/2008
• Est. Priority Date: 08/15/2002
• Status: Active Grant

• Current Assignee: Eyegaze, Inc.
• Original Assignee: LC Technologies Incorporated
• Inventors: Cleveland, Dixon, Joyce, Arthur W. III
• Primary Examiner(s): Vu; Ngoc-Yen
• Assistant Examiner(s): Cutler; Albert H
• Application Number:
  US10/640,248
• Publication Number:
  US 20040080623A1
• Time in Patent Office: 1,601 Days
• Field of Search: 348/208.1, 348/208.4, 348/208.6, 348/208.13, 348/208.14, 348/216.1, 348/217.1, 348/218.1, 348/221.1, 348/223.1, 348/227.1, 348/228.1, 348/234, 348/235, 348/236, 348/335, 348/345, 348/349, 348/352, 348/356, 348/362, 348/363, 348/366, 348/367, 348/368, 348/370, 348/371, 382/117, 382/115, 382/162, 382/163, 382/164, 382/166, 382/167, 382/174, 382/254, 382/255, 382/256, 382/258, 382/266, 382/268, 386/1, 386/6, 386/7, 386/8, 386/46, 386/47, 386/48, 386/49, 386/50, 386/68, 386/72, 386/73, 386/117, 386/120, 386/121
• US Class Current: 348/208.4
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 5/21 Circuitry for suppressing or minimising disturbance, e.g. moiré or halo

Abstract

Embodiments of the present invention relate to systems and methods for minimizing motion clutter in image-generation devices. Temporally-interleaved image-subtraction reduces the magnitude of motion clutter and has no adverse effect on the desired ambient-light cancellation of static images. Embodiments of image-generation devices employing temporally-interleaved image-subtraction include single, double, triple, and series accumulator configurations. All four embodiments allow synchronization with scene illuminators and may be implemented on a single electronic chip. Temporally-interleaved image-subtraction is particularly well suited for use in video eyetracking applications where ambient light and scene motion can cause significant problems.

Claims

1. An apparatus for generating eye images including an image-generation apparatus comprising:

an image sensor that generates an output from a physical wave input;

two constituent-image accumulators;

an accumulator-control switch that directs the image-sensor output to either one of the two constituent-image accumulators;

a controller for the accumulator-control switch that toggles the image sensor output between the two constituent-image accumulators over multiple sub-periods within an overall frame-capture period, wherein the sub-periods of the constituent images are temporally interleaved; and

a processor that subtracts the constituent images from the two constituent-image accumulators to form a composite image.

2. The apparatus according to claim 1, further comprising at least one scene illuminator, wherein the at least one scene illuminator is connected to its power source through a scene illuminator-control switch.

3. The apparatus according to claim 2, wherein the scene illuminator-control switch is synchronized with the accumulator-control switch.

4. The apparatus according to claim 3, wherein the scene illuminator-control switch is synchronized with the accumulator-control switch by the controller'"'"'s generating an illuminator-control signal in response to the accumulator-control switch signal.

5. The apparatus according to claim 4, wherein illuminator-control signal generated by the controller comprises the accumulator-control switch signal.

6. The apparatus according to claim 4, wherein illuminator-control signal generated by the controller does not comprise the accumulator-control switch signal.

7. The apparatus according to claim 3, wherein the scene illuminator-control switch is synchronized with the accumulator-control switch by the controller'"'"'s generating the accumulator-control switch signal in response to an externally generated illuminator-control signal.

8. An apparatus for generating eve images including an image-generation apparatus comprising:

an image sensor that generates an output from a physical wave input;

a sub-period accumulator that accumulates the image-sensor output within individual sub-frame periods;

two constituent-image accumulators that accumulate constituent images comprised of different sets of sub-images from the sub-period accumulator;

an accumulator-control switch that directs the output from the sub-period accumulator to be added to either one of the constituent-image accumulators;

a controller, wherein the controller initializes the sub-period accumulator before each sub-period and generates an accumulator-control switch signal that toggles the output of the sub-period accumulator between the two constituent-image accumulators over multiple sub-periods within an overall frame-capture period, wherein the sub-periods of the constituent images are temporally interleaved; and

a processor that subtracts the constituent images from the two constituent-image accumulators to form a composite image.

9. The apparatus according to claim 8, further comprising at least one scene illuminator, wherein the at least one scene illuminator is connected to its power source through a scene illuminator-control switch.

10. The apparatus according to claim 9, wherein the scene illuminator-control switch is synchronized with the accumulator-control switch.

11. The apparatus according to claim 10, wherein the scene illuminator-control switch is synchronized with the accumulator-control switch by the controller'"'"'s generating an illuminator-control signal in response to the accumulator-control switch signal.

12. The apparatus according to claim 11, wherein illuminator-control signal generated by the controller comprises the accumulator-control switch signal.

13. The apparatus according to claim 11, wherein illuminator-control signal generated by the controller does not comprise the accumulator-control switch signal.

14. The apparatus according to claim 10, wherein the scene illuminator-control switch is synchronized with the accumulator-control switch by the controller'"'"'s generating the accumulator-control switch signal in response to an externally generated illuminator-control signal.

15. An apparatus for generating eye images including an image-generation apparatus comprising;

an image sensor that generates an output from a physical wave input;

an image accumulator that forms a composite image;

an accumulator-control switch that controls whether the image-sensor output is added to or subtracted from the image accumulator; and

a controller for the accumulator-control switch that toggles between addition and subtraction over multiple sub-periods within an overall frame-capture period, wherein the sub-periods of the constituent images are temporally interleaved.

16. The apparatus according to claim 15, further comprising at least one scene illuminator, wherein the at least one scene illuminator is connected to its power source through a scene illuminator-control switch.

17. The apparatus according to claim 16, wherein the scene illuminator-control switch is synchronized with the accumulator-control switch.

18. The apparatus according to claim 17, wherein the scene illuminator-control switch is synchronized with the accumulator-control switch by the controller'"'"'s generating an illuminator-control signal in response to the accumulator-control switch signal.

19. The apparatus according to claim 18, wherein illuminator-control signal generated by the controller comprises the accumulator-control switch signal.

20. The apparatus according to claim 18, wherein illuminator-control signal generated by the controller does not comprise the accumulator-control switch signal.

21. The apparatus according to claim 17, wherein the scene illuminator-control switch is synchronized with the accumulator-control switch by the controller'"'"'s generating the accumulator-control switch signal in response to an externally generated illuminator-control signal.

22. An apparatus for generating eye images including an image-generation apparatus comprising:

an image sensor that generates an output from a physical wave input;

a sub-period accumulator that accumulates the image-sensor output within individual sub-frame periods;

a composite-image accumulator that accumulates a composite image comprised of added and subtracted sets of sub-images;

an accumulator-control switch that directs the output from the sub-period accumulator to be added to or subtracted from the composite-image accumulator; and

a controller that initializes the sub-period accumulator before each sub-period and generates an accumulator-control switch signal that toggles between addition and subtraction over multiple sub-periods within an overall frame-capture period, wherein the sub-periods of the constituent images are temporally interleaved.

23. The apparatus according to claim 22, further comprising at least one scene illuminator, wherein the at least one scene illuminator is connected to its power source through a scene illuminator-control switch.

24. The apparatus according to claim 23, wherein the scene illuminator-control switch is synchronized with the accumulator-control switch.

25. The apparatus according to claim 24, wherein the scene illuminator-control switch is synchronized with the accumulator-control switch by the controller'"'"'s generating an illuminator-control signal in response to the accumulator-control switch signal.

26. The apparatus according to claim 25, wherein illuminator-control signal generated by the controller comprises the accumulator-control switch signal.

27. The apparatus according to claim 25, wherein illuminator-control signal generated by the controller does not comprise the accumulator-control switch signal.

28. The apparatus according to claim 24, wherein the scene illuminator-control switch is synchronized with the accumulator-control switch by the controller'"'"'s generating the accumulator-control switch signal in response to an externally generated illuminator-control signal.

29. A method for generating eye images including a method for suppressing motion clutter comprising:

accumulating a first image during a first sub-period;

accumulating a second image during a second sub-period;

accumulating a third image during a third sub-period;

accumulating a fourth image during a fourth sub-period;

adding the first image to the third image to derive the first sub-composite image;

adding the second image to the fourth image to derive the second sub-composite image; and

subtracting the second sub-composite image from the first sub-composite image to form a composite image,

wherein the second sub-period is between the first and third sub-periods, and the third sub-period is between the second and fourth sub-periods.

30. The method according to claim 29, further comprising providing illumination during the first and third sub-periods.

31. The method according to claim 29, wherein the first sub-period is of a different duration than the second sub-period.

32. A method for generating eye images including a method for suppressing motion clutter comprising:

capturing a first image during a first sub-period;

capturing a second image during a second sub-period;

calculating a first sub-composite image from the difference between the first image and second image;

capturing a third image during a third sub-period;

calculating a second sub-composite image from the difference between the third image and the first sub-composite image;

capturing a fourth image during a fourth sub-period; and

calculating a composite image from the difference between the fourth image and the second sub-composite image.

33. The method according to claim 32, further comprising providing illumination during the first and third sub-periods.

34. The method according to claim 32, wherein the first sub-period is of a different duration than the second sub-period.