Motion clutter suppression for image-subtracting cameras

• US 20040080623A1
• Filed: 08/14/2003
• Published: 04/29/2004
• Est. Priority Date: 08/15/2002
• Status: Active Grant

• Current Assignee: Eyegaze, Inc.
• Original Assignee: LC Technologies Incorporated
• Inventors: Cleveland, Dixon, Joyce, Arthur W. III
• Granted Patent: US 7,315,324 B2
• Time in Patent Office: Days
• US Class Current: 348/208.1
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 5/21 Circuitry for suppressing or minimising disturbance, e.g. moiré or halo

Abstract

Embodiments of the present invention relate to systems and methods for minimizing motion clutter in image-generation devices. Temporally-interleaved image-subtraction reduces the magnitude of motion clutter and has no adverse effect on the desired ambient-light cancellation of static images. Embodiments of image-interleaved generation devices employing temporally-interleaved image-subtraction include single, double, triple, and series accumulator configurations. All four embodiments allow synchronization with scene illuminators and may be implemented on a single electronic chip. Temporally-interleaved image-subtraction is particularly well suited for use in video eyetracking applications where ambient light and scene motion can cause significant problems.

Claims

1. A method for suppressing motion clutter comprising:

accumulating constituent images over multiple sub-periods within an overall frame period, wherein the sub-periods of the constituent images are temporally interleaved; and

combining the sub-periods of the constituent images to form a composite image.

2. The method according to claim 1, wherein combining the sub-periods of the constituent images to form the composite image comprises subtracting the sub-periods of the constituent images to form the composite image.

3. The method according to claim 1, wherein combining the sub-periods of the constituent images to form the composite image comprises adding the sub-periods of the constituent images to form the composite image.

4. The method according to claim 1, further comprising controlling a scene illuminator that is synchronized with the accumulation of the sub-periods of the constituent images.

5. The method according to claim 1, further comprising increasing the number of sub-periods to suppress the motion clutter, wherein the overall frame period is fixed.

6. An apparatus for suppressing motion clutter comprising:

means for accumulating constituent images over multiple sub-periods, wherein the sub-periods of the constituent images are temporally interleaved; and

means for combining the sub-periods of the constituent images to form a composite image.

7. The apparatus according to claim 6, wherein the means for combining the sub-periods of the constituent images to form the composite image is a processor that subtracts the sub-periods of the constituent images to form the composite image.

8. The apparatus according to claim 6, further comprising a scene illuminator that is synchronized with the accumulation of the sub-periods of the constituent images.

9. An image-generation apparatus comprising:

an image sensor that generates an image-sensor output from a physical wave input;

two constituent-image accumulators;

an accumulator-control switch, wherein the accumulator-control switch directs the image-sensor output to either one of the two constituent-image accumulators;

a controller, wherein the controller generates an accumulator-control switch signal that toggles the accumulator-control switch to direct the image sensor output to either one of the two constituent-image accumulators multiple times within an overall frame-capture period; and

a processor that subtracts the constituent images from the two constituent-image accumulators to form a composite image.

10. The apparatus according to claim 9, further comprising at least one scene illuminator, wherein the at least one scene illuminator is connected to its power source through a scene illuminator-control switch.

11. The apparatus according to claim 10, wherein the scene illuminator-control switch is synchronized with the accumulator-control switch.

12. The apparatus according to claim 11, wherein the scene illuminator-control switch is synchronized with the accumulator-control switch by the controller'"'"'s generating an illuminator-control signal in response to the accumulator-control switch signal.

13. The apparatus according to claim 12, wherein illuminator-control signal generated by the controller comprises the accumulator-control switch signal.

14. The apparatus according to claim 12, wherein illuminator-control signal generated by the controller does not comprise the accumulator-control switch signal.

15. The apparatus according to claim 11, wherein the scene illuminator-control switch is synchronized with the accumulator-control switch by the controller'"'"'s generating the accumulator-control switch signal in response to an externally generated illuminator-control signal.

16. The apparatus according to claim 9, wherein the apparatus comprises a single electronic chip.

17. The apparatus according to claim 9, further comprising a video eyetracker that tracks a user'"'"'s eye based on the composite image.

18. An image-generation apparatus comprising:

an image sensor that generates an image-sensor output from a physical wave input;

a sub-period accumulator that accumulates the image-sensor output within individual sub-frame periods;

two constituent-image accumulators that accumulate constituent images comprised of different sets of sub-images from the sub-period accumulator;

an accumulator-control switch, wherein the accumulator-control switch directs the output from the sub-period accumulator to be added to either one of the constituent-image accumulators;

a controller, wherein the controller initializes the sub-period accumulator before each sub-period and generates an accumulator-control switch signal that toggles the output of the sub-period accumulator between the two constituent-image accumulators multiple times within an overall frame-capture period; and

a processor that subtracts the constituent images from the two constituent-image accumulators to form a composite image.

19. The apparatus according to claim 18, further comprising at least one scene illuminator, wherein the at least one scene illuminator is connected to its power source through a scene illuminator-control switch.

20. The apparatus according to claim 19, wherein the scene illuminator-control switch is synchronized with the accumulator-control switch.

21. The apparatus according to claim 20, wherein the scene illuminator-control switch is synchronized with the accumulator-control switch by the controller'"'"'s generating an illuminator-control signal in response to the accumulator-control switch signal.

22. The apparatus according to claim 21, wherein illuminator-control signal generated by the controller comprises the accumulator-control switch signal.

23. The apparatus according to claim 21, wherein illuminator-control signal generated by the controller does not comprise the accumulator-control switch signal.

24. The apparatus according to claim 20, wherein the scene illuminator-control switch is synchronized with the accumulator-control switch by the controller'"'"'s generating the accumulator-control switch signal in response to an externally generated illuminator-control signal.

25. The apparatus according to claim 18, wherein the apparatus comprises a single electronic chip.

26. The apparatus according to claim 18, further comprising a video eyetracker that tracks a user'"'"'s eye based on the composite image.

27. An image-generation apparatus comprising:

an image sensor that generates an image-sensor output from a physical wave input;

an image accumulator that forms a composite image;

an accumulator-control switch, wherein the accumulator-control switch controls whether or not the image-sensor output is added to or subtracted from the image accumulator; and

a controller, wherein the controller generates an accumulator-control switch signal that toggles between addition and subtraction multiple times within an overall frame-capture period.

28. The apparatus according to claim 27, further comprising at least one scene illuminator, wherein the at least one scene illuminator is connected to its power source through a scene illuminator-control switch.

29. The apparatus according to claim 28, wherein the scene illuminator-control switch is synchronized with the accumulator-control switch.

30. The apparatus according to claim 29, wherein the scene illuminator-control switch is synchronized with the accumulator-control switch by the controller'"'"'s generating an illuminator-control signal in response to the accumulator-control switch signal.

31. The apparatus according to claim 30, wherein illuminator-control signal generated by the controller comprises the accumulator-control switch signal.

32. The apparatus according to claim 30, wherein illuminator-control signal generated by the controller does not comprise the accumulator-control switch signal.

33. The apparatus according to claim 29, wherein the scene illuminator-control switch is synchronized with the accumulator-control switch by the controller'"'"'s generating the accumulator-control switch signal in response to an externally generated illuminator-control signal.

34. The apparatus according to claim 27, wherein the apparatus comprises a single electronic chip.

35. The apparatus according to claim 27, further comprising a video eyetracker that tracks a user'"'"'s eye based on the composite image.

36. An image-generation apparatus comprising:

an image sensor that generates an image-sensor output from a physical wave input;

a sub-period accumulator that accumulates the image-sensor output within individual sub-frame periods;

a composite-image accumulator that accumulates a composite image comprised of added and subtracted sets of sub-images;

an accumulator-control switch, wherein the accumulator-control switch directs the output from the sub-period accumulator to be added to or subtracted from the composite-image accumulator; and

a controller, wherein the controller initializes the sub-period accumulator before each sub-period and generates an accumulator-control switch signal that toggles between addition and subtraction multiple times within an overall frame-capture period.

37. The apparatus according to claim 36, further comprising at least one scene illuminator, wherein the at least one scene illuminator is connected to its power source through a scene illuminator-control switch.

38. The apparatus according to claim 37, wherein the scene illuminator-control switch is synchronized with the accumulator-control switch.

39. The apparatus according to claim 38, wherein the scene illuminator-control switch is synchronized with the accumulator-control switch by the controller'"'"'s generating an illuminator-control signal in response to the accumulator-control switch signal.

40. The apparatus according to claim 39, wherein illuminator-control signal generated by the controller comprises the accumulator-control switch signal.

43. The apparatus according to claim 36, wherein the apparatus comprises a single electronic chip.

44. The apparatus according to claim 36, further comprising a video eyetracker that tracks a user'"'"'s eye based on the composite image.

41. The apparatus according to claim 41, wherein illuminator-control signal generated by the controller does not comprise the accumulator-control switch signal.

42. The apparatus according to claim 42, wherein the scene illuminator-control switch is synchronized with the accumulator-control switch by the controller'"'"'s generating the accumulator-control switch signal in response to an externally generated illuminator-control signal.

45. A method for suppressing motion clutter comprising:

accumulating a first image during a first sub-period;

accumulating a second image during a second sub-period;

accumulating a third image during a third sub-period;

accumulating a fourth image during a fourth sub-period;

adding the first image to the third image to derive the first sub-composite image;

adding the second image to the fourth image to derive the second sub-composite image; and

subtracting the second sub-composite image from the first sub-composite image to form a composite image,

wherein the second sub-period is between the first and third sub-periods, and the third sub-period is between the second and fourth sub-periods.

46. The method according to claim 45, further comprising providing illumination during the first and third sub-periods.

47. The method according to claim 45, further comprising tracking movement of an eye in the composite image.

48. The method according to claim 45, wherein the first sub-period is of a different duration than the second sub-period.

49. A method for suppressing motion clutter comprising:

capturing a first image during a first sub-period;

capturing a second image during a second sub-period;

calculating a first sub-composite image from the difference between the first image and second image;

capturing a third image during a third sub-period;

calculating a second sub-composite image from the difference between the third image and the first sub-composite image;

capturing a fourth image during a fourth sub-period; and

calculating a composite image from the difference between the fourth image and the second sub-composite image.

50. The method according to claim 49, further comprising providing illumination during the first and third sub-periods.

51. The method according to claim 49, further comprising tracking movement of an eye in the composite image.

52. The method according to claim 49, wherein the first sub-period is of a different duration than the second sub-period.