Interactive input system and method

• US 9,582,119 B2
• Filed: 01/11/2013
• Issued: 02/28/2017
• Est. Priority Date: 01/11/2012
• Status: Active Grant

• Current Assignee: Smart Technologies ULC (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Smart Technologies ULC (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Lam, Clinton, Popovich, David, Rattray, Robbie, McGibney, Grant
• Primary Examiner(s): Patel, Kumar
• Assistant Examiner(s): ONYEKABA, AMY
• Application Number:
  US13/823,280
• Publication Number:
  US 20140313165A1
• Time in Patent Office: 1,509 Days
• Field of Search: 345156-184, 178 1801- 2004
• US Class Current: 1/1
• CPC Class Codes: F21V 21/14 Adjustable mountings
• CPC Class Codes: F21V 5/04 of lens shape
• CPC Class Codes: G03B 21/134 Projectors combined with typing apparatus or with printing apparatus
• CPC Class Codes: G06F 3/0354 with detection of 2D relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
• CPC Class Codes: G06F 3/03542 Light pens for emitting or receiving light
• CPC Class Codes: G06F 3/0386 for light pen
• CPC Class Codes: G06F 3/0418 for error correction or compensation, e.g. based on parallax, calibration or alignment
• CPC Class Codes: G06F 3/042 by opto-electronic means
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/0428 by sensing at the edges of the touch surface the interruption of optical paths, e.g. an illumination plane, parallel to the touch surface which may be virtual sensing beam interruptions in a planar beam grid of an optical touch-screen G06F3/0421
• CPC Class Codes: H04N 25/50 Control of the SSIS exposure
• CPC Class Codes: H04N 9/3179 Video signal processing therefor
• CPC Class Codes: H04N 9/3194 including sensor feedback

Abstract

An image capture method comprises generating a synchronization signal based on modulated illumination; and synchronizing image frame capture of at least one image sensor using the synchronization signal with the illumination timing of an active pointer within a region of interest in the field of view of the at least one image sensor.

Claims

1. An image capture method comprising:

generating a synchronization signal based on a modulated illumination signal;

using said synchronization signal to synchronize;

(1) illumination timing of an illumination assembly emitting a light curtain over a region of interest, (2) image frame capture by at least one image sensor of the region of interest, and (3) illumination timing of an illumination source of an active pointer positioned within the region of interest such that at least two image frames are captured including;

a first image frame being captured when said illumination assembly is on, and

a second image frame being captured when the illumination source of said active pointer is on and said illumination assembly is off,

wherein the first image frame and the second image frame are successive image frames;

processing the at least two image frames captured to;

identify a passive pointer when there is a bright region in the first image frame and there is no bright region in the second image frame and

determine a location of the passive pointer within the region of interest of the first image frame; and

processing the at least two image frames captured to;

identify said active pointer when there is not bright region in the first image frame and there is a bright region in the second image frame and

determine the location of the active pointer within the region of interest of the second image frame.

2. The method of claim 1, further comprising generating the modulated illumination signal using the illumination source of the active pointer when the active pointer is brought into said region of interest and processing the modulated illumination signal to generate said synchronization signal.

3. The method of claim 2, wherein said modulated illumination signal comprises a carrier signal modulated by periodic signals generated by the illumination source of said active pointer, the periodic signals being generated at a rate to which the frame rate of the at least one image sensor is adjusted.

4. The method of claim 3, wherein the carrier signal has a frequency different from frequencies of signals emitted by conventional consumer electronic device infrared emitters.

5. The method of claim 2, wherein the modulated illumination signal further comprises active pen tool attribute information, the attribute information being associated with the determined location of the active pointer.

6. The method of claim 1, wherein image frames captured when said illumination assembly is on have a longer integration time than image frames captured when said illumination assembly is off.

7. The method of claim 1, wherein the synchronization signal is based on the frequency at which the illumination assembly that emits the light curtain over the region of interest is switched on and off.

8. The method of claim 7, further comprising modulating the light curtain that is emitted over said region of interest by said illumination assembly.

9. The method of claim 8, wherein the active pointer detects the modulated light curtain and the illumination source of the active pointer emits illumination in an on and off pattern that is timed so that the illumination source of the active pointer emits illumination when the modulated light curtain is off.

10. The method of claim 7, further comprising modulating the light curtain that is emitted over said region of interest by said illumination assembly, following each on phase of said light curtain, conditioning said light curtain to a low intensity state before turning said light curtain off.

11. The method of claim 10, wherein the active pointer detects the modulated light curtain in the low intensity condition and the illumination source of the active pointer emits illumination in an on and off pattern that is timed so that the illumination source of the active pointer emits illumination when the modulated light curtain is off.

12. The method of claim 1, further comprising:

orienting a position of a light curtain emitted by an illumination assembly so that the light curtain is parallel to the region of interest on a display surface illuminated by the illumination assembly.

13. The method of claim 1, further comprising:

upon determining that an intensity and a size of the bright region in the second image frame are greater than an intensity and a size of the bright region in the first image frame, identifying that said active pointer is hovering over a display surface when there is a bright region in the first image frame and there is a bright region in the second image frame, wherein the display surface is illuminated by the illumination assembly.

14. The method of claim 1, further comprising:

upon determining that an intensity and a size of the bright region in the second image frame are lesser than an intensity and a size of the bright region in the first image frame, identifying that said active pointer is in contact with a display surface when there is a bright region in the first image frame and there is a bright region in the second image frame, wherein the display surface is illuminated by the illumination assembly.

15. An interactive input system comprising:

at least one image sensor having a field of view aimed at a region of interest;

an illumination assembly configured to emit a light curtain over said region of interest; and

processing structure configured to;

process image frames captured by said at least one image sensor and determine locations of one or more pointers brought into said region of interest, wherein illumination timing of the illumination assembly, image frame capture of said at least one image sensor and illumination timing of an illumination source of an active pointer brought into said region of interest are synchronized using a synchronization signal based on the modulated illumination signal such that at least two image frames are captured including;

a first image frame being captured when said illumination assembly is on, and

a second image frame being captured when the illumination source of said active pointer is on and said illumination assembly is off,

wherein the first image frame and the second image frame are successive image frames;

process the at least two image frames captured to;

identify a passive pointer when there is a bright region in the first image frame and there is no bright region in the second image frame and

determine a location of the passive pointer within the region of interest of the first image frame; and

process the at least two image frames captured to;

identify said active pointer when there is no bright region in the first image frame and there is a bright region in the second image frame and

determine the location of the active pointer within the region of interest of the second image frame.

16. The interactive input system of claim 15, wherein said active pointer is configured to generate the modulated illumination signal when the active pointer is brought into said region of interest and wherein said processing structure is configured to process the modulated illumination signal to generate said synchronization signal.

17. The interactive input system of claim 16, wherein said modulated illumination signal comprises a carrier signal modulated by periodic signals generated by the illumination source of said active pointer, the periodic signals being generated at a rate used to adjust the frame rate of the at least one image sensor.

18. The interactive input system of claim 17, wherein the carrier signal has a frequency different from frequencies of signals emitted by conventional consumer electronic device infrared emitters.

19. The interactive input system of claim 16, wherein the modulated illumination signal further comprises active pen tool attribute information, the attribute information being associated with the determined location of the active pointer.

20. The interactive input system of claim 15, wherein said at least one image sensor is controlled so that image frames captured when said illumination assembly is on have a longer integration time than image frames captured when said illumination assembly is off.

21. The interactive input system of claim 15, wherein said illumination assembly is configured to modulate the light curtain that is emitted over said region of interest.

22. The interactive input system of claim 21, wherein the active pointer is configured to detect the modulated light curtain and the illumination source of the active pointer is configured to emit illumination in an on and off pattern that is timed so that the illumination source of the active pointer emits illumination when the modulated light curtain is off.

23. The interactive input system of claim 15, wherein said illumination assembly is configured to modulate the light curtain that is emitted over said region of interest, following each on phase of said light curtain, the light curtain being conditioned to a low intensity state before being turned off.

24. The interactive input system of claim 23, wherein the active pointer is configured to detect the modulated light curtain in the low intensity condition and the illumination source of the active pointer is configured to emit illumination in an on and off pattern that is timed so that the illumination source of the active pointer emits illumination when the modulated light curtain is off.

25. The interactive input system of claim 15, wherein the processing structure is further configured to:

orient a position of the light curtain emitted by the illumination assembly so that the light curtain intersects the region of interest on a display surface illuminated by the illumination assembly.

26. The interactive input system of claim 15, wherein the processing structure is further configured to:

upon determining that an intensity and a size of the bright region in the second image frame are greater than an intensity and a size of the bright region in the first image frame, identify that said active pointer is hovering over a display surface when there is a bright region in the first image frame and there is a bright region in the second image frame, wherein the display surface is illuminated by the illumination assembly.

27. The interactive input system of claim 15, wherein the processing structure is further configured to:

upon determining that an intensity and a size of the bright region in the second image frame are lesser than an intensity and a size of the bright region in the first image frame, identify that said active pointer is in contact with a display surface when there is a bright region in the first image frame and there is a bright region in the second image frame, wherein the display surface is illuminated by the illumination assembly.