Interactive input system and method

• US 9,274,615 B2
• Filed: 11/13/2012
• Issued: 03/01/2016
• Est. Priority Date: 11/11/2011
• Status: Active Grant

• Current Assignee: PixArt Imaging Incorporated
• Original Assignee: PixArt Imaging Incorporated
• Inventors: Lam, Clinton, Hewitt, Lawrence Bruce, Akitt, Trevor, Ung, Charles
• Primary Examiner(s): Tzeng, Fred
• Application Number:
  US13/675,510
• Publication Number:
  US 20130120252A1
• Time in Patent Office: 1,204 Days
• Field of Search: 345/157, 345/156, 345/173, 345/175
• US Class Current: 1/1
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/042 by opto-electronic means
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/0428 by sensing at the edges of the touch surface the interruption of optical paths, e.g. an illumination plane, parallel to the touch surface which may be virtual sensing beam interruptions in a planar beam grid of an optical touch-screen G06F3/0421

Abstract

An interactive input system comprises at least two imaging assemblies capturing image frames of a region of interest from different vantages, each imaging assembly comprising an image sensor and signal processing circuitry implemented on an application specific integrated circuit (ASIC), image data acquired by the image sensor being pre-processed by the signal processing circuitry; and processing structure processing pre-processed image data output by the imaging assemblies to determine the location of at least one pointer within the region of interest.

Claims

1. An interactive input system comprising:

at least two imaging assemblies capturing image frames of a region of interest wherein the at least two imaging assemblies capturing images from different vantages, each imaging assembly comprising an image sensor and signal processing circuitry implemented on an application specific integrated circuit (ASIC), image data acquired by said image sensor being pre-processed by said signal processing circuitry; and

processing structure processing pre-processed image data output by said imaging assemblies to determine a location of at least one pointer within the region of interest.

2. The interactive input system of claim 1 wherein said processing structure comprises at least one digital signal processor.

3. The interactive input system of claim 2 wherein said processing structure further comprises an image data concentrator, said image data concentrator combining pre-processed image data received from said imaging assemblies into an image data stream and communicating the image data stream to said at least one digital signal processor.

4. The interactive input system of claim 3 wherein said image data concentrator is implemented on a field programmable gate array.

5. The interactive input system of claim 2 wherein said processing structure comprises a plurality of digital signal processors.

6. The interactive input system of claim 5 wherein one of the plurality of digital signal processors processes the pre-processed image data to generate pointer data and wherein another of the plurality of digital signal processors receives the pointer data from the one digital signal processor and triangulates the pointer data to determine location of the at least one pointer in (x,y) coordinates relative to an interactive surface.

7. The interactive input system of claim 6 wherein the one digital signal processor processes the pre-processed image data at least to determine pointer contact status and pointer boundaries.

8. The interactive input system of claim 7 wherein the one digital signal processor further processes the pre-processed image data to remove false pointer contact data.

9. The interactive input system of claim 6 wherein said processing structure further comprises an image data concentrator, said image data concentrator combining pre-processed image data received from said imaging assemblies into an image data stream and communicating the image data stream to said one digital signal processor.

10. The interactive input system of claim 9 wherein said image data concentrator is implemented on a field programmable gate array.

11. The interactive input system of claim 1 wherein each imaging assembly captures image frames in an image frame sequence and wherein said signal processing circuitry processes the image frames of the image frame sequence thereby to generate the pre-processed image data.

12. The interactive input system of claim 11 wherein said signal processing circuitry processes image frames to remove ambient light artifacts.

13. The interactive input system of claim 11 wherein said signal processing circuitry processes image frames to identify regions of interest therein representing potential pointers.

14. The interactive input system of claim 11 wherein said signal processing circuitry processes image frames to determine the identity of an active pointer.

15. The interactive input system of claim 13 wherein said signal processing circuitry identifies the regions of interest by performing at least one vertical intensity profile calculation and comparing the results with threshold data.

16. The interactive input system of claim 15 wherein said signal processing circuitry identifies the regions of interest by performing a plurality of vertical intensity profile calculations and comparing the results with threshold data.

17. The interactive input system of claim 16 wherein said signal processing circuitry performs the vertical intensity profile calculations in parallel.

18. The interactive input system of claim 17 wherein the image frame sequence comprises one image frame captured when the region of interest is flooded with illumination and a plurality of image frames captured in the absence of the illumination, said signal processing circuitry generating difference image data using said one image frame and a background image captured in the absence of the illumination and performing the vertical intensity profile calculations using the difference image data and the background image.

19. The interactive input system of claim 18 wherein said signal processing circuitry processes the plurality of image frames captured in the absence of the illumination to identify an active pointer.

20. The interactive input system of claim 1 wherein said processing structure is configured to provide configuration data to said imaging assemblies.

21. The interactive input system of claim 20 wherein said configuration data specifies a window of pixels in said captured image frames forming said image data to be pre-processed.

22. The interactive input system of claim 21 wherein said configuration data is modifiable to adjust the shape of said window.

23. The interactive input system of claim 21 further comprising a bezel generally surrounding said region of interest that is in the fields of view of said imaging assemblies and wherein said window has a shape generally conforming to the shape of said bezel as it appears in said captured image frames.

24. The interactive input system of claim 23 wherein said window has a shape generally resembling a chevron.

25. The interactive input system of claim 21 wherein said window has a dimension in the range of about ten to about thirty-two pixel rows.

26. An imaging assembly for an interactive input system comprising:

an image sensor having a field of view looking into a region of interest; and

processing circuitry receiving image data acquired by said image sensor and processing the image data, wherein said image sensor and processing circuitry are implemented on an application specific integrated circuit (ASIC);

wherein said imaging assembly is configured to capture image frames in an image frame sequence and wherein said processing circuitry processes the image frames of the image frame sequence to identify regions of interest therein representing potential pointers by performing a plurality of vertical intensity profile calculations in parallel and comparing the results with threshold data; and

, the image frame sequence comprises one image frame captured when the region of interest is flooded with illumination and a plurality of image frames captured in the absence of the illumination, said processing circuitry generating difference image data using said one image frame and a background image captured in the absence of the illumination and performing the vertical intensity profile calculations using the difference image data and the background image.

27. The imaging assembly of claim 26 wherein said processing circuitry processes a pair of image frames of said sequence to remove ambient light artifacts.

28. The imaging assembly of claim 26 wherein said processing circuitry processes image frames to determine the identity of an active pointer.

29. The imaging assembly of claim 26 further comprising a bezel map storing configuration data.

30. The imaging assembly of claim 29 wherein said configuration data specifies a window of pixels of said captured image frames forming said image data.

31. The imaging assembly of claim 30 wherein said configuration data is modifiable to adjust the shape of said window.

32. The imaging assembly of claim 30 wherein said window has a shape generally conforming to the shape of a bezel surrounding said region of interest as it appears in said captured image frames.

33. The imaging assembly of claim 32 wherein said window has a shape generally resembling a chevron.

34. The imaging assembly of claim 30 wherein said window has a dimension in the range of about ten to about thirty-two pixel rows.

35. An interactive input system comprising:

at least two imaging assemblies having fields of view encompassing a region of interest wherein the at least two imaging assemblies capturing images from different vantages, each imaging assembly capturing a sequence of image frames, one image frame being captured when the region of interest is flooded with illumination and a plurality of image frames being captured in the absence of the illumination, each imaging assembly having signal processing circuitry generating difference image data using said one image frame and a background image frame captured in the absence of the illumination and performing vertical intensity calculations using the difference image data and the background image thereby to identify one or more potential pointers; and

processing structure processing the output of said imaging assemblies to determine a location of each pointer within the region of interest.

36. The interactive input system of claim 35 wherein said signal processing circuitry processes the plurality of image frames captured in the absence of the illumination to identify an active pointer.

37. An interactive input system comprising:

at least two imaging assemblies capturing image frames of a region of interest that is surrounding by bezel segments from different vantages, each imaging assembly processing image data within a window of the captured image frames, the window having a shape generally conforming to the shape of a bezel including the bezel segments as it appears in the captured image frames; and

processing structure processing the image data output by the imaging assemblies to determine a location of at least one pointer within the region of interest.

38. The interactive input system of claim 37 wherein said window is modifiable to adjust the shape thereof.

39. The interactive input system of claim 38 wherein said window has a shape generally resembling a chevron.

40. The interactive input system of claim 37 wherein said window has a dimension in the range of about ten to about thirty-two pixel rows.

41. An imaging assembly for an interactive input system comprising:

an image sensor having a field of view looking into a region of interest; and

processing circuitry processing a sequence of image frames captured by the image sensor, one image frame being captured when the region of interest is flooded with backlight illumination and a plurality of image frames being captured in the absence of the backlight illumination, said processing circuitry generating difference image data using said one image frame and a background image captured in the absence of the backlight illumination and performing vertical intensity profile calculations using the difference image data and the background image thereby to identify one or more potential pointers in the image frames.