Interactive input system and method

• US 9,298,318 B2
• Filed: 12/01/2011
• Issued: 03/29/2016
• Est. Priority Date: 12/01/2010
• Status: Active Grant

• Current Assignee: Smart Technologies ULC (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Smart Technologies ULC (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Morrison, Gerald, Newton, John, Bell, Gareth
• Primary Examiner(s): Wang-Hurst, Kathy
• Assistant Examiner(s): SHEN, PEIJIE
• Application Number:
  US13/309,377
• Publication Number:
  US 20120139835A1
• Time in Patent Office: 1,580 Days
• Field of Search: 345/156, 345/173, 345/175, 345/419
• US Class Current: 1/1
• CPC Class Codes: G06F 3/0421 by interrupting or reflecting a light beam, e.g. optical touch-screen
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386

Abstract

An interactive input system comprises a display panel; a first illumination source providing illumination across an outer surface of the display panel such that when a pointer is brought into proximity with the outer surface, the pointer reflects illumination from the first illumination source through the display panel; at least one imaging device having a field of view aimed at an opposite outer surface of the display panel, the at least one imaging device capturing images including illumination reflected by the pointer through the display panel; and processing structure in communication with the at least one imaging device, said processing structure processing the captured images to determine a location of the pointer relative to the outer surface.

Claims

1. An interactive input system comprising:

a liquid crystal display panel;

a first illumination source configured to provide a sheet of illumination across and contiguous with an exterior outer surface of the display panel such that when a pointer is brought into proximity with the outer surface and into the sheet of illumination, the pointer reflects illumination of the sheet of illumination through the display panel;

a second illumination source configured to provide illumination to an opposite exterior outer surface of the display panel thereby to backlight said display panel;

a plurality of imaging devices arranged at spaced locations proximate to but spaced from the opposite exterior outer surface of the display panel, each imaging device having a field of view aimed at an associated sub-region of the opposite exterior outer surface of the display panel with the fields of view of neighboring imaging devices partially overlapping, the imaging devices configured to capture images including the illumination reflected by the pointer through the display panel;

processing structure in communication with the imaging devices, said processing structure configured to process the captured images to determine a location of the pointer relative to the exterior outer surface of the display panel, wherein the number of imaging devices and dimensions of the sub regions are selected to reduce dimensions of the interactive input system; and

a reflector positioned between the second illumination source and the imaging devices, the reflector having apertures corresponding to locations of the imaging devices and having a reflective surface facing the opposite exterior outer surface of the display panel, the reflective surface configured to reflect illumination impinging thereon to the opposite exterior outer surface of the display panel.

2. The interactive input system of claim 1 wherein the first illumination source is an infrared illumination source.

3. The interactive input system of claim 2 wherein the first illumination source comprises at least one infrared light emitting diode (LED).

4. The interactive input system of claim 3 wherein the at least one LED comprises a plurality of infrared LEDs.

5. The interactive input system of claim 4 wherein the infrared LEDs are positioned about the periphery of the exterior outer surface of the display panel.

6. The interactive input system of claim 5 wherein the infrared LEDs are positioned below a bezel extending about the periphery of the exterior outer surface of the display panel.

7. The interactive input system of claim 2 wherein the first illumination source further comprises a diffuser.

8. The interactive input system of claim 7 wherein the diffuser is one of an infrared diffuser and a holographic diffuser.

9. The interactive input system of claim 4 wherein the infrared LEDs are positioned along at least one side of the display panel and wherein a retro-reflective surface substantially normal to the exterior outer surface of the display panel extends along each side of the display panel opposite infrared LEDs.

10. The interactive input system of claim 2 wherein each imaging device is an infrared imaging device.

11. The interactive input system of claim 1 wherein the illumination provided by the second illumination source is visible light.

12. The interactive input system of claim 11 wherein the display panel is supported by support structure.

13. The interactive input system of claim 12 wherein the display panel is supported horizontally by the support structure.

14. The interactive input system of claim 11 wherein the display panel, second illumination source and the imaging devices are accommodated by a housing.

15. The interactive input system of claim 14 further comprising at least one diffuser positioned between the display panel and the imaging devices to diffuse illumination provided by the second illumination source to the opposite exterior outer surface of said display panel.

16. The interactive input system of claim 15 wherein the at least one diffuser is positioned adjacent the display panel or adjacent the imaging devices.

17. The interactive input system of claim 16 comprising a pair of diffusers, one of said diffusers being positioned adjacent the display panel and the other of said diffusers being positioned adjacent the second illumination source.

18. The interactive input system of claim 16 wherein the at least one diffuser when positioned adjacent the imaging devices comprises openings axially aligned with the imaging devices.

19. The interactive input system of claim 18 wherein the openings are covered by a layer of polymer-dispersed liquid crystal film.

20. The interactive input system of claim 19 wherein the polymer-dispersed liquid crystal film is in electrical communication with a power source.

21. A method comprising:

the interactive input system of claim 20 wherein each imaging device is spaced from the opposite exterior outer surface of the display panel by a distance between about 6 cm to 20 cm and wherein the plurality of imaging devices comprises between six imaging devices and twenty imaging devices.

22. The interactive input system of claim 14 wherein the processing structure is accommodated by the housing.

23. The interactive input system of claim 14 wherein the processing structure is positioned outside the housing.

24. A method comprising:

providing a sheet of illumination generally across and contiguous with an exterior outer surface of a liquid crystal display panel using a first illumination sources positioned about the periphery of the liquid crystal display panel;

providing illumination using a second illumination source to an opposite exterior outer surface of said display panel thereby to backlight said display panel;

capturing images of a pointer brought into proximity with the exterior outer surface and into the sheet of illumination, using a plurality of imaging devices proximate to but spaced from the opposite exterior outer surface of the display panel, each imaging device having a field of view aimed at an associated sub-region of the opposite exterior outer surface of the display panel with the fields of view of neighboring imaging devices partially overlapping, as a result of illumination from the sheet of illumination provided by the first illumination sources being reflected by the pointer towards the imaging devices;

reflecting illumination using a reflector positioned between the second illumination source and the imaging devices, the reflector having apertures corresponding to locations of the imaging devices and having a reflective surface facing the opposite exterior outer surface of the display panel, the reflective surface configured to reflect illumination impinging thereon to the opposite exterior outer surface of the display panel;

processing the captured images to determine a location of the pointer.

25. The method of claim 24 further comprising processing the captured images to determine at least one characteristic of the pointer.

26. The method of claim 25 wherein the at least one characteristic is at least one of a size of the pointer and a shape of the pointer.

27. The method of claim 25 wherein processing the captured images comprises performing at least one of blob detection and morphology operations.

28. The method of claim 24 wherein the capturing is performed only in an image capture mode.

29. The method of claim 28 wherein the first illumination source only provides illumination across and contiguous with the exterior outer surface of the display panel during the image capture mode.

30. An interactive input system comprising:

a liquid crystal display panel;

at least a first illumination source configured to provide a sheet of illumination across and contiguous with an exterior outer surface of the display panel;

at least a second illumination source configured to provide backlight illumination to an opposite exterior outer surface of the display panel;

a plurality of imaging devices arranged at spaced locations proximate to but spaced from the opposite exterior surface of the display panel, each imaging device having a field of view aimed at a respective portion of the opposite exterior outer surface with the fields of view of neighboring imaging devices partially overlapping, at least one of said imaging devices configured to capture images of at least one pointer brought into proximity with the exterior outer surface of the display panel and into the sheet of illumination as a result of illumination from the sheet of illumination provided by the first illumination source being reflected by the pointer through the display panel and towards the imaging devices;

processing structure configured to process the captured images to determine a location of the at least one pointer;

a reflector positioned between the second illumination source and the imaging devices, the reflector having apertures corresponding to locations of the imaging devices and having a reflective surface facing the opposite exterior outer surface of the display panel, the reflective surface configured to reflect illumination impinging thereon to the opposite exterior outer surface of the display panel; and

a housing at least accommodating the display panel, first illumination source, second illumination source, reflector and imaging devices, the number of imaging devices and the respective portion dimensions being selected to reduce the dimensions of said housing.

31. An interactive input system comprising:

a liquid crystal display panel;

a first illumination source configured to provide a sheet of illumination across and contiguous with an exterior outer surface of the display panel such that when a pointer is brought into proximity with the outer surface and into the sheet of illumination, the pointer reflects illumination of the sheet of illumination through the display panel;

a second illumination source configured to provide illumination to an opposite exterior outer surface of the display panel thereby to backlight said display panel;

a plurality of imaging devices arranged at spaced locations proximate to but spaced from the opposite exterior outer surface of the display panel, each imaging device having a field of view aimed at an associated sub-region of the opposite exterior outer surface of the display panel with the fields of view of neighboring imaging devices partially overlapping, the imaging devices configured to capture images including the illumination reflected by the pointer through the display panel; and

processing structure in communication with the imaging devices, said processing structure configured to process the captured images to determine a location of the pointer relative to the exterior outer surface of the display panel and to determine whether the pointer is a hover pointer or a touch pointer, wherein the number of imaging devices and dimensions of the sub regions are selected to reduce dimensions of the interactive input system, and wherein pointers above a threshold distance from the outer surface of the display panel and in the sheet of illumination are hover pointers and pointers at or below the threshold distance from the outer surface of the display panel are touch pointers.

32. The interactive input system of claim 31, wherein the threshold distance is approximately 2 mm.