Self-contained interactive video display system

• US 20050122308A1
• Filed: 09/20/2004
• Published: 06/09/2005
• Est. Priority Date: 05/28/2002
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Bell, Matthew, Zide, Joshua, Shaughnessy, Helen, Gleckman, Philip
• Application Number:
  US10/946,084
• Publication Number:
  US 20050122308A1
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 2203/04109 FTIR in optical digitiser, i.e. touch detection by frustrating the total internal reflection within an optical waveguide due to changes of optical properties or deformation at the touch location
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/0428 by sensing at the edges of the touch surface the interruption of optical paths, e.g. an illumination plane, parallel to the touch surface which may be virtual sensing beam interruptions in a planar beam grid of an optical touch-screen G06F3/0421
• CPC Class Codes: G06V 10/145 Illumination specially adapted for pattern recognition, e.g. using gratings
• CPC Class Codes: G06V 40/107 Static hand or arm

Abstract

A self-contained interactive video display system. A flat-panel display screen displays a visual image for presentation to a user on a front side of the flat-panel display screen. A first illuminator illuminates the flat-panel display screen with visible light. A second illuminator illuminates an object. A camera detects interaction of an illuminated object with the visual image, wherein the camera is operable to view the object through the flat-panel display screen. A computer system directs the projector to change the visual image in response to the interaction.

Claims

1. A self-contained interactive video display system comprising:

a flat-panel display screen for displaying a visual image for presentation to a user on a front side of said flat-panel display screen;

a first illuminator for illuminating said flat-panel display screen with visible light;

a second illuminator for illuminating an object;

a camera for detecting interaction of an illuminated object with said visual image, wherein said camera is operable to view said object through said flat-panel display screen; and

a computer system for directing said flat-panel display screen to change said visual image in response to said interaction.

2. The self-contained interactive video display system as recited in claim 1 wherein said flat-panel display screen is a liquid crystal display (LCD) panel.

3. The self-contained interactive video display system as recited in claim 1 wherein said second illuminator is an infrared illuminator for illuminating said object with infrared illumination, and wherein said camera is an infrared camera for detecting infrared illumination.

4. The self-contained interactive video display system as recited in claim 1 further comprising a diffusing screen for diffusing light of said first illuminator.

5. The self-contained interactive video display system as recited in claim 1 wherein said camera, said first illuminator, said second illuminator, and said computer system are comprised within an enclosure, and wherein one side of said enclosure comprises said flat-panel display screen.

6. The self-contained interactive video display system as recited in claim 1, wherein said second illuminator projects illumination through said flat-panel display screen onto said object.

7. The self-contained interactive video display system as recited in claim 6, wherein said second illuminator is positioned so as to reduce potential for glare effects on said camera.

8. The self-contained interactive video display system as recited in claim 1 wherein said second illuminator is located next to said flat-panel display screen such that said second illuminator does not project illumination through said flat-panel display screen.

9. The self-contained interactive video display system as recited in claim 1 comprising a plurality of said second illuminators, wherein said second illuminators are located next to said flat-panel display screen and do not project illumination through said flat-panel display screen and are also located behind said screen.

10. The self-contained interactive video display system as recited in claim 1 wherein said camera is located behind said flat-panel display screen and pointed toward said screen, allowing said camera to view the area on and in front of the screen.

11. The self-contained interactive video display system as recited in claim 1 further comprising a series of mirror strips positioned at a distance from said screen to correct distortion of a view of said camera.

12. The self-contained interactive video display system as recited in claim 1 further comprising a Fresnel lens positioned adjacent to said screen to correct distortion of a view of said camera.

13. The self-contained interactive video display system as recited in claim 1 further comprising a wavelength-based diffuser positioned adjacent to said flat-panel display screen.

14. The self-contained interactive video display system as recited in claim 13, where said wavelength-based diffuser is substantially transparent to infrared light and substantially translucent to visible light.

15. The self-contained interactive video display system as recited in claim 13, where said wavelength-based diffuser is a material that causes Rayleigh scattering.

16. The self-contained interactive video display system as recited in claim 1 further comprising a diffuser positioned adjacent to said flat-panel display screen, wherein said diffuser is substantially translucent to light passing through said diffuser at an oblique angle and substantially transparent to light passing through said diffuser at a substantially perpendicular angle, wherein said first illuminator is placed at an oblique angle to said diffuser.

17. The self-contained interactive video display system as recited in claim 1 further comprising a scattering polarizer positioned adjacent to said flat-panel display screen and for scattering light from said first illuminator, and wherein said camera is sensitive to light of a polarization not scattered by said scattering polarizer.

18. The self-contained interactive video display system as recited in claim 17, wherein said flat-panel display is a liquid crystal display panel, and wherein said scattering polarizer is oriented such that light polarized in a direction for which said scattering polarizer scatters light passes through said liquid crystal display panel and light polarized in a direction for which said scattering polarizer does not scatter light is absorbed by said liquid crystal display panel.

19. The self-contained interactive video display system as recited in claim 17, further comprising a linear polarizer for polarizing light received at said camera at wavelengths to which said camera is sensitive, so as to allow said camera to ignore light scattered by said scattering polarizer.

20. The self-contained interactive video display system as recited in claim 1 further comprising a diffusing material that can change from substantially translucent to substantially transparent, wherein said diffusing material is substantially translucent when said first illuminator is visible to a human user on said front side of said flat-panel display screen, and is substantially transparent when said camera is detecting objects in front of said flat-panel display screen, wherein said diffusing material is placed behind said flat-panel display screen.

21. The self-contained interactive video display system as recited in claim 1, wherein said second illuminator is strobed in time with exposures of said camera.

22. The self-contained interactive video display system as recited in claim 1 wherein an image of said camera is calibrated to said visual image such that said interaction caused by said object is matched to a physical position of said object proximate to said the screen.

23. The self-contained interactive video display system as recited in claim 1 wherein said self-contained interactive video display system is operable to determine information about the distance of said object from the said screen.

24. The self-contained interactive video display system as recited in claim 23 wherein said camera is a stereo camera.

25. The self-contained interactive video display system as recited in claim 23 wherein said camera is a time-of-flight camera.

26. The self-contained interactive video display system as recited in claim 25 wherein said time-of-flight camera is positioned such that said time-of-flight camera does not reflect back onto itself.

27. The self-contained interactive video display system as recited in claim 1 wherein said self-contained interactive video display system provides touchscreen functionality when said object is proximate said screen.

28. The self-contained interactive video display system as recited in claim 27 further comprising a transparent touchscreen adjacent said front side of said screen.

29. The self-contained interactive video display system as recited in claim 27 further comprising an edge-lit transparent sheet adjacent said front side of said screen, and wherein said camera is operable to distinguish light created when said object comes in contact with said edge-lit transparent sheet.

30. The self-contained interactive video display system as recited in claim 1 wherein said object is a body part of a human user.

31. The self-contained interactive video display system as recited in claim 1, wherein said camera and said second illuminator comprise a time-of-flight camera.

32. The self-contained interactive video display system as recited in claim 1, wherein a plurality of time-of-flight cameras are placed in a manner so as to provide complete coverage of the area in front of the display and angled so as to prevent specular reflection into said time-of-flight cameras due to said screen.

33. A method for presenting an interactive visual image using a self-contained interactive video display system, said method comprising:

displaying a visual image on a flat-panel display screen, wherein said visual image is for presentation to a user on a front side of said flat-panel display screen;

illuminating a back side of said flat-panel display screen with visible light;

illuminating an object on said front side of said flat-panel display screen;

detecting interaction of said object with said visual image through said flat-panel display screen; and

changing said visual image in response to said interaction.

34. The method as recited in claim 33 wherein illuminating said object comprises illuminating said object with infrared illumination, and wherein said detecting said interaction comprises detecting infrared illumination projected onto said object.

35. The method as recited in claim 33 wherein said self-contained interactive video display system is comprised within an enclosure, and wherein one side of said enclosure comprises said flat-panel display screen.

36. The method as recited in claim 33, wherein said illuminating said object comprises projecting illumination through said flat-panel display screen onto said object.

37. The method as recited in claim 33 further comprising correcting distortion using a Fresnel lens positioned adjacent to said screen.

38. The method as recited in claim 33 further comprising an infrared-transparent visible light-translucent screen positioned adjacent to said flat-panel display screen.

39. The method as recited in claim 33 further comprising a transparent screen having a physical texture positioned adjacent to said flat-panel display screen and for scattering light from said illuminating said back side of said flat-panel display screen with visible light.

40. The method as recited in claim 33 further comprising a transparent screen of Rayleigh scattering material positioned adjacent to said flat-panel display screen and for scattering light from said illuminating said back side of said flat-panel display screen with visible light.

41. The method as recited in claim 33 further comprising a scattering polarizer positioned adjacent to said flat-panel display screen and for scattering light from said illuminating said back side of said flat-panel display screen with visible light.

42. The method as recited in claim 33 wherein said self-contained interactive video display system further comprises a wavelength-based diffuser positioned adjacent to said flat-panel display screen.

43. The method as recited in claim 33 wherein said self-contained interactive video display system further comprises a diffusing material that can change from substantially translucent to substantially transparent, wherein said diffusing material is substantially translucent during said illuminating said back side of said flat-panel display screen with visible light, and is substantially transparent during said detecting interaction of said object with said visual image through said flat-panel display screen, wherein said diffusing material is placed behind said flat-panel display screen.

44. The method as recited in claim 33, wherein said illuminating said object is strobed in time with said detecting.

45. The method as recited in claim 33 further comprising calibrating an image captured at said detecting to said visual image such that said interaction caused by said object is matched to a physical position of said object proximate to said screen.

46. The method as recited in claim 33 further comprising determining information about a distance of said object from said screen.

47. The method as recited in claim 46 wherein said detecting is performed by a stereo camera.

48. The method as recited in claim 46 wherein detecting is performed by a time-of-flight camera.

49. The method as recited in claim 48 further comprising positioning said time-of-flight camera such that the time-of-flight camera does not reflect back onto itself.

50. The method as recited in claim 33 wherein further comprising providing touchscreen functionality when said object is proximate said screen.

51. The method as recited in claim 50 further comprising providing a transparent touchscreen adjacent said front side of said screen.

52. The method as recited in claim 50 further comprising providing an edge-lit transparent sheet adjacent said front side of said screen, and wherein said detecting is operable to distinguish light created when said object comes in contact with said edge-lit transparent sheet.

53. The method as recited in claim 33 wherein said object is a body part of a human user.

54. A self-contained interactive video display system comprising:

a liquid crystal display screen for displaying a visual image for presentation to a user on a front side of said liquid crystal display screen;

a visible light illuminator for illuminating said liquid crystal display screen with visible light;

a infrared illuminator for illuminating an object;

a infrared camera for detecting interaction of an illuminated object with said visual image, wherein said infrared camera is operable to view said object through said liquid crystal display screen; and

a computer system for directing said liquid crystal display screen to change said visual image in response to said interaction;

wherein said infrared camera, said visible light illuminator, said infrared illuminator, and said computer system are comprised within an enclosure, and wherein one side of said enclosure comprises said liquid crystal display screen.

55. The self-contained interactive video display system as recited in claim 54 further comprising a diffusing screen for diffusing light of said visible light illuminator.

56. The self-contained interactive video display system as recited in claim 54, wherein said infrared illuminator projects illumination through said liquid crystal display screen onto said object.

57. The self-contained interactive video display system as recited in claim 56, wherein said infrared illuminator is positioned so as to reduce potential for glare effects on said infrared camera.

58. The self-contained interactive video display system as recited in claim 54 wherein said infrared illuminator is located next to said liquid crystal display screen such that said infrared illuminator does not project illumination through said liquid crystal display screen.

59. The self-contained interactive video display system as recited in claim 54 comprising a plurality of said infrared illuminators, wherein said infrared illuminators are located next to said flat-panel display screen and do not project illumination through said flat-panel display screen and are also located behind said screen.

60. The self-contained interactive video display system as recited in claim 54 wherein said infrared camera is located behind said liquid crystal display screen and pointed toward said screen, allowing said infrared camera to view the area on and in front of the screen.

61. The self-contained interactive video display system as recited in claim 54 further comprising a series of mirror strips positioned at a distance from said screen to correct distortion of a view of said infrared camera.

62. The self-contained interactive video display system as recited in claim 54 further comprising a Fresnel lens positioned adjacent to said screen to correct distortion of a view of said infrared camera.

63. The self-contained interactive video display system as recited in claim 54 further comprising a wavelength-based diffuser positioned adjacent to said liquid crystal display screen.

64. The self-contained interactive video display system as recited in claim 63, where said wavelength-based diffuser is substantially transparent to infrared light and substantially translucent to visible light.

65. The self-contained interactive video display system as recited in claim 63, where said wavelength-based diffuser is a material that causes Rayleigh scattering.

66. The self-contained interactive video display system as recited in claim 54 further comprising a diffuser positioned adjacent to said liquid crystal display screen, wherein said diffuser is substantially translucent to light passing through said diffuser at an oblique angle and substantially transparent to light passing through said diffuser at a substantially perpendicular angle, wherein said visible light illuminator is placed at an oblique angle to said diffuser.

67. The self-contained interactive video display system as recited in claim 54 further comprising a scattering polarizer positioned adjacent to said liquid crystal display screen and for scattering light from said visible light illuminator, and wherein said infrared camera is sensitive to light of a polarization not scattered by said scattering polarizer.

68. The self-contained interactive video display system as recited in claim 67, wherein said liquid crystal display is an liquid crystal display panel, and wherein said scattering polarizer is oriented such that light polarized in a direction for which said scattering polarizer scatters light passes through said liquid crystal display panel and light polarized in a direction for which said scattering polarizer does not scatter light is absorbed by said scattering polarizer.

69. The self-contained interactive video display system as recited in claim 67, further comprising a linear polarizer for polarizing light received at said infrared camera at wavelengths to which said infrared camera is sensitive, so as to allow said infrared camera to ignore light scattered by said scattering polarizer.

70. The self-contained interactive video display system as recited in claim 54 further comprising a diffusing material that can change from substantially translucent to substantially transparent, wherein said diffusing material is substantially translucent when said first illuminator is visible to a human user on said front side of said flat-panel display screen, and is substantially transparent when said infrared camera is detecting objects in front of said liquid crystal display screen, wherein said diffusing material is placed behind said liquid crystal display screen.

71. The self-contained interactive video display system as recited in claim 54, wherein said infrared illuminator is strobed in time with exposures of said infrared camera.

72. The self-contained interactive video display system as recited in claim 54 wherein an image of said infrared camera'"'"'s image is calibrated to said visual image such that said interaction caused by said object is matched to a physical position of said object proximate to said the screen.

73. The self-contained interactive video display system as recited in claim 54 wherein said self-contained interactive video display system is operable to determine information about the distance of said object from the said screen.

74. The self-contained interactive video display system as recited in claim 73 wherein said infrared camera is a stereo infrared camera.

75. The self-contained interactive video display system as recited in claim 73 wherein said infrared camera is a time-of-flight infrared camera.

76. The self-contained interactive video display system as recited in claim 75 wherein said time-of-flight infrared camera is positioned such that said time-of-flight infrared camera does not reflect back onto itself.

77. The self-contained interactive video display system as recited in claim 54 wherein said self-contained interactive video display system provides touchscreen behavior functionality when said object is proximate said screen.

78. The self-contained interactive video display system as recited in claim 77 further comprising a transparent touchscreen adjacent said front side of said screen.

79. The self-contained interactive video display system as recited in claim 77 further comprising an edge-lit transparent sheet adjacent said front side of said screen, and wherein said infrared camera is operable to distinguish light created when said object comes in contact with said edge-lit transparent sheet.

80. The self-contained interactive video display system as recited in claim 54 wherein said object is a body part of a human user.

81. The self-contained interactive video display system as recited in claim 54, wherein said infrared camera and said infrared illuminator comprise a time-of-flight infrared camera.

82. The self-contained interactive video display system as recited in claim 81, wherein a plurality of time-of-flight infrared cameras are placed in a manner so as to provide complete coverage of the area in front of the display and angled so as to prevent specular reflection into said time-of-flight infrared cameras due to said screen.