Computer vision based touch screen

• US 20080150913A1
• Filed: 10/30/2007
• Published: 06/26/2008
• Est. Priority Date: 05/28/2002
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Bell, Matthew, Zide, Joshua, Shaughnessy, Helen, Gleckman, Philip
• Granted Patent: US 8,035,624 B2
• Time in Patent Office: Days
• US Class Current: 345/175
• CPC Class Codes: A63F 2300/1068 being specially adapted to detect the point of contact of the player on a surface, e.g. floor mat, touch pad
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/0428 by sensing at the edges of the touch surface the interruption of optical paths, e.g. an illumination plane, parallel to the touch surface which may be virtual sensing beam interruptions in a planar beam grid of an optical touch-screen G06F3/0421

Abstract

A self-contained interactive video display system. A projector projects a visual image onto a screen for displaying the visual image, wherein the projector projects the visual image onto a back side of the screen for presentation to a user on a front side of the screen. An illuminator illuminates an object near the front side of the screen. A camera detects interaction of an illuminated object with the visual image, wherein the screen is at least partially transparent to light detectable to the camera, allowing the camera to detect the illuminated object through the screen. A computer system directs the projector to change the visual image in response to the interaction.

Claims

1-79. -79. (canceled)

80. An interactive video display system, comprising:

a display screen configured to display an image, the display screen having a front side and a back side;

a transparent panel located in front of the display screen, the panel having an interior;

an illuminator located proximate to the edge of the transparent panel, the illuminator configured to emit a beam of non-visible wavelength light into the edge of the transparent panel;

a camera located behind and capable of imaging the transparent panel through the display screen, the camera configured to detect non-visible wavelength light escaping from the transparent panel when an object contacts a front side of the transparent panel; and

a computing device configured to;

analyze information generated by the camera in response to the detection of non-visible wavelength light escaping from the transparent panel, and

generate a responsive image for display on the display screen, the responsive image based on the analyzed information.

81. The touch screen apparatus of claim 80, wherein the beam of non-visible wavelength light has a high angle of incidence relative to the transparent panel.

82. The touch screen apparatus of claim 81, wherein the beam of non-visible wavelength light repeatedly reflects off the front and back side of the interior of the transparent panel as a result of the high angle of incidence.

83. The touch screen apparatus of claim 80, wherein the non-visible wavelength light includes infrared light.

84. The touch screen apparatus of claim 80, further comprising a projector, the projector configured to project the responsive image onto the back side of the display screen for presentation to a user on the front side of the display screen.

85. The touch screen apparatus of claim 84, wherein the projector generates polarized light.

86. The touch screen apparatus of claim 85, wherein the display screen includes material translucent to the polarized light of the projector.

87. The touch screen apparatus of claim 85, wherein the display screen includes material transparent to light of a polarization opposite that of the polarized light generated by the projector.

88. The touch screen apparatus of claim 80, wherein the display screen includes a material that changes from substantially translucent to substantially transparent based on a time cycle.

89. The touch screen apparatus of claim 88, wherein the display screen is substantially translucent when a projector is projecting the responsive image onto the back side of the display screen.

90. The touch screen apparatus of claim 88, wherein the display screen is substantially transparent when the camera is detecting objects contacting the front side of the display screen.

91. The touch screen apparatus of claim 84, wherein the display screen includes a material that is substantially translucent at a first angle and substantially transparent at a second angle.

92. The touch screen apparatus of claim 91, wherein the projector projects the responsive image onto the back side of the display from the first angle.

93. The touch screen apparatus of claim 80, wherein the display screen includes a liquid crystal display and a backlight.

94. The touch screen apparatus of claim 93, wherein the liquid crystal display includes a material translucent to the polarization of visible light that can pass through the liquid crystal display.

95. The touch screen apparatus of claim 94, wherein the material is transparent to infrared light of an opposite polarization of the visible light that can pass through the liquid crystal display.

96. The touch screen apparatus of claim 80, wherein the camera is sensitive to infrared light.

97. The touch screen apparatus of claim 96, wherein the display screen includes a scattering polarizer.

99. The touch screen apparatus of claim 96, wherein the display screen includes a material that substantially scatters light of wavelengths emitted by a projector but not those detectable by the camera.

100. The touch screen apparatus of claim 80, further comprising a second illuminator, the second illuminator configured to emit a beam of non-visible wavelength light onto an object proximate to but not in contact with the transparent panel, and wherein the camera is further configured to detect the object proximate to but not in contact with the transparent panel.

106. The touch screen apparatus of claim 80, further comprising a second illuminator, the second illuminator configured to emit a beam of non-visible wavelength light onto an object proximate to but not in contact with the transparent panel and wherein a second camera is configured to detect the object proximate to but not in contact with the transparent panel.

98. The touch screen apparatus of 96, wherein the display screen includes a material whereby the degree of scattering of light is greater for visible light than it is for infrared light.

101. A method for interacting with a video display, the method comprising:

displaying an image on a display screen having a front side and a back side;

emitting a beam of non-visible wavelength light into the edge of a transparent panel located in front of the display screen;

detecting non-visible wavelength light escaping from the transparent panel when an object contacts a front side of the transparent panel, wherein detection of the non-visible light occurs at a camera imaging the transparent panel through the display screen;

analyzing information generated by the camera in response to the detection of non-visible wavelength light escaping from the transparent panel; and

generating a responsive image for display on the display screen, the responsive image based on the analyzed information.

102. The method of claim 101, wherein the non-visible wavelength-light includes infrared light.

103. The method of claim 101, wherein the responsive image is projected onto the back side of the display screen for presentation to a user on the front side of the display screen.

104. The method of claim 101, further comprising:

emitting a beam of non-visible wavelength light onto an object proximate to, but not in contact with the transparent panel; and

detecting the object proximate to but not in contact with the transparent panel.

105. An interactive video display system, comprising:

means for displaying an image, the means for displaying having a front side and a back side;

transparent means located in front of the means for displaying;

means for emitting a beam of non-visible wavelength light into the edge of the transparent means, the means for emitting located proximate to the edge of the transparent means;

detection means located behind and capable of imaging the transparent means through the means for displaying, the detection means configured to detect non-visible wavelength light escaping from the transparent means when an object contacts a front side of the transparent means; and

computing means configured to;

analyze information generated by the detection means in response to the detection of non-visible wavelength light escaping from the transparent means, and

generate a responsive image for display on the means for displaying, the responsive image based on the analyzed information.