Interactive video window display system

• US 20050110964A1
• Filed: 09/20/2004
• Published: 05/26/2005
• Est. Priority Date: 05/28/2002
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Intellectual Ventures Holding 67 LLC (Intellectual Ventures LLC)
• Inventors: Bell, Matthew, Zide, Joshua, Shaughnessy, Helen, Gleckman, Philip
• Granted Patent: US 7,710,391 B2
• Time in Patent Office: Days
• US Class Current: 353/122
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/0428 by sensing at the edges of the touch surface the interruption of optical paths, e.g. an illumination plane, parallel to the touch surface which may be virtual sensing beam interruptions in a planar beam grid of an optical touch-screen G06F3/0421

Abstract

An interactive video window display system. A projector projects a visual image. A screen displays the visual image, wherein the projector projects the visual image onto a back side of the screen for presentation to a user on a front side of the screen, and wherein the screen is adjacent to a window. An illuminator illuminates an object on a front side of the window. A camera detects interaction of an illuminated object with the visual image, wherein the screen is at least partially transparent to light detectable by the camera, allowing the camera to detect the illuminated object through the screen. A computer system directs the projector to change the visual image in response to the interaction. The projector, the camera, the illuminator, and the computer system are located on the same side of the window.

Claims

1. An interactive video window display system comprising:

a projector for projecting a visual image;

a screen for displaying said visual image, wherein said projector projects said visual image onto a back side of said screen for presentation to a user on a front side of said screen, and wherein said screen is adjacent to a window;

an illuminator for illuminating an object on a front side of said window;

a camera for detecting interaction of an illuminated object with said visual image, wherein said screen is at least partially transparent to light detectable by said camera, allowing said camera to detect said illuminated object through said screen; and

a computer system for directing said projector to change said visual image in response to said interaction;

wherein said projector, said camera, said illuminator, and said computer system are located on the same side of said window.

2. The interactive video window display system as recited in claim 1 wherein said illuminator is an infrared illuminator for illuminating said object with infrared illumination, and wherein said camera is an infrared camera for detecting infrared illumination.

3. The interactive video window display system as recited in claim 1 wherein said projector is an off-axis projector.

4. The interactive video window display system as recited in claim 3 wherein said projector is an off-axis projector and a first side of said screen comprises a screen material adjacent to said screen that is substantially transparent when viewed from a substantially perpendicular angle and is substantially translucent when viewed at an oblique angle.

5. The interactive video window display system as recited in claim 4 wherein said screen material comprises a pleated material adjacent to said screen.

6. The interactive video window display system as recited in claim 4 wherein said screen material comprises a microlouver material adjacent to said screen.

7. The interactive video window display system as recited in claim 4 wherein said screen material comprises a transparent material with embedded sheetlike translucent material adjacent to said screen.

8. The interactive video window display system as recited in claim 4 wherein said screen material comprises a view control film adjacent to said screen.

9. The interactive video window display system as recited in claim 4, wherein said screen material is a wavelength-selective scattering material that substantially scatters light projected by said projector and does not substantially scatter light emitted by said illuminator.

10. The interactive video window display system as recited in claim 9, wherein said screen material causes Rayleigh scattering, said camera is a near-infrared camera, and said projector is a visible-light projector.

11. The interactive video window display system as recited in claim 1 in which said projector produces polarized light, and said screen material is a scattering polarizer that is oriented such that it substantially scatters the projector'"'"'s light, and said camera is sensitive to light of a polarization not substantially scattered by said scattering polarizer.

12. The interactive video window display system as recited in claim 11 wherein said projector is a liquid crystal display projector, and further comprising a color selective polarization rotator which reorients light from said liquid crystal display projector so that it is polarized in the same direction.

13. The interactive video window display system as recited in claim 11, further comprising a linear polarizer adjacent said front side of said screen to absorb light of the polarization not scattered by said screen.

14. The interactive video window display system as recited in claim 11 wherein said projector is a visible-light projector and said camera is a visible-light camera sensitive to light polarized in the transparent direction of said scattering polarizer.

15. The interactive video window display system as recited in claim 14, wherein projected output of said interactive video window display system incorporates a live color image of said user.

16. The interactive video window display system as recited in claim 1, wherein said camera is located below the bottom of said screen, allowing said interactive video window display system to ignore objects that are not near said window.

17. The interactive video window display system as recited in claim 11 further comprising a linear polarizer for polarizing light received at said camera at wavelengths to which said camera is sensitive, so as to allow said camera to ignore light scattered by said scattering polarizer.

18. The interactive video window display system as recited in claim 1 further comprising a micro-prism material adjacent to said screen.

19. The interactive video window display system as recited in claim 1 further comprising a mirror for redirecting said camera'"'"'s view of said screen, wherein said camera is pointed indirectly at said screen and said mirror redirects said camera'"'"'s view of said screen.

20. The interactive video window display system as recited in claim 1, wherein said illuminator projects illumination through said screen onto said object.

21. The interactive video window display system as recited in claim 1 wherein said illuminator is located next to said screen such that said illuminator does not project illumination through said screen.

22. The interactive video window display system as recited in claim 1 further comprising a Fresnel lens positioned adjacent to said screen to correct distortion of a view of said camera.

23. The interactive video window display system as recited in claim 1 wherein said screen comprises a material that can change from substantially translucent to substantially transparent, wherein said screen is substantially translucent when said projector is projecting an image, and is substantially transparent when said camera is detecting an object in front of said screen.

24. The interactive video window display system as recited in claim 1 wherein said screen comprises a material that is substantially translucent at a particular angle and substantially transparent at other angles, wherein said projector projects light onto said screen from said particular angle.

25. The interactive video window display system as recited in claim 1, wherein said illuminator is strobed in time with exposures of said camera.

26. The interactive video window display system as recited in claim 1 wherein an image of said camera is calibrated to said visual image such that said interaction caused by said object is matched to a physical position of said object proximate to said screen.

27. The interactive video window display system as recited in claim 1 wherein said self-contained interactive video display system is operable to determine information about a distance of said object from said screen.

28. The interactive video window display system as recited in claim 27 wherein said camera is a stereo camera.

29. The interactive video window display system as recited in claim 27 wherein said camera is a time-of-flight camera.

30. The interactive video window display system as recited in claim 29 wherein said time-of-flight camera is positioned such that said time-of-flight camera does not reflect back onto itself.

31. The interactive video window display system as recited in claim 1 wherein said self-contained interactive video display system provides touchscreen behavior functionality when said object is proximate said window.

32. The interactive video window display system as recited in claim 31 further comprising a transparent touchscreen adjacent said front side of said screen.

33. The interactive video window display system as recited in claim 31 further comprising an edge-lit transparent sheet adjacent said front side of said screen, and wherein said camera is operable to distinguish light created when said object comes in contact with said edge-lit transparent sheet.

34. The interactive video window display system as recited in claim 1 wherein said object is a body part of a human user.

35. The interactive video window display system as recited in claim 1 wherein said interactive video window display system may be tiled with a second interactive video window display system.

36. The interactive video window display system as recited in claim 1 further comprising a second camera, a first Fresnel lens for said camera, and a second Fresnel lens for said second camera, allowing spaces viewed by said camera and said second camera to be stitched together.

37. A method for presenting an interactive visual image using a interactive video window display system, said method comprising:

projecting a visual image on a screen, wherein said visual image is projected onto a back side of said screen for presentation to a user on a front side of said screen, and wherein said screen is adjacent to a window;

illuminating an object on said front side of said screen;

detecting interaction through said screen of said object with said visual image; and

changing said visual image in response to said interaction.

38. The method as recited in claim 37 wherein illuminating said object comprises illuminating said object with infrared illumination, and wherein said detecting said interaction comprises detecting infrared illumination projected onto said object.

39. The method as recited in claim 37 wherein said projecting said visual image further comprises projecting said visual image onto said screen via a mirror, wherein said projector is pointed indirectly at said screen such that said mirror redirects said visual image to said screen

40. The method as recited in claim 37, wherein said illuminating said object comprises projecting illumination through said screen onto said object.

41. The method as recited in claim 37 wherein said projecting is performed by an off-axis projector.

42. The method as recited in claim 41 wherein a first side of said screen comprises a screen material adjacent to said screen that is substantially transparent when viewed from a substantially perpendicular angle and is substantially translucent when viewed at an oblique angle.

43. The method as recited in claim 42 wherein said screen material comprises a pleated material adjacent to said screen.

44. The method as recited in claim 42 wherein said screen material comprises a microlouver material adjacent to said screen.

45. The method as recited in claim 42 wherein said screen material comprises a transparent material with embedded sheetlike translucent material adjacent to said screen.

46. The method as recited in claim 42 wherein said screen material comprises a view control film adjacent to said screen.

47. The method as recited in claim 42, wherein said screen material is a wavelength-selective scattering material that scatters light projected by said projecting and does not scatter light emitted by said illuminating.

48. The method as recited in claim 47, wherein said screen material includes Rayleigh scattering, said detecting is performed by a near-infrared camera, and said projecting is performed by a visible-light projector.

49. The method as recited in claim 37 in which said projecting produces polarized light, and said screen material is a scattering polarizer that substantially scatters said polarized light.

50. The method as recited in claim 49, further comprising absorbing light of the polarization not substantially scattered by said screen using a linear polarizer adjacent said front side of said screen.

51. The method as recited in claim 49 wherein said projector is a visible-light projector and said camera is a visible-light camera sensitive to light polarized in the transparent direction of the scattering polarizer.

52. The method as recited in claim 51, further comprising incorporating a live color image of said user into said visual image.

53. The method as recited in claim 37 further comprising a micro-prism material adjacent to said screen.

54. The method as recited in claim 37, wherein said illuminating said object is strobed in time with said detecting.

55. The method as recited in claim 37 further comprising calibrating an image captured at said detecting to said visual image such that said interaction caused by said object is matched to a physical position of said object proximate to said screen.

56. The method as recited in claim 37 further comprising determining information about a distance of said object from said screen.

57. The method as recited in claim 56 wherein said detecting is performed by a stereo camera.

58. The method as recited in claim 56 wherein said detecting is performed by a time-of-flight camera.

59. The method as recited in claim 58 further comprising positioning said time-of-flight camera such that the time-of-flight camera does not reflect back onto itself.

60. The method as recited in claim 37 wherein further comprising providing touchscreen functionality when said object is proximate said screen.

61. The method as recited in claim 60 further comprising providing a transparent touchscreen adjacent said front side of said screen.

62. The method as recited in claim 60 further comprising providing an edge-lit transparent sheet adjacent said front side of said screen, and wherein said detecting is operable to distinguish light created when said object comes in contact with said edge-lit transparent sheet.

63. The method as recited in claim 37 wherein said object is a body part of a human user.

64. The method as recited in claim 37 further comprising tiling said interactive video window display system with a second interactive video window display system.

65. An interactive video window display system comprising:

a projector for projecting a visual image;

a screen for displaying said visual image, wherein said projector projects said visual image onto a back side of said screen for presentation to a user on a front side of said screen, and wherein said screen is adjacent to a window;

an infrared illuminator for illuminating an object on a front side of said window;

a infrared camera for detecting interaction of an illuminated object with said visual image, wherein said screen is at least partially transparent to light detectable by said infrared camera, allowing said infrared camera to detect said illuminated object through said screen; and

a computer system for directing said projector to change said visual image in response to said interaction;

wherein said projector, said infrared camera, said infrared illuminator, and said computer system are located on the same side of said window.

66. The interactive video window display system as recited in claim 65 wherein said projector is an off-axis projector.

67. The interactive video window display system as recited in claim 66 wherein a first side of said screen comprises a screen material adjacent to said screen that is substantially transparent when viewed from a substantially perpendicular angle and is substantially translucent when viewed at an oblique angle.

68. The interactive video window display system as recited in claim 67 wherein said screen material comprises a pleated material adjacent to said screen.

69. The interactive video window display system as recited in claim 67 wherein said screen material comprises a microlouver material adjacent to said screen.

70. The interactive video window display system as recited in claim 69 wherein said screen material comprises a transparent material with embedded sheetlike translucent material adjacent to said screen.

71. The interactive video window display system as recited in claim 69 wherein said screen material comprises a view control film adjacent to said screen.

72. The interactive video window display system as recited in claim 69, wherein said screen material is a wavelength-selective scattering material that substantially scatters light projected by said projector and does not substantially scatter light emitted by said infrared illuminator.

73. The interactive video window display system as recited in claim 72, wherein said screen material causes Rayleigh scattering, said infrared camera is a near-infrared infrared camera, and said projector is a visible-light projector.

74. The interactive video window display system as recited in claim 65 in which said projector produces polarized light, and said screen material is a scattering polarizer that is oriented such that it substantially scatters the projector'"'"'s light, and said camera is sensitive to light of a polarization not substantially scattered by said scattering polarizer.

75. The interactive video window display system as recited in claim 74 wherein said projector is a liquid crystal display projector, and further comprising a color selective polarization rotator which reorients light from said liquid crystal display projector so that it is polarized in the same direction.

76. The interactive video window display system as recited in claim 74, further comprising a linear polarizer adjacent said front side of said screen to absorb light of the polarization not scattered by said screen.

77. The interactive video window display system as recited in claim 74 wherein said projector is a visible-light projector and said infrared camera is a visible-light infrared camera sensitive to light polarized in the transparent direction of the scattering polarizer.

78. The interactive video window display system as recited in claim 77, wherein projected output of said interactive video window display system incorporates a live color image of said user.

79. The interactive video window display system as recited in claim 65, wherein said infrared camera is located below the bottom of said screen, allowing said interactive video window display system to ignore objects that are not near said window.

80. The interactive video window display system as recited in claim 74 further comprising a linear polarizer for polarizing light received at said infrared camera at wavelengths to which said infrared camera is sensitive, so as to allow said infrared camera to ignore light scattered by said scattering polarizer.

81. The interactive video window display system as recited in claim 65 further comprising a micro-prism material adjacent to said screen.

82. The interactive video window display system as recited in claim 65 further comprising a mirror for redirecting said infrared camera'"'"'s view of said screen, wherein said infrared camera pointed indirectly at said screen and said mirror redirects said infrared camera'"'"'s view of said screen.

83. The interactive video window display system as recited in claim 65, wherein said infrared illuminator projects illumination through said screen onto said object.

84. The interactive video window display system as recited in claim 65 wherein said infrared illuminator is located next to said screen such that said infrared illuminator does not project illumination through said screen.

85. The interactive video window display system as recited in claim 65 further comprising a Fresnel lens positioned adjacent to said screen to correct distortion of a view of said camera.

86. The interactive video window display system as recited in claim 65 wherein said screen comprises a material that can change from substantially translucent to substantially transparent, is substantially translucent when said projector is projecting an image, and is substantially transparent when said infrared camera is detecting an object in front of said screen.

87. The interactive video window display system as recited in claim 65 wherein said screen comprises a material that is substantially translucent at a particular angle and substantially transparent at other angles, wherein said projector projects light onto said screen from said particular angle.

88. The interactive video window display system as recited in claim 65, wherein said infrared illuminator is strobed in time with exposures of said infrared camera.

89. The interactive video window display system as recited in claim 65 wherein an image of said infrared camera is calibrated to said visual image such that said interaction caused by said object is matched to a physical position of said object proximate to said screen.

90. The interactive video window display system as recited in claim 65 wherein said self-contained interactive video display system is operable to determine information about a distance of said object from said screen.

91. The interactive video window display system as recited in claim 90 wherein said infrared camera is a stereo infrared camera.

92. The interactive video window display system as recited in claim 90 wherein said infrared camera is a time-of-flight infrared camera.

93. The interactive video window display system as recited in claim 92 wherein said time-of-flight infrared camera is positioned such that said time-of-flight infrared camera does not reflect back onto itself.

94. The interactive video window display system as recited in claim 65 wherein said self-contained interactive video display system provides touchscreen behavior functionality when said object is proximate said screen.

95. The interactive video window display system as recited in claim 94 further comprising a transparent touchscreen adjacent said front side of said screen.

96. The interactive video window display system as recited in claim 94 further comprising an edge-lit transparent sheet adjacent said front side of said screen, and wherein said infrared camera is operable to distinguish light created when said object comes in contact with said edge-lit transparent sheet.

97. The interactive video window display system as recited in claim 65 wherein said object is a body part of a human user.

98. The interactive video window display system as recited in claim 65 wherein said interactive video window display system may be tiled with a second interactive video window display system.

99. The interactive video window display system as recited in claim 65 further comprising a second camera, a first Fresnel lens for said camera, and a second Fresnel lens for said second camera, allowing spaces viewed by said camera and said second camera to be stitched together.