Interactive video display system

• US 7,348,963 B2
• Filed: 08/05/2005
• Issued: 03/25/2008
• Est. Priority Date: 05/28/2002
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Reactrix Systems Incorporated
• Inventors: Bell, Matthew
• Primary Examiner(s): Shankar, Vijay
• Application Number:
  US11/197,941
• Publication Number:
  US 20060139314A1
• Time in Patent Office: 963 Days
• Field of Search: 345156-168, 345418-428, 463 30- 37
• US Class Current: 345/156
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/2145 the surface being also a display device, e.g. touch screens
• CPC Class Codes: A63F 13/219 for aiming at specific areas on the display, e.g. light-guns
• CPC Class Codes: A63F 13/285 Generating tactile feedback signals via the game input device, e.g. force feedback
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/52 involving aspects of the displayed game scene
• CPC Class Codes: A63F 13/61 using advertising information
• CPC Class Codes: A63F 13/65 automatically by game devices or servers from real world data, e.g. measurement in live racing competition
• CPC Class Codes: A63F 13/655 by importing photos, e.g. of the player
• CPC Class Codes: A63F 2300/1037 being specially adapted for converting control signals received from the game device into a haptic signal, e.g. using force feedback
• CPC Class Codes: A63F 2300/1075 using a touch screen
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/5506 using advertisements
• CPC Class Codes: A63F 2300/66 for rendering three dimensional images
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: A63F 2300/695 Imported photos, e.g. of the player
• CPC Class Codes: G01S 17/87 Combinations of systems using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 17/894 3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 : Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0425 : using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/0428 : by sensing at the edges of the touch surface the interruption of optical paths, e.g. an illumination plane, parallel to the touch surface which may be virtual sensing beam interruptions in a planar beam grid of an optical touch-screen G06F3/0421
• CPC Class Codes: G06T 2207/10021 : Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/30196 : Human being; Person
• CPC Class Codes: G06T 7/596 : from three or more stereo images
• CPC Class Codes: H04N 13/204 : using stereoscopic image cameras stereoscopic photography G03B35/00

Abstract

An interactive video display system. A display screen is for displaying a visual image for presentation to a user. A camera is for detecting an object in an interactive area located in front of the display screen, the camera operable to capture three-dimensional information about the object. A computer system is for directing the display screen to change the visual image in response to the object.

Claims

1. An interactive video display system comprising:

means for displaying video data to a user;

means for providing three-dimensional information about the user;

means for processing the three-dimensional information;

means for determining a hand gesture of a hand of the user without the hand of the user contacting the means for displaying video data, wherein the hand gesture includes the hand of the user moving toward the video display and through a threshold separating an interactive zone from a touch zone; and

means for generating visual effects on the means for displaying video data, wherein the visual effects are based on the three-dimensional information and the hand gesture.

2. A method for receiving a user selection of an object displayed on an interactive video display, the method comprising:

displaying the object using the interactive video display, the interactive video display being visible to the user;

tracking positions of a hand of the user using a video camera, the video camera configured to provide three-dimensional information about the positions;

selecting the object based on a hand gesture of the hand of the user and without the hand of the user contacting the interactive video display, wherein the hand gesture includes the hand of the user moving toward the object and through a threshold, the threshold separating an interactive zone from a touch zone.

3. The method of claim 2, wherein the interactive zone and the touch zone are volumetric regions at different distances from the interactive video display.

4. The method of claim 2, wherein the hand gesture further includes a grabbing motion.

5. The method of claim 2, further comprising releasing the selected object if the hand of the user is opened.

6. The method of claim 2, further comprising releasing the selected object if the hand of the user is moved out of the touch zone.

7. The method of claim 2, further comprising dragging the selected object in response to movement of the hand of the user.

8. The method of claim 2, further comprising:

tracking the user using the video camera; and

generating visual effects using the interactive video display, the visual effects based on the tracking of the user.

9. The method of claim 8, further comprising displaying a shape or outline of the user using the interactive video display.

10. The method of claim 2, further comprising adjusting a volumetric region of the interactive zone when the field of view of the video camera includes a plurality of users.

11. The method of claim 2, further comprising adjusting a volumetric region of the interactive zone to exclude one or more other users, wherein the one or more other users are beyond a closest user to interactive video display.

12. The method of claim 2, wherein the object is a virtual button.

13. The method of claim 12, wherein selecting the object based on the hand gesture includes the hand of the user moving over an image of the virtual button.

14. An interactive video display system comprising:

a video display;

a stereo camera configured to provide three-dimensional information about a user; and

a computer system coupled to the video display and the video camera, the computer system configured to;

execute instructions in memory to generate visual effects using the video display, the visual effects based on the three-dimensional information;

utilize a stereopsis algorithm to generate a depth map; and

determine different volumetric regions based on the depth map.

15. The interactive video display system of claim 14, wherein the visual effects are responsive to a movement of the user.

16. The interactive video display system of claim 14, wherein the different volumetric regions are further based on whether a field of view of the stereo camera includes a plurality of users.

17. The interactive video display system of claim 16, wherein one of the different volumetric regions is adjusted to exclude one or more of the plurality of users when the one or more of the plurality of users is beyond a closest user to the video display.

18. The interactive video display of claim 14, wherein the visual effects are further based on a hand gesture of the hand of the user and without the hand of the user contacting the video display, wherein the hand gesture includes the hand of the user moving toward the video display and through a threshold, the threshold separating an interactive zone from a touch zone.

19. The interactive video display system of claim 18, wherein the computer system is further configured to select an object on the video display when the hand of the user moves toward the object.

20. The interactive video display of claim 18, further comprising a virtual button displayed using the video display.

21. The interactive video display of claim 20, wherein the computer system is further configured to select the virtual button when the hand of the user is in a touch zone and the hand of the user moves toward the virtual button.