Interactive video based games using objects sensed by TV cameras

• US 20060033713A1
• Filed: 07/22/2005
• Published: 02/16/2006
• Est. Priority Date: 08/22/1997
• Status: Active Grant

• Current Assignee: Motion Games LLC
• Original Assignee: Timothy R. Pryor
• Inventors: Pryor, Timothy R.
• Granted Patent: US 7,843,429 B2
• Time in Patent Office: Days
• US Class Current: 345/158
• CPC Class Codes: A63F 2300/1012 involving biosensors worn by the player, e.g. for measuring heart beat, limb activity
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: G06F 17/00 Digital computing or data processing equipment or methods, specially adapted for specific functions information retrieval, database structures or file system structures therefor G06F16/00
• CPC Class Codes: G06F 2203/0331 Finger worn pointing device
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/042 by opto-electronic means
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

A method and apparatus for interactive TV camera based games in which position or orientation of points on a player or of an object held by a player are determined and used to control a video display. Both single camera and stereo camera pair based embodiments are disclosed, preferably using stereo photogrammetry where multi-degree of freedom information is desired. Large video displays, preferably life-size may be used where utmost realism of the game experience is desired.

Claims

1. A video display based game comprising:

a game object held in the hand of a player of said game;

at least one TV camera to obtain images of said game object;

a computer to analyze images of said at least one TV camera and determine information including a position of said game object; and

a display driven by said computer to display video images relating to said game which are varied as a result of said determination.

2. Apparatus according to claim 1, wherein said determined information further includes the orientation of said object.

3. Apparatus according to claim 2, wherein said orientation is determined in more than one plane.

4. Apparatus according to claim 1, wherein said determined information further includes data concerning the position of one or more points on said player.

5. Apparatus according to claim 1, wherein said information further includes data relating to the movement of said object.

6. Apparatus according to claim 1, wherein an image from a single camera is analyzed by said computer.

7. Apparatus according to claim 1, wherein position of a plurality of points on said object are determined.

8. Apparatus according to claim 1, wherein images from a stereo pair of cameras are analyzed by said computer.

9. Apparatus according to claim 1, wherein stereo photogrammetry is used to determine the 3D coordinates of said object.

10. Apparatus according to claim 1, wherein stereo photogrammetry is used to determine at least 4 degrees of freedom of said object.

11. Apparatus according to claim 1, wherein a virtual image displayed on said display reacts to said determined information.

12. Apparatus according to claim 1, wherein said display is larger than 40 inches measured diagonally.

13. Apparatus according to claim 1, wherein said display is substantially life size.

14. A video display based game comprising:

a game object held in the hand of a player of said game;

at least one TV camera to obtain images of said game object;

a computer to analyze images of said at least one TV camera and to determine information including an orientation of said game object; and

a display driven by said computer to display video images relating to said game which are varied as a result of said determination.

15. Apparatus according to claim 14, wherein said determined information further includes the position of said object.

16. Apparatus according to claim 14, wherein an image from a single camera is analyzed by said computer.

17. A TV camera based video gaming method, comprising the steps of:

providing a computer used to analyze TV camera images and generate game images;

providing a video display on which game related images generated by said computer are displayed;

using at least one TV camera, obtaining images of one or more objects freely held in the hand of one or more players of said game;

analyzing said TV images using said computer to determine information concerning the position, orientation or movement of said one or more objects; and

using said information, changing the video display of said game.

18. A method according to claim 17, wherein said changed video display is used to indicate a new game situation calling for a response by at least one of said players.

19. A method according to claim 17, wherein said object is a toy such as a hand puppet, airplane, or weapon.

20. A method according to claim 17, including the further step of providing an audio signal generated by said computer.