Interactive video based games using objects sensed by TV cameras

• US 7,843,429 B2
• Filed: 07/22/2005
• Issued: 11/30/2010
• Est. Priority Date: 08/22/1997
• Status: Expired due to Fees

• Current Assignee: Motion Games LLC
• Original Assignee: Timothy R. Pryor
• Inventors: Pryor, Timothy R.
• Primary Examiner(s): Abdulselam, Abbas I
• Application Number:
  US11/186,898
• Publication Number:
  US 20060033713A1
• Time in Patent Office: 1,957 Days
• Field of Search: 345/156, 345/158, 345/419, 345/204, 345/157, 463/30, 463/31, 463/32, 463/37
• US Class Current: 345/158
• CPC Class Codes: A63F 2300/1012 involving biosensors worn by the player, e.g. for measuring heart beat, limb activity
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: G06F 17/00 Digital computing or data processing equipment or methods, specially adapted for specific functions information retrieval, database structures or file system structures therefor G06F16/00
• CPC Class Codes: G06F 2203/0331 Finger worn pointing device
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/042 by opto-electronic means
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

A method and apparatus for interactive TV camera based games in which position or orientation of points on a player or of an object held by a player are determined and used to control a video display. Both single camera and stereo camera pair based embodiments are disclosed, preferably using stereo photogrammetry where multi-degree of freedom information is desired. Large video displays, preferably life-size may be used where utmost realism of the game experience is desired.

Claims

1. A video display based game having changing video game images displayed comprising:

an object having an orientation and a position, said object being adapted to be freely held in a hand of a player of a game, and which said object is moved in space in both orientation and position in at least one axis in order to play the game;

at least one TV camera which obtains image data at least 30 times a second concerning said freely held object as the orientation and position in at least one axis of the freely held object is moved by the player from one orientation and position to another orientation and position in order to play the game;

a computer which analyzes the image data of said freely held object obtained at least 30 times a second by said at least one TV camera and which determines information about said freely held object including the one and another orientation and position in at least one axis of the freely held object as the freely held object is moved by the player in order to play the game; and

a computer controlled display which displays the changing video game images relating to the game, which said changing video game images are varied as a result of said determined information from the image data of said freely held object obtained at least 30 times a second by said at least one TV camera about the one and another orientation and position in at least one axis of the freely held object as the freely held object is moved in order to play the game.

2. Apparatus according to claim 1, wherein said determined information further includes the three dimensional position of said object.

3. Apparatus according to claim 1, wherein said orientation of the object is determined in more than one plane.

4. Apparatus according to claim 1, wherein said determined information further includes data concerning the position of one or more points on a player.

5. Apparatus according to claim 1, wherein said information further includes data relating to the movement of said object.

6. Apparatus according to claim 1, wherein an image from a single camera is analyzed by said computer.

7. Apparatus according to claim 1, wherein position of a plurality of points on said object are determined.

8. Apparatus according to claim 1, wherein images from a stereo pair of cameras are analyzed by said computer.

9. Apparatus according to claim 1, wherein stereo photogrammetry is used to determine the 3D coordinates of said object.

10. Apparatus according to claim 1, wherein stereo photogrammetry is used to determine at least 4 degrees of freedom of said object.

11. Apparatus according to claim 1, further including an interrupt member which provides a signal to said computer indicative of a player command.

12. Apparatus according to claim 1, wherein said display is substantially life size.

13. Apparatus according to claim 1, wherein said computer also controls said computer controlled display.

14. Apparatus according to claim 1, wherein said computer controlled display is controlled by a second computer different from said first mentioned computer, and further including a transmission means for transmitting said determined information from the first mentioned computer to the second computer.

15. A video display based gaming method in which changing video game images are displayed, comprising the steps of:

displaying, using a computer, changing video game images;

providing an object having an orientation and a position, the object being adapted to be freely held in a hand of a player of a game;

moving of the freely held object in both orientation and position in at least one axis by the player in the course of playing the game;

obtaining image data at least 30 times a second concerning the freely held object using at least one TV camera as the orientation and position in at least one axis of the freely held object is moved by the player from one orientation and position to another orientation and position in the course of playing the game;

analyzing, using a computer, the TV camera image data obtained at least 30 times a second concerning the freely held object as the freely object is moved in the course of playing the game, and determining information about the freely held object including the one and another orientations and positions in at least one axis of the freely held object; and

changing, with said displaying computer, said displayed video game images wherein said images are varied as a result of said determined information from said analyzing step of the TV camera image data obtained at least 30 times a second about the one and another orientations and positions in at least one axis of the freely held object as the freely held object is moved in the course of playing the game.

16. A method according to claim 15, wherein said determined information further includes the three dimensional position of said object.

17. A method according to claim 15, wherein said obtaining data step also determines data relating to a movement of the object.

18. A method according to claim 15, wherein said object is a shaped to resemble an object displayed in a video game image displayed by the displaying computer.

19. A method according to claim 15, including the further step of providing an audio signal generated by said displaying computer to the player based on said determined information.

20. A method according to claim 15, including the further step of providing a signal indicative of a user command to said displaying computer.

21. A method according to claim 15, wherein said displayed images relate at least in part to determined information transmitted over the internet.

22. A method according to claim 15, including the further step of moving said object in response to said varied game image.

23. A method according to claim 15, further including the step of transmitting the determined information to the displaying computer.

24. A method according to claim 15, wherein the object is a surrogate for an object displayed in the video images.

25. A method according to claim 15, wherein said analyzing step includes using an image from a single TV camera in order to determine the information about the object.

26. A video display based gaming method in which changing video game images are displayed, comprising the steps of:

displaying, using a computer, changing video game images;

providing an object having an orientation and a position, the object being adapted to be freely held in a hand of a player of a game;

moving of the freely held object in both orientation and position in at least one axis by the player in the course of playing the game;

obtaining image data at least 30 times a second concerning the freely held object using at least one TV camera;

analyzing the TV camera image data obtained at least 30 times a second concerning the freely held object as the freely held object is moved in the course of playing the game to determine information about the freely held object including the orientations and positions in at least one axis of the freely held object;

inputting, by said player, an input command when the freely held object is in a desired orientation and position in at least one axis as the freely held object is moved in the course of playing the game; and

changing, with the displaying computer, said displayed video game images wherein said images are varied as a result of said determined information from said analyzing step of the TV camera image data obtained at least 30 times a second and of said input command.

27. A method according to claim 26, wherein said input command is provided by an interrupt member.

28. A method according to claim 27, wherein said interrupt member is a trigger associated with said object.

29. A method according to claim 26, wherein said determined information further includes the three dimensional position of said object.

30. A method according to claim 26, wherein an image from a single camera is analyzed by said computer in order to determine said information.