Interactive video based games using objects sensed by TV cameras

• US 8,614,668 B2
• Filed: 10/06/2011
• Issued: 12/24/2013
• Est. Priority Date: 08/22/1997
• Status: Expired due to Fees

• Current Assignee: Motion Games LLC
• Original Assignee: Motion Games LLC
• Inventors: Pryor, Timothy R.
• Primary Examiner(s): ABDULSELAM, ABBAS I
• Application Number:
  US13/267,044
• Publication Number:
  US 20120040755A1
• Time in Patent Office: 810 Days
• Field of Search: 345/157, 345/158, 345/419, 463/37
• US Class Current: 345/157
• CPC Class Codes: A63F 2300/1012 involving biosensors worn by the player, e.g. for measuring heart beat, limb activity
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: G06F 17/00 Digital computing or data processing equipment or methods, specially adapted for specific functions information retrieval, database structures or file system structures therefor G06F16/00
• CPC Class Codes: G06F 2203/0331 Finger worn pointing device
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/042 by opto-electronic means
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

A method and apparatus for interactive TV camera based games in which position or orientation of points on a player or of an object held by a player are determined and used to control a video display. Both single camera and stereo camera pair based embodiments are disclosed, preferably using stereo photogrammetry where multi-degree of freedom information is desired. Large video displays, preferably life-size may be used where utmost realism of the game experience is desired.

Claims

1. A multi-player gaming system utilizing a single display, comprising:

an electro-optical sensor having a field of view encompassing at least a portion of each of a plurality of players of a video game, the electro-optical sensor having an output; and

a computer adapted to determine, from the electro-optical sensor output, information concerning the plurality of players and to provide an output corresponding to at least one desired game-related object to be displayed on the display using the determined information, wherein the determination is made at least 30 times per second.

2. The gaming system of claim 1 wherein the portion includes at least one of a hand, a finger, a head, a foot, an arm, an upper torso, and a leg.

3. The gaming system of claim 1 wherein the determined information includes the position of a portion of a player.

4. The gaming system of claim 3 wherein the position of a portion of a player is determined to sub pixel resolution.

5. The gaming system of claim 3 wherein the display is a stereoscopic 3D display.

6. The gaming system of claim 1 wherein the determined information includes the orientation of a portion of a player.

7. The gaming system of claim 1 wherein the electro-optical sensor includes at least one of a CMOS camera and a CCD camera.

8. The gaming system of claim 1 wherein the single display provides a plurality of virtual objects that are controllable based on the determined information of at least a portion of each of the plurality of players.

9. The gaming system of claim 1 further including an internet connection over which changed game information can be transmitted.

10. The gaming system of claim 1 wherein the determined information includes the shape of a portion of a player.

11. The gaming system of claim 1 further including a microphone to sense audio information relating to at least one of the plurality of players.

12. The gaming system of claim 1 wherein the electro-optical sensor includes a stereo pair of cameras.

13. The gaming system of claim 1 further including at least one laser to project light onto the plurality of players.

14. A method for multi-player video gaming comprising:

providing a computer controlled display viewable by a plurality of players;

providing an electro-optical sensor defining a field of view encompassing at least a portion of each of the plurality of players, the electro-optical sensor having an output;

determining, using a computer, the position of a portion of at least one of the plurality of players based on the electro-optical sensor output, wherein the determination is made at least 30 times per second; and

displaying at least one game-related virtual object on the computer controlled display, the game-related virtual object being controlled based on the determined position.

15. The method according to claim 14 further including recording one or more images of at least one of the plurality of players.

16. The method according to claim 14 further including determining the orientation of a portion of at least one of the plurality of players based on the electro-optical sensor output.

17. The method according to claim 14 further including step of determining, using voice recognition, information spoken by at least one of the plurality of players.

18. The method according to claim 14 wherein the game-related virtual object is a first virtual object, the method further including displaying a second virtual object controllable based on the electro-optical sensor output.

19. A multi-player gaming system utilizing a single display, comprising:

an electro-optical sensor having a field of view encompassing at least a portion of each of a plurality of players of a video game, the electro-optical sensor having an output; and

a computer adapted to determine, from the electro-optical sensor output, the position of a portion of one of the plurality of players and to provide an output corresponding to at least one desired game-related object to be displayed on the display using the determined position, wherein the position is determined to sub-pixel resolution.

20. The gaming system of claim 19 wherein the determination is made at least 30 times per second.

21. The gaming system of claim 19 wherein the position is determined in three-dimensions.

22. The gaming system of claim 19 wherein the electro-optical sensor includes first and second cameras.

23. The gaming system of claim 19 wherein the electro-optical sensor includes at least one of a CMOS camera and a CCD camera.

24. The gaming system of claim 19 further including a microphone to sense audio information relating to at least one of the plurality of players.

25. The gaming system of claim 19 further including at least one laser to project light onto the plurality of players.

26. A multi-player gaming system utilizing a single display, comprising:

an electro-optical sensor having a field of view encompassing at least a portion of each of a plurality of players of a video game, the electro-optical sensor having an output; and

a computer adapted to determine, from the electro-optical sensor output, the three-dimensional location of a portion of one of the plurality of players and to provide an output corresponding to at least one desired game-related object to be displayed on the display using the determined location, wherein the determination is made at least 30 times per second.

27. The gaming system of claim 26 wherein the location is determined to sub-pixel resolution.

28. The gaming system of claim 26 wherein the electro-optical sensor includes first and second cameras.

29. The gaming system of claim 26 wherein the electro-optical sensor includes at least one of a CMOS camera and a CCD camera.

30. The gaming system of claim 26 further including a microphone to sense audio information relating to at least one of the plurality of players.

31. The gaming system of claim 26 further including at least one laser to project light onto the plurality of players.