Interactive video based games using objects sensed by TV cameras

• US 8,736,548 B2
• Filed: 03/26/2013
• Issued: 05/27/2014
• Est. Priority Date: 08/22/1997
• Status: Expired due to Fees

• Current Assignee: PSTP Technologies, LLC
• Original Assignee: PSTP Technologies, LLC
• Inventors: Pryor, Timothy R.
• Primary Examiner(s): ABDULSELAM, ABBAS I
• Application Number:
  US13/850,561
• Publication Number:
  US 20130249791A1
• Time in Patent Office: 427 Days
• Field of Search: 345/156, 345/158, 345/473, 463/37, 463/45
• US Class Current: 345/156
• CPC Class Codes: A63F 2300/1012 involving biosensors worn by the player, e.g. for measuring heart beat, limb activity
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: G06F 17/00 Digital computing or data processing equipment or methods, specially adapted for specific functions information retrieval, database structures or file system structures therefor G06F16/00
• CPC Class Codes: G06F 2203/0331 Finger worn pointing device
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/042 by opto-electronic means
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

A method and apparatus for interactive TV camera based games in which position or orientation of points on a player or of an object held by a player are determined and used to control a video display. Both single camera and stereo camera pair based embodiments are disclosed, preferably using stereo photogrammetry where multi-degree of freedom information is desired. Large video displays, preferably life-size may be used where utmost realism of the game experience is desired.

Claims

1. A control interface comprising:

first and second cameras each including a field of view and an output; and

a processing unit operatively coupled to the outputs of the first and second cameras and adapted to;

detect a gesture performed in three-dimensions by a user in the field of view of the first and second cameras,

assign a user-designated meaning to the detected gesture based on an input by the user,

record the user-designated meaning to computer readable memory for subsequent use of the gesture by the user,

detect a subsequent use of the gesture in the field of view of the first and second cameras; and

look-up the user-designated meaning of the gesture in the computer readable memory.

2. The control interface of claim 1 wherein the user-designated meaning and the detected gesture are stored to a look-up table in the computer readable memory.

3. The control interface of claim 1 further including a display viewable by the user and fixed in relation to the electro-optical sensor.

4. The control interface of claim 1 further including a light source to illuminate an object in the field of view of the first and second cameras.

5. The control interface of claim 4 wherein the light source is an infrared light source.

6. The control interface of claim 4 wherein the light source is one of a light emitting diode and a laser.

7. The control interface of claim 1 wherein the gesture is one of a grip gesture, a pinch gesture, a grasp gesture, a rotate gesture, a pull gesture, and a point gesture.

8. A computer implemented method comprising:

providing first and second cameras each including a field of view;

detecting, using the first and second cameras, a gesture performed in three-dimensions by a user in the fields of view of the first and second cameras;

receiving an additional input from the user to assign a user-designated meaning to the gesture performed in the fields of view of the first and second cameras; and

recording the user-designated meaning to a computer readable memory for subsequent use of the detected gesture by the user;

detecting, using the electro-optical sensor, a subsequent use of the gesture by the user in the fields of view of the first and second cameras; and

identifying the user-designated meaning of the gesture in the computer readable memory.

9. The method according to claim 8 wherein the additional input includes a voice instruction.

10. The method according to claim 8 wherein the additional input includes a keyboard instruction.

11. The method according to claim 8 further including storing the user-designated meaning and the detected gesture to a lookup table in the computer readable memory.

12. The method according to claim 11 wherein the lookup table includes a plurality of gestures each corresponding to an assigned meaning.

13. The method according to claim 8 further including providing a processing unit to detect the gesture repeated by the user.

14. The method according to claim 13 wherein the processing unit is adapted to perform the user-designated meaning in response to the repeated gesture.

15. The method according to claim 8 further including providing a light source to illuminate an object in the fields of view of the first and second cameras.

16. The method according to claim 15 wherein the light source is an infrared light source.

17. The method according to claim 15 wherein the light source is one of a light emitting diode and a laser.

18. The method according to claim 8 wherein the gesture is one of a grip gesture, a pinch gesture, a grasp gesture, a rotate gesture, a pull gesture, and a point gesture.

19. A control interface comprising:

a stereo camera pair including first and second cameras each including a field of view and adapted to detect a gesture performed in each camera field of view; and

a processing unit operatively coupled to the stereo camera pair and adapted to;

assign a user-designated meaning to the detected gesture in response to a user designation, wherein the user designation includes a voice instruction,

record the user-designated meaning to a computer readable memory for subsequent use of the gesture by the user;

detect a subsequent use of the gesture in the fields of view of the first and second cameras; and

identify the user-designated meaning of the gesture from the computer readable memory.

20. The control interface of claim 19 wherein the user-designated meaning and the gesture are stored to a look-up table in the computer readable memory.

21. The control interface of claim 19 further including a display viewable by the user and fixed in relation to the stereo camera pair.

22. The control interface of claim 19 further including a light source to illuminate an object in the fields of view of the first and second cameras.