Camera based man machine interfaces

• US 7,401,783 B2
• Filed: 07/19/2004
• Issued: 07/22/2008
• Est. Priority Date: 07/08/1999
• Status: Expired due to Fees

• Current Assignee: Timothy R. Pryor
• Original Assignee: Timothy R. Pryor
• Inventors: Pryor, Timothy R.
• Primary Examiner(s): Mendiratta; Vishu K.
• Application Number:
  US10/893,534
• Publication Number:
  US 20050129273A1
• Time in Patent Office: 1,464 Days
• Field of Search: 273/237, 273/454, 273/274, 273/309, 463/1, 463/40, 463/42, 463/43
• US Class Current: 273/237
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/219 for aiming at specific areas on the display, e.g. light-guns
• CPC Class Codes: A63F 13/23 for interfacing with the game device, e.g. specific interfaces between game controller and console
• CPC Class Codes: A63F 13/27 characterised by a large display in a public venue, e.g. in a movie theatre, stadium or game arena
• CPC Class Codes: A63F 13/426 involving on-screen location information, e.g. screen coordinates of an area at which the player is aiming with a light gun
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/52 involving aspects of the displayed game scene
• CPC Class Codes: A63F 13/803 Driving vehicles or craft, e.g. cars, airplanes, ships, robots or tanks
• CPC Class Codes: A63F 2009/2435 using a video camera
• CPC Class Codes: A63F 2300/1006 having additional degrees of freedom
• CPC Class Codes: A63F 2300/1025 details of the interface with the game device, e.g. USB version detection
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/66 for rendering three dimensional images
• CPC Class Codes: A63F 2300/8023 the game being played by multiple players at a common site, e.g. in an arena, theatre, shopping mall using a large public display
• CPC Class Codes: A63F 3/00643 Electric board games; Electric features of board games electric word or number games A63F3/0421; computer chess G06F; electric raffle games A63F3/081
• CPC Class Codes: G06F 3/01 Input arrangements or combined input and output arrangements for interaction between user and computer G06F3/16 takes precedence
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 : Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0325 : using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/0425 : using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06T 2207/10021 : Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/30196 : Human being; Person
• CPC Class Codes: G06T 2207/30204 : Marker
• CPC Class Codes: G06T 7/246 : using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/593 : from stereo images
• CPC Class Codes: G06T 7/73 : using feature-based methods
• CPC Class Codes: G06V 10/143 : Sensing or illuminating at different wavelengths
• CPC Class Codes: G06V 40/20 : Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

Disclosed herein are new forms of computer inputs particularly using TV Cameras, and providing affordable methods and apparatus for data communication with respect to people and computers using optically inputted information from specialized datum'"'"'s on objects and/or natural features of objects. Particular embodiments capable of fast and reliable acquisition of features and tracking of motion are disclosed, together with numerous applications in various fields of endeavor.

Claims

1. A board game apparatus comprising:

a board on which a play of a game takes place, said board containing information relating to the play of the game;

a physical marker used for the play of the game, said marker resting on said board and capable of being moved by a person during the play of the game;

at least one TV Camera positioned to view said board and said marker, said TV camera producing an output; and

a computer, connected to said at least one TV camera, wherein said computer comprises;

a) means for analyzing the output of said TV camera and recognizing from the analysis a relative position of said marker with respect to the information on said board,

b) means for analyzing and recognizing, after a movement of said marker during the play of the game which is viewed by said TV camera, a new position of said marker with respect to the information on said board, and

c) means for automatically generating, after the new position of said marker is recognized, a sensory output designed to be capable of being perceived by the person, said sensory output being different from a view of said board and marker thereon and being associated with the recognized new position of said marker with respect to the information on said board.

2. Apparatus according to claim 1, wherein said computer further comprises means for recognizing at least one identifying characteristic of said marker from the output of said TV camera.

3. Apparatus according to claim 1, further including means for displaying video imagery or for outputting audio information to the person as said sensory output which is generated by said means.

4. Apparatus according to claim 3:

wherein said means to output is a video display, and

wherein said computer means outputs image data of a structure maintained in a computer memory, which structure is associated with the new position of said marker.

5. Apparatus according to claim 1, wherein said computer further comprises means for determining information concerning other objects used in the game or other persons playing the game.

6. Apparatus according to claim 1, wherein said board is stiff so as to not easily be deformed during transport or play.

7. Apparatus according to claim 1, wherein said board is information provided on a computer generated video display.

8. Apparatus according to claim 7, wherein said computer further comprises means for providing a signal to said video display to output an image therein associated with the new position of said marker.

9. A method for board game play comprising the steps of:

providing a board on which a physical markers is moved by a person during a play of the game, said board containing information relating to the play of said game;

providing at least one TV Camera positioned to view said board and said marker thereon, the TV camera producing an output which is fed to a computer;

utilizing the computer to analyze the output of the TV camera and then to recognize, from the analyzed output a relative position of said marker with respect to the information provided on said board;

after a movement of said marker during the play of the game to a new position on the board which is viewed by the TV camera, utilizing the computer to analyze the output of the TV camera and then to recognize, from the analyzed output, the new position of said marker with respect to the information on said board; and

generating with the computer, as a result of the new position recognized by the computer, a video or audio sensory output different from a view of said board and associated with the recognized new position of said marker with respect to the information on said board which sensation is designed to be capable of being perceived by the person playing said game.

10. A method according to claim 9, including the further step of establishing in the computer a reference coordinate system for the board and marker.

11. A method according to claim 9, including the further step of recognizing the marker with the computer.

12. A method according to claim 11, wherein said recognition pertains as well to the player associated with the identified marker.

13. A method according to claim 9, wherein said marker has a specialized datum to aid in recognizing position by said computer.

14. A method according to claim 9, wherein said sensation is selected from a computer memory to correspond to one of a plurality of choices for a given game.

15. A method according to claim 14, in which a MONOPOLY®

game is enhanced by using local or otherwise familiar scenes or sounds to represent the various properties and other features in the game.

16. A method according to claim 9, wherein said sensory output is downloaded by said computer from remote sources.

17. A method according to claim 9,

wherein said generating step provides a video sensory output which is an image on a video display, and

wherein the video sensory output is derived from image data of a structure maintained in computer memory which is associated with the new position of said marker.