Man machine interfaces and applications

• US 8,111,239 B2
• Filed: 05/08/2006
• Issued: 02/07/2012
• Est. Priority Date: 08/22/1997
• Status: Expired due to Fees

• Current Assignee: Motion Games LLC
• Original Assignee: Motion Games LLC
• Inventors: Pryor, Timothy R., Smith, Peter
• Primary Examiner(s): Nguyen, Kevin M
• Assistant Examiner(s): Lee, Jr., Kenneth B
• Application Number:
  US11/429,305
• Publication Number:
  US 20060202953A1
• Time in Patent Office: 2,101 Days
• Field of Search: 345156-158, 345/168, 348/77
• US Class Current: 345/156
• CPC Class Codes: A63F 13/00 Video games, i.e. games using an electronically generated display having two or more dimensions gaming systems which provide a financial reward G07F17/32
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/219 for aiming at specific areas on the display, e.g. light-guns
• CPC Class Codes: A63F 13/245 specially adapted to a particular type of game, e.g. steering wheels
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/52 involving aspects of the displayed game scene
• CPC Class Codes: A63F 2300/1043 being characterized by constructional details
• CPC Class Codes: A63F 2300/1062 being specially adapted to a type of game, e.g. steering wheel
• CPC Class Codes: A63F 2300/1075 using a touch screen
• CPC Class Codes: A63F 2300/1081 Input via voice recognition
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/66 for rendering three dimensional images
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: G01S 5/16 using electromagnetic waves other than radio waves
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/0346 : with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/038 : Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/042 : by opto-electronic means
• CPC Class Codes: G06F 3/0425 : using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/0428 : by sensing at the edges of the touch surface the interruption of optical paths, e.g. an illumination plane, parallel to the touch surface which may be virtual sensing beam interruptions in a planar beam grid of an optical touch-screen G06F3/0421
• CPC Class Codes: G06T 19/20 : Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2219/2016 : Rotation, translation, scaling
• CPC Class Codes: G06T 2219/2021 : Shape modification
• CPC Class Codes: G06T 7/73 : using feature-based methods
• CPC Class Codes: G10H 1/0008 : Associated control or indicating means
• CPC Class Codes: G10H 2220/321 : Garment sensors, i.e. musical control means with trigger surfaces or joint angle sensors, worn as a garment by the player, e.g. bracelet, intelligent clothing
• CPC Class Codes: G16H 40/63 : for local operation
• CPC Class Codes: G16H 40/67 : for remote operation
• CPC Class Codes: G16H 50/50 : for simulation or modelling of medical disorders

Abstract

Affordable methods and apparatus are disclosed for inputting position, attitude (orientation) or other object characteristic data to computers for the purpose of Computer Aided Design, Painting, Medicine, Teaching, Gaming, Toys, Simulations, Aids to the disabled, and internet or other experiences. Preferred embodiments of the invention utilize electro-optical sensors, and particularly TV Cameras, providing optically inputted data from specialized datum'"'"'s on objects and/or natural features of objects. Objects can be both static and in motion, from which individual datum positions and movements can be derived, also with respect to other objects both fixed and moving. Real-time photogrammetry is preferably used to determine relationships of portions of one or more datums with respect to a plurality of cameras or a single camera processed by a conventional PC.

Claims

1. A game comprising:

a game member adapted to be freely held in a hand of a player of said game and moved in three dimensions by the player in order to play the game, the game member adapted to provide an output indicating a player command;

a single TV camera adapted to obtain images of said game member as the game member is moved in three dimensions and adapted to obtain images of a body part separate from the hand as the body part is moved in three dimensions;

a means to analyze said images obtained by the TV camera to provide three dimensional movement information of at least one of said game member and the body part as the game member and the body part are moved in three dimensions, wherein the three dimensional movement information includes a change in distance relative to said camera;

a display providing a video representation of an object; and

a computer means to control and to move the video representation of the object on said display using said three dimensional movement information.

2. Apparatus according to claim 1 wherein said video representation is of a same type of object as said game member.

3. Apparatus according to claim 1 wherein said video representation is of an enhanced object which is of the same type as the game member.

4. Apparatus according to claim 1 wherein said three dimensional position information also concerns an orientation or change in orientation of said game member.

5. Apparatus according to claim 1 wherein one or more natural features of said game member are utilized to provide said information.

6. Apparatus according to claim 1 wherein one or more artificial features of said game member are utilized to provide said information.

7. Apparatus according to claim 1 wherein said computer provides sounds relating to the position of said game member.

8. Apparatus according to claim 1 wherein said game member is a toy airplane or doll.

9. A method for playing a video game comprising the steps of:

providing a game member adapted to provide an output indicating a player command;

moving in three dimensions the game member freely held in a hand of a game player and moving in three dimensions a body part separate from the hand to play the video game;

displaying on a computer controlled display a video representation of a game object whose movement in the display is controlled by the three dimensional movement of the game member or the body part;

using a single TV camera, obtaining an image of said game member as the game member moves in three dimensions and obtaining an image of the body part as the body part moves in three dimensions;

analyzing said images of said game member and the body part obtained by the single TV camera to provide three dimensional movement information of said game member and the body part as the game member and the body part are moved in three dimensions, wherein the three dimensional movement information includes a change in distance relative to the camera; and

controlling and moving the game object in said display using said three dimensional movement information.

10. A method according to claim 9 wherein said game member is a toy.

11. A method according to claim 10 wherein said game member is a doll or airplane.

12. A method according to claim 9 wherein said game member is a simplified representation of the game object.

13. A method according to claim 9 wherein said game member is a weapon.

14. A method according to claim 12 wherein said weapon is a sword or gun.

15. A method according to claim 9 wherein said game member is a member used in playing a sport.

21. A method according to claim 9 wherein the camera includes a CMOS image sensor or a CCD image sensor.

22. A method according to claim 9 further including determining the orientation of the game member.

16. A method by which a player may play a video game, comprising the steps of:

providing a computer controlled display;

displaying a representation of a game related object on said display;

simulating, with at least one of a game member and a body part of said player, at least a portion of said game related object depicted on said display;

imaging said at least one of a game member and a body part of said player with a single TV camera as the game member or body part moves in three dimensions from a first position to a second position to play the video game;

using the images obtained by said single TV camera, determining three dimensional movement information concerning said game member if said game member is imaged and said body part if said body part is imaged as the game member or body part is moved in three dimensions from the first position and to the second position, wherein the three dimensional movement information includes a change in distance relative to the camera; and

from said determined information, controlling said display to move said game related object in the display which game related object movement minors the three dimensional movement of the game member or body part from the first position to the second position thereof.

17. A method according to claim 16 wherein said three dimensional position information further includes orientation of said body part.

18. A method according to claim 16 wherein said game related object is an airplane and said body parts are the players arms representing the wings of said airplane.

19. A method according to claim 16 wherein said game related object depicted on said display is a portion of a person.

20. A method according to claim 16 wherein the camera includes a CMOS image sensor or a CCD image sensor.