Method and system to create three-dimensional mapping in a two-dimensional game

• US 8,840,466 B2
• Filed: 04/20/2012
• Issued: 09/23/2014
• Est. Priority Date: 04/25/2011
• Status: Active Grant

• Current Assignee: KAYA Dynamics LLC
• Original Assignee: Aquifi, Inc.
• Inventors: Kareemi, Nazim, Rafii, Abbas
• Primary Examiner(s): Rowland, Steve
• Application Number:
  US13/506,474
• Publication Number:
  US 20120270653A1
• Time in Patent Office: 886 Days
• Field of Search: 463/31
• US Class Current: 463/31
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/2145 the surface being also a display device, e.g. touch screens
• CPC Class Codes: A63F 13/22 Setup operations, e.g. calibration, key configuration or button assignment
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/44 involving timing of operations, e.g. performing an action within a time slot
• CPC Class Codes: A63F 13/52 involving aspects of the displayed game scene
• CPC Class Codes: A63F 13/56 Computing the motion of game characters with respect to other game characters, game objects or elements of the game scene, e.g. for simulating the behaviour of a group of virtual soldiers or for path finding
• CPC Class Codes: A63F 13/573 using trajectories of game objects, e.g. of a golf ball according to the point of impact
• CPC Class Codes: A63F 13/65 automatically by game devices or servers from real world data, e.g. measurement in live racing competition
• CPC Class Codes: A63F 13/837 Shooting of targets
• CPC Class Codes: A63F 13/92 Video game devices specially adapted to be hand-held while playing
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/1075 using a touch screen
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/204 the platform being a handheld device
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/646 for calculating the trajectory of an object

Abstract

Natural three-dimensional (x_w, y_w, z_w, t_w) gesture player interaction with a two-dimensional game application rendered on a two or three dimensional display includes mapping acquired (x_w, y_w, z_w, t_w) gesture data to virtual game-world (x_v, y_v, z_v, t_v) coordinates or vice versa, and scaling if needed. The game application is caused to render at least one image on the display responsive to the mapped and scaled (x_w, y_w, z_w) data, where the display and game interaction is rendered from the player'"'"'s perception viewpoint. The (x_w, y_w, z_w) data preferably is acquired using spaced-apart two-dimensional cameras coupled to software to reduce the acquired images to a relatively small number of landmark points, from which player gestures may be recognized. The invention may be implemented in a handheld device such as a smart phone or tablet, which device may include a gyroscope and/or accelerometer.

Claims

1. A system enabling a game application, characterizable in at least three game-world (x_v, Y_v, z_v) coordinates and an implicit fourth (t_v) time coordinate, and executed by a game device coupled to a display, to respond to at least a natural gesture three-dimensional (x_w, y_w, z_w) input and an implicit fourth (t_w) time input generated from a player interacting with said game application to enable said game application to present player-interactable depth on said display, the system comprising:

said display, coupled to said system to render images commensurate with player interaction with said game application;

an optical acquisition system to acquire at least three-dimensional (x_w, y_w, z_w, t_w) data representing player interaction with said game application;

means for recognizing from sensed at least said (x_w, y_w, z_w, t_w) data acquired by said optical acquisition system at least one player gesture;

said optical acquisition system to acquire at least three-dimensional (x_w, y_w, z_w, t_w) data and said means for recognizing from sensed at least said (x_w, y_w, z_w, t_w) data including at least spaced-apart first and second two-dimensional cameras whose overlapping fields of view acquire images including data representing potential landmark points and data imaging at least a portion of interaction by said player with said game application;

executable software, coupled to said first and second two-dimensional cameras, to represent said player from acquired said images using as least some said landmark points from which at least one user gesture is discernible;

means for mapping acquired said (x_w, y_w, z_w, t_w) and virtual game-world (x_v, y_v, z_v, t_v) coordinates into a unified four dimensional scaled coordinate system (x_u, y_u, z_u, t_u);

means for using at least one of mapped and scaled said unified-world (x_u, y_u, z_u, t_u) coordinates to cause said game application to react to said (x_w, y_w, z_w) data acquired by said optical acquisition system; and

means for causing, said game application to render at least one image on said display responsive to said means for using;

wherein said display and game interaction is rendered from a perception viewpoint of said player; and

wherein said game application is characterized by a representation selected from a group consisting of (i) a two-dimensional virtual representation, and (ii) a three-dimensional virtual representation.

2. The system of claim 1, wherein said display and said game device share a common housing.

3. The system of claim 2, wherein said game device is selected from a group consisting of (a) a smart phone, (b) a portable device, (c) a hand-held device, (d) a netbook, and (e) a laptop.

4. The system of claim 1, further including means for scaling mapped said at least virtual game-world (x_v, y_v, z_v) coordinates;

wherein said means for causing uses mapped and scaled said game-world (x_v, y_v, z_v) coordinates to cause said game application to react to acquired said (x_w, y_w, z_w) data.

5. The system of claim 1, wherein acquired said (x_w, y_w, z_w, t_w) data represents at least one natural interface gesture interaction by said player with said game application, said natural interface gesture including at least one gesture selected from a group consisting of (i) an open hand of said player, (ii) a clenched hand of said player, (iii) a movement of a hand of said player, (iv) a movement of an arm of said player, (v) a movement of a torso of said player, and (vi) a movement of a head of said player.

6. The system of claim 1, further including means for mapping acquired said (x_w, y_w, z_w, t_w) data to virtual game-world (x_v, y_v, z_v, t_v) coordinates such that a representation of said player is also rendered on said display.

7. The system of claim 1, wherein said optical acquisition system includes a time-of-flight camera and associated software to acquire images from which at least one user natural gesture is discernible.

8. The system of claim 1, wherein said game device includes at least one of a gyroscope and an accelerometer, and player interaction with said game application includes player physical manipulation of said device.