Method and system to create three-dimensional mapping in a two-dimensional game

• US 9,504,920 B2
• Filed: 09/03/2014
• Issued: 11/29/2016
• Est. Priority Date: 04/25/2011
• Status: Expired due to Fees

• Current Assignee: KAYA Dynamics LLC
• Original Assignee: Aquifi, Inc.
• Inventors: Kareemi, Nazim, Rafii, Abbas
• Primary Examiner(s): Rowland, Steve
• Application Number:
  US14/476,666
• Publication Number:
  US 20150057082A1
• Time in Patent Office: 818 Days
• US Class Current: 1/1
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/2145 the surface being also a display device, e.g. touch screens
• CPC Class Codes: A63F 13/22 Setup operations, e.g. calibration, key configuration or button assignment
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/44 involving timing of operations, e.g. performing an action within a time slot
• CPC Class Codes: A63F 13/52 involving aspects of the displayed game scene
• CPC Class Codes: A63F 13/56 Computing the motion of game characters with respect to other game characters, game objects or elements of the game scene, e.g. for simulating the behaviour of a group of virtual soldiers or for path finding
• CPC Class Codes: A63F 13/573 using trajectories of game objects, e.g. of a golf ball according to the point of impact
• CPC Class Codes: A63F 13/65 automatically by game devices or servers from real world data, e.g. measurement in live racing competition
• CPC Class Codes: A63F 13/837 Shooting of targets
• CPC Class Codes: A63F 13/92 Video game devices specially adapted to be hand-held while playing
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/1075 using a touch screen
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/204 the platform being a handheld device
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/646 for calculating the trajectory of an object

Abstract

Natural three-dimensional (x_w, y_w, z_w, t_w) gesture player interaction with a two-dimensional game application rendered on a two or three dimensional display includes mapping acquired (x_w, y_w, z_w, t_w) gesture data to virtual game-world (x_v, y_v, z_v, t_v) coordinates or vice versa, and scaling if needed. The game application is caused to render at least one image on the display responsive to the mapped and scaled (x_w, y_w, z_w) data, where the display and game interaction is rendered from the player'"'"'s perception viewpoint. The (x_w, y_w, z_w) data preferably is acquired using spaced-apart two-dimensional cameras coupled to software to reduce the acquired images to a relatively small number of landmark points, from which player gestures may be recognized. The invention may be implemented in a handheld device such as a smart phone or tablet, which device may include a gyroscope and/or accelerometer.

Claims

1. A method to enable a game application, characterizable in at least three game-world (x_v, y_v, z_v,) coordinates and an implicit fourth (t_v) time coordinate, executed by a device comprising a processor and coupled to a display, to respond to at least three-dimensional (x_w, y_w, z_w) input and an implicit fourth (t_w) time input generated from a player interacting with said game application, to enable said game application to present player-interactable depth on said display, the method comprising:

(a) acquiring, by the processor, (x_w, y_w, z_w, t_w) data representing a real world position of player interaction with said game application;

(b) mapping and transforming, by the processor, acquired said (x_w, y_w, z_w, t_w) data and virtual game-world (x_v, y_v, z_v, t_v) coordinates into a unified four dimensional scaled coordinate system (x_u, y_u, z_u, t_u);

(c) using, by the processor, at least one of mapped and scaled said unified-world (x_u, y_u, z_u, t_u) coordinates to cause said game application to react to said (x_w, y_w, z_w) data acquired at step (a); and

(d) causing, by the processor, said game application to render at least one image on said display responsive to step (c);

wherein said display and game interaction is rendered from a perception viewpoint of said player; and

wherein said game application is characterized by a representation selected from a group consisting of (i) a two-dimensional virtual representation, and (ii) a three-dimensional virtual representation.

2. The method of claim 1, wherein in step (b), said (x_u, y_u, z_u, t_u) coordinate system has a characteristic selected from a group consisting of (i) said (x_u, y_u, z_u, t_u) coordinate system is the same coordinate system as said virtual game coordinate system (x_v, y_v, z_v, t_v), and (ii) said (x_u, y_u, z_u, t_u) coordinate system is the same as real world coordinate system (x_w, y_w, z_w, t_w).

3. The method of claim 1, further including, intermediate step (b) and step (c) a step of scaling mapped said virtual game-world (x_v, y_v, z_v, t_v) coordinates;

wherein step (c) includes using mapped and scaled said game-world (x_v, y_v, z_v, t_v) coordinates to cause said game application to react to said (x_w, y_w, z_w, t_w) data acquired at step (a).

4. The method of claim 1, wherein said (x_w, y_w, z_w, t_w) data acquired at step (a) represents at least one natural interface gesture interaction by said player with said game application.

5. The method of claim 1, further including mapping acquired said (x_w, y_w, z_w, t_w) data to virtual game-world (x_v, y_v, z_v, t_v) coordinates such that a representation of said player is rendered on said display.

6. The method of claim 1, wherein step (a) includes acquiring at least said (x_w, y_w, z_w) data using information obtained from spaced-apart two-dimensional cameras whose overlapping fields of view enable imaging at least a portion of interaction by said player with said game application, and includes software, coupled to said cameras, to represent said player from acquired images using landmark points from which at least one user gesture is discernible.

7. The method of claim 1, wherein step (a) includes acquiring at least said (x_w, y_w, z_w) data using a time-of-flight camera and associated software to acquire images from which at least one user gesture is discernible.

8. The method of claim 1, further including providing a portable device that includes and houses said display, said game application, and means for implementing step (a), step (b), step (c) and step (d).

9. The method of claim 8, wherein said portable device includes at least one of a gyroscope and an accelerometer, and player interaction with said game application includes player physical manipulation of said device.

10. A system enabling a game application, characterizable in at least three game-world (x_v, y_v, z_v,) coordinates and an implicit fourth (t_v) time coordinate, and executed by a game device coupled to a display, to respond to at least three-dimensional (x_w, y_w, z_w) input and an implicit fourth (t_w) time input generated from a player interacting with said game application to enable said game application to present player-interactable depth on said display, the system comprising:

said display, coupled to said system to render images commensurate with player interaction with said game application;

means for acquiring at least three-dimensional (x_w, y_w, z_w, t_w) data representing a real world position of player interaction with said game application and for recognizing from sensed at least said (x_w, y_w, z_w, t_w) data at least one player gesture;

means for mapping acquired said (x_w, y_w, z_w, t_w) and virtual game-world (x_v, y_v, z_v, t_v) coordinates into a unified four dimensional scaled coordinate system (x_u, y_u, z_u, t_u);

means for using at least one of mapped and scaled said unified-world (x_u, y_u, z_u, t_u) coordinates to cause said game application to react to said (x_w, y_w, z_w) data acquired by said means for acquiring and for recognizing; and

means for causing said game application to render at least one image on said display responsive to said means for using;

wherein said display and game interaction is rendered from a perception viewpoint of said player; and

wherein said game application is characterized by a representation selected from a group consisting of (i) a two-dimensional virtual representation, and (ii) a three-dimensional virtual representation.

11. The system of claim 10, wherein said display and said game device share a common housing.

12. The system of claim 11, wherein said game device is selected from a group consisting of (a) a smart phone, (b) a portable device, (c) a hand-held device, (d) a netbook, and (e) a laptop.

13. The system of claim 10, further including means for scaling mapped said at least virtual game-world (x_v, y_v, z_v) coordinates;

wherein said means for causing uses mapped and scaled said game-world (x_v, y_v, z_v) coordinates to cause said game application to react to acquired said (x_w, y_w, z_w) data.

14. The system of claim 10, wherein acquired said (x_w, y_w, z_w, t_w) data represents at least one natural interface gesture interaction by said player with said game application.

15. The system of claim 10, further including means for mapping acquired said (x_w, y_w, z_w, t_w) data to virtual game-world (x_v, y_v, z_v, t_v) coordinates such that a representation of said player is also rendered on said display.

16. The system of claim 10, wherein said means for acquiring and for recognizing includes at least spaced-apart first and second two-dimensional cameras whose overlapping fields of view enable imaging at least a portion of interaction by said player with said game application, and further includes software, coupled to said first and second two-dimensional cameras, to represent said player from acquired images using landmark points from which at least one user gesture is discernible.

17. The system of claim 10, wherein said means for acquiring and for recognizing includes a time-of-flight camera and associated software to acquire images from which at least one user gesture is discernible.

18. The system of claim 10, wherein said game device includes at least one of a gyroscope and an accelerometer, and player interaction with said game application includes player physical manipulation of said device.

19. A portable hand-held device including a game-playing system and a display presenting images of said game including images representing detected player interaction with said device to interact with said game, the device including:

first and second two-dimensional cameras whose overlapping fields of view enable imaging at least a portion of interaction by said player with said game application, and further includes software, coupled to said first and second two-dimensional cameras, to represent said player from acquired images using landmark points from which at least one user gesture is discernible, said first and second two-dimensional cameras acquiring at least three-dimensional (x_w, y_w, z_w) data and implicit fourth (t_w) time input data representing player interaction with said game application and enabling recognition from sensed at least said (x_w, y_w, z_w, t_w) data at least one player gesture;

processing software and at least one processor executing said processing software to carry out;

mapping of acquired said (x_w, y_w, z_w, t_w) data to virtual game-world (x_v, y_v, z_v, t_v) coordinates;

using at least one of mapped and scaled said game-world (x_v, y_v, z_v, t_v) coordinates to cause said game application to react to said (x_w, y_w, z_w, t_w) data acquired by said first and second two-dimensional cameras and

causing said game application to render at least one image on said display commensurate with acquired and processed (x_w, y_w, z_w, t_w) data representing detected player interaction with said game;

wherein said display and game interaction is rendered from a perception viewpoint of said player; and

wherein said game application is characterized by a representation selected from a group consisting of (i) a two-dimensional virtual representation, and (ii) a three-dimensional virtual representation.

20. The portable hand-held device of claim 19, wherein said device has at least one characteristic selected from a group consisting of (a) said device is a smart phone, (b) said device is a tablet, (c) said device is a netbook, (d) said device is a laptop, (e) said device includes a gyroscope and player interaction with said game application includes player physical manipulation of said device, and (f) said device includes an accelerometer and player interaction with said game application includes player physical manipulation of said device.