Method and system for controlling movements of objects in a videogame

• US 20090221374A1
• Filed: 04/26/2009
• Published: 09/03/2009
• Est. Priority Date: 11/28/2007
• Status: Active Grant

• Current Assignee: Wei Yen
• Original Assignee: AiLive, Inc.
• Inventors: Reynolds, Stuart, Tu, Xiaoyuan, Funge, John, Wright, Ian, Yen, Wei, Wilkinson, Dana, Musick, JR., Charles, Powers, III, William Robert
• Granted Patent: US 8,419,545 B2
• Time in Patent Office: Days
• US Class Current: 463/42
• CPC Class Codes: A63F 13/20 Input arrangements for video game devices
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/212 using sensors worn by the player, e.g. for measuring heart beat or leg activity
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/335 using Internet
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/52 involving aspects of the displayed game scene
• CPC Class Codes: A63F 13/55 Controlling game characters or game objects based on the game progress
• CPC Class Codes: A63F 13/65 automatically by game devices or servers from real world data, e.g. measurement in live racing competition
• CPC Class Codes: A63F 13/67 adaptively or by learning from player actions, e.g. skill level adjustment or by storing successful combat sequences for re-use
• CPC Class Codes: A63F 13/75 Enforcing rules, e.g. detecting foul play or generating lists of cheating players
• CPC Class Codes: A63F 13/79 involving player-related data, e.g. identities, accounts, preferences or play histories
• CPC Class Codes: A63F 13/847 Cooperative playing, e.g. requiring coordinated actions from several players to achieve a common goal
• CPC Class Codes: A63F 2300/1012 involving biosensors worn by the player, e.g. for measuring heart beat, limb activity
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/208 for storing personal settings or data of the player
• CPC Class Codes: A63F 2300/407 Data transfer via internet
• CPC Class Codes: A63F 2300/5533 using program state or machine event data, e.g. server keeps track of the state of multiple players on in a multiple player game
• CPC Class Codes: A63F 2300/5546 using player registration data, e.g. identification, account, preferences, game history
• CPC Class Codes: A63F 2300/5586 : for enforcing rights or rules, e.g. to prevent foul play
• CPC Class Codes: A63F 2300/6009 : for importing or creating game content, e.g. authoring tools during game development, adapting content to different platforms, use of a scripting language to create content
• CPC Class Codes: A63F 2300/6027 : using adaptive systems learning from user actions, e.g. for skill level adjustment
• CPC Class Codes: A63F 2300/6045 : for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/66 : for rendering three dimensional images
• CPC Class Codes: A63F 2300/69 : Involving elements of the real world in the game world, e.g. measurement in live races, real video

Abstract

Techniques for controlling movements of an object in a videogame are disclosed. At least one video camera is used at a location where at least a player plays the videogame, the video camera captures various movements of the player. A designated device (e.g., a game console or computer) is configured to the video data to derive the movements of the player from the video data, and cause the object to respond to the movements of the player. When the designated device receives video data from more than one locations, players at the respective locations can play a networked videogame that may be built upon a shared space representing some or all of the real-world spaces of the locations. The video game is embedded with objects, some of which respond to the movements of the players and interact with other objects in accordance with rules of the video games.

Claims

1. A method for controlling movements of two or more objects in a shared game space for a networked videogame being played by at least two players separately located from each other, the method comprising:

receiving at least a first video stream from at least a first camera associated with a first location capturing movements of at least a first player at the first location, and a second video stream from at least a second camera associated with a second location capturing movements of at least a second player at the second location;

deriving the movements of the first and second players respectively from the first and second video data streams; and

causing at least a first object in the shared game space to respond to the derived movements of the first player and at least a second object in the shared game space to respond to the derived movements of the second player, wherein the first and second locations are not necessarily co-located; and

displaying a depiction of the shared space on at least one display of each of the first and second locations.

2. The method as recited in claim 1, wherein at least one of the first and second players holds a device as a controller containing one or more inertial sensors, and sensor data from the sensors is combined with one of the first and second video streams to derive the respective movements of the first or second player.

3. The method as recited in claim 1, wherein more video cameras being set up at the first location to monitor an effective play area in which the movements of the first player are captured, the video cameras produce one or more of full color images, depth image and infrared imaging data.

4. The method as recited in claim 3, wherein the effective play area is defined by one or more of:

a field of view of the cameras, optical parameters of the cameras and surrounding lighting condition.

5. The method as recited in claim 1, wherein at least one of the first and second players has a device containing one or more inertial sensors, and sensor data from the sensors is combined with the video streams to derive the respective movements of the first or second player.

6. The method as recited in claim 5, wherein the effective play area goes beyond the field of view of the cameras, the movements of either one of the first and second players falling into a portion of the effective play area beyond the field of view of the camera are tracked by the inertial sensors.

7. The method as recited in claims 6, wherein that device is one or more of:

hand-held, wearable, or attachable to the human body.

8. The method as recited in claim 6, wherein the device is a hand-held motion-sensitive game controller.

9. The method as recited in claim 1, further comprising:

embedding gaming rules in the videogame to enable various interactions between the first and second objects, and among the first and second objects and virtual objects.

10. The method as recited in claim 9, further comprising:

mapping the movements of the first and second players to motions of the first and second objects in accordance with a predefined rule in the videogame.

11. The method as recited in claim 10, wherein the mapping of the movements maps the movements of the first player toward the first camera into the motions of the first object in a particular direction.

12. The method as recited in claim 10, wherein the mapping of the movements maps the movements of the first player and second players so that two corresponding objects in the shared game space move complimentarily.

13. The method as recited in claim 10, wherein the mapping of the movements maps the movements of the first player and second players in accordance with a choice made by the first and second players.

14. The method as recited in claim 10, wherein the mapping of the movements maps the movements of either one of the first player and second players towards a camera in a way that a corresponding object is displayed moving left or right on a display.

15. The method as recited in claim 10, wherein the mapping of the movements maps the movements of either one of the first player and second players in a way that a corresponding object moves to avoid a predefined virtual object.

16. The method as recited in claim 10, wherein the mapping of the movements maps the movements of either one of the first player and second players in a way that a 3D human-like figure moves substantially similar to either one of the first player and second players.

17. The method as recited in claim 10, wherein the mapping of the movements scales the derived movements in a way that small translations in a real-world space correspond to large translations in the shared game space of the videogame, or vice versa.

18. The method as recited in claim 10, wherein the mapping of the movements depends on one or more of game context of the videogame, the player, gaming history or experience of one or more of the first and second players with the videogame.

23. The method as recited in claim 2, wherein said mapping of the movements of the player to motions of the object depends on one or more of game context of the videogame, the player, gaming history or experience of the player with the videogame.

19. A method for controlling movements of an object in a videogame, the method comprising:

receiving at least one video stream from a video camera capturing various movements of a player of the videogame;

deriving the movements of the player from the video data; and

causing the object to respond to the movements of the player.

20. The method as recited in claim 19, further comprising:

mapping the movements of the player to motions of the object in accordance with a predefined rule in the videogame.

21. The method as recited in claim 20, wherein the motions are scaled so that small translations in a real-world space of the player correspond to large translations in a game space of the videogame, or vice versa.

22. The method as recited in claim 21, wherein the motions are scaled non-linearly so that small movements of the player are mapped almost faithfully, but large movements of the player are damped.

24. The method as recited in claim 19, wherein the video camera is provided to monitor an effective play area in which the movements of the player are captured.

25. The method as recited in claim 24, wherein the effective play area is defined by one or more of:

a field of view of the camera, optical parameters of the camera and surrounding lighting conditions.

26. The method as recited in claim 25, wherein the effective play area goes beyond the field of view of the camera, the movements of the player falling into a portion of the effective play area beyond the field of view of the camera are tracked by inertial sensors in a controller being held by the player.

27. The method as recited in claim 19, further comprising:

embedding gaming rules in the videogame to enable various interactions between the object and other virtual objects, wherein each of the interactions happens as the player manipulates at least a controller including inertial sensors.

28. The method as recited in claim 27, wherein sensor data from the inertial sensors is used to derive relative translational and angular movements of the controller, both of the sensor data and the video data are combined to derive absolute translational and angular movements of the controller.

29. A system for controlling movements of two or more objects in a shared game space for a networked videogame being played by at least two players remotely located from one another, the system comprising:

at least a first camera and a second camera, respectively, with a first location and a second location, the first camera producing a first video stream capturing movements of at least a first player at the first location, the second camera producing a second video stream capturing movements of at least a second player at the second location;

a designated computing device configured to derive the movements of the first and second players respectively from the first and second video data streams and cause at least a first object in the shared game space to respond to the derived movements of the first player and at least a second object in the shared game space to respond to the derived movements of the second player, wherein the first and second locations are not necessarily co-located; and

first and second displays respectively displaying the networked videogame including the shared space.

30. The system as recited in claim 29, wherein the shared space is built upon 3D representation of real-world spaces of the first and second locations.

31. The system as recited in claim 30, wherein at least one of the first and second players holds a device containing one or more inertial sensors, and sensor data from the sensors is combined with one of the first and second video streams to derive the respective movements of the first or second player.

32. The system as recited in claim 29, wherein more video cameras are being set up in the first location to monitor an effective play area in which the movements of the first player are captured.

33. The system as recited in claim 32, wherein the effective play area is defined by one or more of:

a field of view of the cameras, optical parameters of the cameras and surrounding lighting conditions.

34. The system as recited in claim 29, wherein at least one of the first and second players has a device containing one or more inertial sensors, and sensor data from the sensors is combined with the video streams to derive the respective movements of the first or second player.

35. The system as recited in claim 34, wherein the effective play area goes beyond a field of view of the first or second camera, the movements of the first or second player falling into a portion of the effective play area beyond the field of view of the first or second camera are tracked by the inertial sensors.

36. The system as recited in claims 35, wherein that device is one or more of:

hand-held, wearable, or attachable to the human body.

37. The system as recited in claim 35, wherein the device is a hand-held motion-sensitive game controller.

38. The system as recited in claim 29, wherein the device is configured to embed gaming rules in the videogame to enable various interactions between the first and second objects, and among the first and second objects and virtual objects.

39. The system as recited in claim 29, wherein the device is configured to map the movements of the first and second players to motions of the first and second objects in accordance with a predefined rule in the videogame.

40. The system as recited in claim 29, wherein the device is configured to map the movements of the first player toward the first camera into the motions of the first object in a particular direction.

41. The system as recited in claim 29, wherein the device is configured to map the movements of the first player and second players so that two corresponding objects in the shared game space move complimentarily.

42. The system as recited in claim 29, wherein the device is configured to map the movements of the first player and second players in accordance with a choice made by the first and second players.

43. The system as recited in claim 29, wherein the device is configured to map the movements of either one of the first player and second players towards a camera in a way that a corresponding object is displayed moving left or right on a display.

44. The system as recited in claim 29, wherein the device is configured to map the movements of either one of the first player and second players in a way that a corresponding object moves to avoid a predefined virtual object.

45. The system as recited in claim 29, wherein the device is configured to map the movements of either one of the first player and second players in a way that a 3D human-like figure moves substantially similar to either one of the first player and second players.

46. The system as recited in claim 29, wherein the device is configured to scale the derived movements in a way that small translations in a real-world space correspond to large translations in the shared game space of the videogame, or vice versa.

47. The system as recited in claim 29, wherein the device is configured to map the movements in accordance with one or more of game context of the videogame, the player, gaming history or experience of one or more of the first and second players with the videogame.

48. A system for controlling movements of an object in a videogame, the system comprising:

at least a video camera capturing various movements of a player of the videogame;

a designated device receiving at least one video stream from the video camera to derive the movements of the player from the video data, and cause the object to respond to the movements of the player,

49. The system as recited in claim 48, wherein the designated device is configured to map the movements of the player to motions of the object in accordance with a predefined rule in the videogame.

50. The system as recited in claim 48, wherein the motions are scaled so that small translations in a real-world space of the player correspond to large translations in a game space of the videogame, or vice versa.

51. The system as recited in claim 48, wherein the motions are scaled non-linearly so that small movements of the player are mapped almost faithfully, but large movements of the player are damped.

52. The system as recited in claim 48, wherein the predefined rule in the videogame depends on one or more of game context of the videogame, the player, gaming history or experience of the player with the videogame.

53. The system as recited in claim 48, wherein the video camera is provided to monitor an effective play area in which the movements of the player are captured.

54. The system as recited in claim 53, wherein the effective play area is defined by one or more of:

a field of view of the camera, optical parameters of the camera and surrounding lighting conditions.

55. The system as recited in claim 54, wherein the effective play area goes beyond the field of view of the camera, the movements of the player falling into a portion of the effective play area beyond the field of view of the camera are tracked by inertial sensors in a controller being held by the player.

56. The system as recited in claim 48, wherein the designated device is configured to embed gaming rules in the videogame to enable various interactions between the object and other virtual objects, each of the interactions happens as the player manipulates at least a controller including inertial sensors.

57. The system as recited in claim 56, wherein sensor data from the inertial sensors is used to derive relative translational and angular movements of the controller, both of the sensor data and the video data are combined to derive absolute translational and angular movements of the controller.