Interactive system and method for rendering an object

• US 9,132,352 B1
• Filed: 06/16/2011
• Issued: 09/15/2015
• Est. Priority Date: 06/24/2010
• Status: Active Grant

• Current Assignee: Gregory S. Rabin, Steven Mark Hoffberg
• Original Assignee: Gregory S. Rabin, Steven Mark Hoffberg
• Inventors: Rabin, Gregory S., Hoffberg, Steven M.
• Primary Examiner(s): Zarka, David
• Assistant Examiner(s): PRINGLE-PARKER, JASON A
• Application Number:
  US13/161,820
• Time in Patent Office: 1,552 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: A63F 13/30 Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
• CPC Class Codes: A63F 13/57 Simulating properties, behaviour or motion of objects in the game world, e.g. computing tyre load in a car race game A63F13/56 takes precedence
• CPC Class Codes: A63F 13/65 automatically by game devices or servers from real world data, e.g. measurement in live racing competition
• CPC Class Codes: A63F 13/812 Ball games, e.g. soccer or baseball
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 15/40 Hidden part removal
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2200/08 involving all processing steps from image acquisition to 3D model generation
• CPC Class Codes: G06T 2207/30221 Sports video; Sports image
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 9/00 Image coding bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00

Abstract

A system and method for receiving an ordered set of images and analyzing the images to determine at least one position in space and at least one motion vector in space and time for at least one object represented in the images is disclosed. Using these vectors, a four dimensional model of at least a portion of the information represented in the images is formulated. This model generally obeys the laws of physics, though aberrations may be imposed. The model is then exercised with an input parameter, which, for example, may represent a different perspective than the original set of images. The four dimensional model is then used to produce a modified set of ordered images in dependence on the input parameter and optionally the set of images, e.g., if only a portion of the data represented in the images is modeled. The set of images may then be rendered on a display device.

Claims

1. A method, comprising:

storing a computer physical model of at least a human player and a projectile object within a physical scene in a memory, developed based on a sports game having an objective based on a movement of the projectile object by the human player, a set of rules and penalties for violation of the rules, and being subject to human player injuries, represented in at least a first series of images of the physical scene comprising the human player and the projectile object, the computer physical model representing at least three physical dimensions of each of the human player and the projectile object, and the physical scene, and implementing at least physical laws comprising conservation of momentum and conservation of energy;

retrieving information from a database containing information representing at least statistical probabilities of an outcome of a physical interaction between the human player and the projectile object dependent on an identification of the human player and a distance between the projectile object and an element within the physical scene;

receiving a second series of images representing a first physical interaction of the human player and the projectile object within the physical scene representing the sports game, causing an alteration of respective motion parameters of the human player and the projectile object, at a subsequent time after creation of the computer physical model, resulting in a first outcome dependent on the first physical interaction;

automatically extracting a set of motion parameters for the human player and the projectile object for at least the first interaction, each comprising a motion vector describing a respective position, velocity and acceleration;

modifying the first physical interaction of the human player and the projectile object within the physical scene to represent a second physical interaction different from the first physical interaction, to thereby modify the extracted set of motion parameters for the respective models of human player and the projectile object, to cause a second outcome dependent on at least the modified extracted set of motion parameters and the statistical probabilities retrieved from the database dependent on the identification of the human player and the distance between the projectile object and the element within the physical scene;

determining penalties and predicted human player injuries based on the second outcome; and

rendering with an automated processor, a third series of images of the identified human player and the projectile object within the physical scene according to the second physical interaction, comprising the second outcome.

2. The method of claim 1, wherein said rendering comprises representing the physical scene as an ordered set of two dimensional images.

3. The method of claim 1, wherein said rendering comprises representing the physical scene as an ordered set of first ordered set of three dimensional images.

4. The method of claim 1, wherein the computer physical model is derived from a live broadcast sports game.

5. The method of claim 4, wherein the computer physical model implements the physical law comprising conservation of mass.

6. The method of claim 1, wherein the computer physical model is derived from automated analysis of a series of images from a first perspective, and the modified motion parameters results in a rendering of the human player and the projectile object from a different perspective than the first perspective.

7. The method of claim 1, wherein the computer physical model is generated based on automated analysis of at least a live broadcast video representation of the physical scene, comprising at least two human players and the projectile object.

8. The method of claim 7, wherein the computer physical model is generated based on automated analysis of at least acoustic information.

9. The method of claim 1, wherein the modified set of motion parameters comprise an altered parameter associated with at least one of momentum, energy, and mass of the at least one of the human player and the projectile object.

10. The method of claim 1, further comprising automatically analyzing the received series of images to determine at least an inconsistency with a law of physics, wherein the rendering is selectively responsive to the determined inconsistency.

11. The method of claim 1, wherein the computer physical model comprises a four dimensional model which represents at least a volume and a mass, and the automatically extracted set of motion parameters, corresponding to each of a plurality of physical objects represented in the received series of images.

12. The method of claim 1, wherein the computer physical model comprises a texture mapping model configured to impose appropriate surface textures derived and extrapolated from an ordered set of images.

13. The method of claim 1, wherein the computer physical model comprises an interaction model according to the laws of physics representing physical contact interactions between the human player and the projectile object which models a volume, a mass, the position, the velocity, and the acceleration of the human player and the projectile object in the scene, and implements the physical laws comprising conservation of momentum, conservation of energy, and conservation of mass.

14. The method of claim 1, further comprising interacting with a human user through a human user-machine interface configured to provide interactive feedback to the human user with respect to an effect of the modifying of the motion parameters for the at least one of the human player and the projectile object on the rendering, wherein the human user is distinct from the human player.

15. A system, comprising:

at least one memory configured to store a computer physical model of at least a human player and a projectile object within a physical scene in a memory, developed based on a sports game having an objective based on a movement of the projectile object by the human player, having a set of rules and penalties for violation of the rules, and subject to human player injuries, represented in at least a first series of images of the physical scene comprising the human player and the projectile object, the computer physical model representing at least three physical dimensions of each of the human player and the projectile object, and the physical scene, and implementing at least physical laws comprising conservation of momentum and conservation of energy;

at least one processor configured to;

retrieve information from a database containing information representing at least statistical probabilities of an outcome of a physical interaction between the human player and the projectile object dependent on an identification of the human player and a distance between the projectile object and an element within the physical scene;

receive a second series of images representing a first physical interaction of the human player and the projectile object within the physical scene representing the sports game, causing an alteration of respective motion parameters of the human player and the projectile object, at a subsequent time after creation of the physical model, resulting in a first outcome dependent on the first physical interaction;

automatically extract a set of motion parameters for the human player and the projectile object for at least the first interaction, each comprising a motion vector describing a respective position, velocity and acceleration;

modify the first physical interaction of the human player and the projectile object within the physical scene to represent a second physical interaction different from the first physical interaction, to thereby modify the extracted set of motion parameters for the respective models of human player and the projectile object, to cause a second outcome dependent on at least the modified extracted set of motion parameters and the statistical probabilities retrieved from the database dependent on the identification of the human player and the distance between the projectile object and the element within the physical scene;

determine penalties and predicted human player injuries based on the second outcome; and

render with an automated processor, a third series of images of the identified human player and the projectile object within the physical scene according to the second physical interaction, comprising the second outcome.

16. The system of claim 15, wherein the computer physical model of the human player and the projectile object is derived based on at least an ordered set of images from a broadcast video.

17. The system of claim 15, wherein the computer physical model implements the physical law comprising conservation of mass.

18. The system of claim 15, wherein the computer physical model is generated based on an automated analysis of at least a video representation of the physical scene from a first perspective, and wherein the modified set of motion parameters results in a rendering of the physical scene from a different perspective than the first perspective.

19. The system of claim 15, wherein the computer physical model comprises a four dimensional model which represents at least a volume and a mass representation corresponding to each of the human player and the projectile object, and the automatically extracted set of motion parameters, represented in an ordered set of images.

20. The method of claim 15,

wherein the computer physical model comprises an interaction model according to the laws of physics representing physical contact interactions between the human player and the projectile object, wherein the interaction model represents a volume, a mass, the position, the velocity, and the acceleration,

the at least one processor being further configured to render the image of the physical scene to implement the physical laws comprising conservation of momentum, conservation of energy, and conservation of mass.

21. A method, comprising:

automatically generating a computer physical model representing at least four dimensions in accordance with the laws of physics comprising at least conservation of mass, conversation of momentum and conservation of energy, of a plurality of interacting physical objects in a physical scene, comprising at least a human player and a projectile object, developed based on a sports game having an objective based on a movement of the projectile object by the human player, having a set of rules and penalties for violation of the rules, and subject to human player injuries, based on a series of images;

retrieving information from a database representing a random variable associated with a statistical result of an attempted action involving the human player and the projectile object within the physical scene, dependent on an identification of the human player, and at least one distance relative to the human player or the projectile object within the physical scene;

receiving a stream of images representing the plurality of interacting physical objects in the physical scene representing the sports game, and analyzing the received stream of images to extract a set of motion parameters for the plurality of interacting physical objects in the physical scene, comprising at least motion vectors for the plurality of interacting physical objects comprising at least position, velocity and acceleration, wherein the plurality of interacting physical objects have a first physical interaction which causes an alteration of respective motion parameters of respective interacting physical objects in the physical scene, at a subsequent time after creation of the computer physical model, resulting in a first outcome dependent on the first physical interaction;

receiving an input defining a modification of the set of motion parameters of the plurality of interacting physical objects independent of the stream of images to thereby represent a second physical interaction according to the computer physical model and the laws of physics comprising at least conservation of mass, conversation of momentum and conservation of energy, different from an actual state of the physical scene represented by the first physical interaction in the received stream of images;

determining penalties and predicted human player injuries based on the second outcome; and

rendering a series of images representing a predicted motion of the plurality of interacting physical objects in the physical scene with an automated processor, in accordance with the second physical interaction, the random variable, the at least one distance relative to the human player or the projectile object within the physical scene, and the laws of physics comprising at least conservation of mass, conservation of momentum and conservation of energy and the automatically generated computer physical model.