Interactive system and method

• US 10,549,197 B1
• Filed: 10/05/2018
• Issued: 02/04/2020
• Est. Priority Date: 06/24/2010
• Status: Active Grant

• Current Assignee: Steven Mark Hoffberg
• Original Assignee: Steven Mark Hoffberg
• Inventors: Rabin, Gregory S., Hoffberg, Steven M.
• Primary Examiner(s): Pringle-Parker, Jason A
• Application Number:
  US16/153,627
• Time in Patent Office: 487 Days
• Field of Search: None
• CPC Class Codes: A63F 13/30 Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
• CPC Class Codes: A63F 13/57 Simulating properties, behaviour or motion of objects in the game world, e.g. computing tyre load in a car race game A63F13/56 takes precedence
• CPC Class Codes: A63F 13/65 automatically by game devices or servers from real world data, e.g. measurement in live racing competition
• CPC Class Codes: A63F 13/812 Ball games, e.g. soccer or baseball
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 15/40 Hidden part removal
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2200/08 involving all processing steps from image acquisition to 3D model generation
• CPC Class Codes: G06T 2207/30221 Sports video; Sports image
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 9/00 Image coding bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00

Abstract

A system and method for receiving an ordered set of images and analyzing the images to determine at least one position in space and at least one motion vector in space and time for at least one object represented in the images is disclosed. Using these vectors, a four dimensional model of at least a portion of the information represented in the images is formulated. This model generally obeys the laws of physics, though aberrations may be imposed. The model is then exercised with an input parameter, which, for example, may represent a different perspective than the original set of images. The four dimensional model is then used to produce a modified set of ordered images in dependence on the input parameter and optionally the set of images, e.g., if only a portion of the data represented in the images is modeled. The set of images may then be rendered on a display device.

Claims

1. A method, comprising:

receiving at least one stream of live action video images of a game;

processing the at least one stream of live action video images to define a plurality of respective physical models of a plurality of respective objects in the video, representing, for each respective modelled object, at least a surface configuration, a mass, and a motion of the respective object, the plurality of respective physical models together describing dynamic interactions between the plurality of respective objects over time, said processing comprising hidden surface extrapolation, wherein said processing applies physical laws;

determining a motion-dependent outcome probability statistic for at least one of the plurality of respective objects;

determining game logic of the game dependent on at least the motion-dependent outcome;

receiving a user input representing a change in a dynamic state of at least one respective object, the received user input altering a predicted path of the at least one respective object extrapolated from the live action video images, changing at least one predicted interaction comprising physical contact between at least two respective objects in the game according to the laws of physics applied to the changed dynamic state; and

generating a synthetic stream of action video images, dependent on the received at least one stream of live action video images of the game, the plurality of respective physical models, the game logic, the determined probability statistic, the received user input, and the physical laws.

2. The method according to claim 1, wherein the game logic comprises use of a random variable to determine an outcome.

3. The method according to claim 2, wherein the random variable comprises a probability.

4. The method according to claim 2, wherein the random variable is based on a statistical analysis.

5. The method according to claim 1, wherein the synthetic stream of action video images differs from the at least one stream of live action video images of the game with respect to at least one advertisement.

6. The method according to claim 1, further comprising determining a consistency of the user input with the defined plurality of physical models and the physical laws.

7. The method according to claim 1, wherein the predicted interaction comprises a collision of at least two respective objects, and the received user input avoids the collision.

8. The method according to claim 1, wherein the user input comprises a modification of at least one motion vector of at least one respective object.

9. The method according to claim 1, wherein the user input comprises a modification of at least one acceleration of at least one respective object.

10. The method according to claim 1, wherein said processing comprises recognizing a respective object in the at least one stream of live action video images, performing a database lookup for respective object physical model, and parameterizing the respective object physical model based on physical parameters derived from the at least one stream of live action video images.

11. The method according to claim 10, wherein the recognized respective object is a respective player in a term sport game, and said performing a database lookup comprises accessing an Internet database representing characteristics of the respective player.

12. The method according to claim 11, wherein the characteristics of the respective player comprise team sport game performance statistics.

13. The method according to claim 10, wherein the respective object physical model comprises a texture mapping model.

14. The method according to claim 1, wherein the synthetic stream of action video images represent at least the hidden surface extrapolation with respect to the at least one stream of live action video images.

15. The method according to claim 1, wherein the processing of the at least one stream of live action video images is performed on a single-instruction multiple data processor.

16. The method according to claim 1, wherein the generating the synthetic stream of action video images comprises generating a synthetic perspective view of the physical contact between at least two respective objects in the game different from a perspective represented in the at least one stream of live action video images.

17. The method according to claim 1, wherein the processing the at least one stream of live action video images to define the plurality of respective physical models comprises inferring a velocity, an acceleration, and a set of mass relationships of the plurality of respective objects, and defining a set of collision parameters for the at least one predicted interaction.

18. The method according to claim 1, wherein the processing the at least one stream of live action video images to define the plurality of respective physical models comprises analyzing a respective object represented in a plurality of streams of live action video images at the same time from at least two different perspectives to assess at least a position, a velocity, and an acceleration of the respective object.

19. A method, comprising:

receiving at least one stream of live action video images of a game;

identifying a plurality of objects represented in the at least one stream of live action video images of the game;

performing a database lookup for a plurality of respective identified objects of respective physical models of the identified plurality of objects;

determining a motion-dependent outcome probability statistic for at least one of the plurality of respective identified objects;

processing the at least one stream of live action video images to parameterize the plurality of respective physical models, to describe dynamic interactions of the identified plurality of objects, each respective parameterized model representing at least a surface configuration, a mass, and a movement of a respective object;

determining game logic dependent on at least the motion-dependent outcome;

receiving a user input representing a change in prospective dynamic interactions, to alter a predicted path of at least one object and at least one predicted interaction comprising contact between at least two objects; and

generating a synthetic stream of action video images having altered dynamic interactions comprising at least one altered contact condition between the identified plurality of objects, dependent on the respective parameterized plurality of physical models, the determined game logic, and the received user input.

20. A computer readable medium, storing therein nontransitory instructions for controlling a programmable processor, comprising:

instructions for processing at least one stream of live action video images of a game to parameterize a plurality of physical models of a respective plurality of objects in the video, the plurality of physical models representing, for each modelled object, at least a surface configuration, a mass, and a motion, said processing comprising hidden surface extrapolation;

instructions for modelling dynamic interactions between the plurality of objects which interact over time in the at least one stream of live action video images, comprising at least one contact state between at least two objects, consistent with physical laws;

instructions for determining a motion-dependent outcome probability statistic for at least one of the plurality of objects;

instructions for defining game logic of the game dependent on at least the motion-dependent outcome;

instructions for receiving a user input representing a change in at least one dynamic interaction with respect to the at least one stream of live action video images, to alter a predicted path of at least one object and an interaction between at least two objects and change the at least one contact state between at least two objects; and

instructions for generating a synthetic stream of action video images, dependent on the parameterized plurality of physical models, the modelled dynamic interactions, the game logic, and the user input.