Method and system for generating a 3D representation of a dynamically changing 3D scene
First Claim
Patent Images
1. A method for generating a 3D representation of a dynamically changing 3D scene including human players, comprising the steps of:
- a) acquiring at least two video streams from at least two cameras located at different locations, observing the same 3D scene, and outputting texture data of said at least two video streams for all frames, wherein each of the at least two cameras are side view cameras which do not look down at a surface of a playing field in a vertical direction;
b) determining camera parameters, which comprise the position, orientation and internal parameters, for said at least two cameras;
c) tracking the 2D movement of players and determining 2D shapes of the players in the at least two video streams based only on the texture data of said at least two video streams, wherein the 2D shapes of players is determined by evaluating the texture data and defining an area that includes at least one of a player'"'"'s position and size;
d) determining the 3D position of the players by combining information from the at least two video streams to generate a 3D computational model of the 3D scene which includes data for rendering a synthesized view of the 3D scene from a virtual viewpoint that is distinct from viewpoints associated with the locations of the at least two cameras; and
e) computing extrapolated 2D positions of the players in the at least two video streams from the 3D positions of the players;
wherein the step of tracking the 2D movement of the players in the at least two video streams and determining the 2D shapes of the players in the at least two video streams uses said computed extrapolated 2D positions of the players in the at least two video streams that stem from one or more earlier instants in time.
3 Assignments
0 Petitions
Accused Products
Abstract
A method for generating a 3D representation of a dynamically changing 3D scene, which includes the steps of:
- acquiring at least two synchronised video streams (120) from at least two cameras located at different locations and observing the same 3D scene (102);
- determining camera parameters, which comprise the orientation and zoom setting, for the at least two cameras (103);
- tracking the movement of objects (310a,b, 312a,b; 330a,b, 331a,b, 332a,b; 410a,b, 411a,b; 430a,b, 431a,b; 420a,b, 421a,b) in the at least two video streams (104);
- determining the identity of the objects in the at least two video streams (105);
- determining the 3D position of the objects by combining the information from the at least two video streams (106);
- wherein the step of tracking (104) the movement of objects in the at least two video streams uses position information derived from the 3D position of the objects in one or more earlier instants in time.
As a result, the quality, speed and robustness of the 2D tracking in the video streams is improved.
-
Citations
27 Claims
-
1. A method for generating a 3D representation of a dynamically changing 3D scene including human players, comprising the steps of:
-
a) acquiring at least two video streams from at least two cameras located at different locations, observing the same 3D scene, and outputting texture data of said at least two video streams for all frames, wherein each of the at least two cameras are side view cameras which do not look down at a surface of a playing field in a vertical direction; b) determining camera parameters, which comprise the position, orientation and internal parameters, for said at least two cameras; c) tracking the 2D movement of players and determining 2D shapes of the players in the at least two video streams based only on the texture data of said at least two video streams, wherein the 2D shapes of players is determined by evaluating the texture data and defining an area that includes at least one of a player'"'"'s position and size; d) determining the 3D position of the players by combining information from the at least two video streams to generate a 3D computational model of the 3D scene which includes data for rendering a synthesized view of the 3D scene from a virtual viewpoint that is distinct from viewpoints associated with the locations of the at least two cameras; and e) computing extrapolated 2D positions of the players in the at least two video streams from the 3D positions of the players; wherein the step of tracking the 2D movement of the players in the at least two video streams and determining the 2D shapes of the players in the at least two video streams uses said computed extrapolated 2D positions of the players in the at least two video streams that stem from one or more earlier instants in time. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23)
-
-
24. A data processing system for generating a 3D representation of a dynamically changing 3D scene, the system comprising:
a computer with a processor, memory, and input output devices, and the data processing system further comprising; a) a data acquisition module acquiring at least two video streams from at least two cameras located at different locations, observing the same 3D scene, and outputting texture data of said at least two video streams for all frames, wherein each of the at least two cameras are side view cameras which do not look down at a surface of a playing field in a vertical direction; b) a camera calibration module for determining camera parameters, which comprise the position, orientation and internal parameters, for said at least two cameras; c) a 2D tracking module that is configured to track the 2D movement of players and determine 2D shapes of the players in the at least two video streams based only on the texture data of said at least two video streams by evaluating the texture data and defining an area that includes at least one of a player'"'"'s position and size; d) a 3D merging and 3D object position calculation module for determining the 3D position of the players by combining the information determined from the at least two video streams to generate a 3D computational model of the 3D scene which includes data for rendering a synthesized view of the 3D scene from a virtual viewpoint that is distinct from viewpoints associated with the locations of the at least two cameras; and e) an extrapolation module for computing extrapolated 2D positions of the players in the at least two video streams from the 3D positions of the players; wherein the 2D tracking module is configured to use extrapolated 2D positions of the players in the at least two video streams that stem from one or more earlier instants in time, and wherein the computer comprises at least one of;
the data acquisition module, the camera calibration module, the 2D tracking module, the 3D merging and 3D object position calculation module, and the extrapolation module.- View Dependent Claims (25, 26, 27)
Specification