IMAGE PROCESSING APPARATUS AND METHOD
First Claim
1. A method of generating a three dimensional representation of a scene, which includes a plurality of objects disposed on a plane, the three dimensional representation being generated from one or more video images of the scene, which video images include the objects on the plane produced from a view of the scene by a video camera, the method comprising:
- processing the captured video images so as to extract one or more image features from each object,comparing the one or more image features with sample image features from a predetermined set of possible example objects which the video images may contain,identifying the objects from the comparison of the image features with the stored image features of the possible example objects,generating object path data, which includes object identification data for each object which identifies the respective object; and
provides a position of the object on the plane in the video images with respect to time;
calculating a projection matrix for projecting the position of each of the objects according to the object path data from the plane into a three dimensional model of the plane, for generating the three dimensional representation of the scene, by projecting the position of the objects according to the object path data into the plane of the three dimensional model of the scene using the projection matrix.
1 Assignment
0 Petitions
Accused Products
Abstract
An image processing apparatus and method generate a three dimensional representation of a scene which includes a plurality of objects disposed on a plane. The three dimensional representation is generated from one or more video images of the scene, which include the objects on the plane produced from a view of the scene by a video camera. The method comprises processing the captured video images so as to extract one or more image features from each object, comparing the one or more image features with sample image features from a predetermined set of possible example objects which the video images may contain, and identifying the objects from the comparison of the image features with the stored image features of the possible example objects. The method also includes generating object path data, which includes object identification data for each object which identifies the respective object; and provides a position of the object on the plane in the video images with respect to time. The method further includes calculating a projection matrix for projecting the position of each of the objects according to the object path data from the plane into a three dimensional model of the plane. As such a three dimensional representation of the scene which includes a synthesised representation of each of the plurality of objects on the plane can be produced, by projecting the position of the objects according to the object path data into the plane of the three dimensional model of the scene using the projection matrix and a predetermined assumption of the height of each of the objects. Accordingly, a three dimensional representation of a live video image of, for example, a football match can be generated, or tracking information included on the live video images. As such, a change in a relative view of the generated three dimensional representation can be made, so that a view can be provided in the three dimensional representation of the scene from a view point at which no camera is actually present to capture video images of the live scene.
100 Citations
19 Claims
-
1. A method of generating a three dimensional representation of a scene, which includes a plurality of objects disposed on a plane, the three dimensional representation being generated from one or more video images of the scene, which video images include the objects on the plane produced from a view of the scene by a video camera, the method comprising:
-
processing the captured video images so as to extract one or more image features from each object, comparing the one or more image features with sample image features from a predetermined set of possible example objects which the video images may contain, identifying the objects from the comparison of the image features with the stored image features of the possible example objects, generating object path data, which includes object identification data for each object which identifies the respective object; and
provides a position of the object on the plane in the video images with respect to time;calculating a projection matrix for projecting the position of each of the objects according to the object path data from the plane into a three dimensional model of the plane, for generating the three dimensional representation of the scene, by projecting the position of the objects according to the object path data into the plane of the three dimensional model of the scene using the projection matrix. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
-
11. An image processing apparatus operable to generate a three dimensional representation of a scene which includes a plurality of objects disposed on a plane, the three dimensional representation being generated from one or more video images of the scene, which video images include the objects on the plane produced from a view of the scene by a video camera, the image processing apparatus comprising a data processor operable
to process the captured video images so as to extract one or more image features from each object, to compare the one or more image features with sample image features from a predetermined set of possible example objects which the video images may contain, to identify the objects from the comparison of the image features with the predetermined image features of the possible example objects, to generate object path data, which includes object identification data for each object which identifies the respective object; - and provides a position of the object on the plane in the video images with respect to time;
to calculate a projection matrix for projecting the position of each of the objects according to the object path data from the plane into a three dimensional model of the plane, for generating the three dimensional representation of the scene, by projecting the position of the objects according to the object path data into the plane of the three dimensional model of the scene using the projection matrix. - View Dependent Claims (12, 13, 14, 15, 16)
- and provides a position of the object on the plane in the video images with respect to time;
-
17. A method of enhancing video images of a scene, which includes a plurality of objects disposed on a plane produced from a view of the scene by a video camera, the method comprising:
-
processing the captured video images so as to extract one or more image features from each object, comparing the one or more image features with sample image features from a predetermined set of possible example objects which the video images may contain, identifying the objects from the comparison of the image features with the stored image features of the possible example objects, generating object path data, which includes object identification data for each object which identifies the respective object; and
provides a position of the object on a three dimensional model of the plane in the video images with respect to time;calculating a projection matrix for projecting the position of each of the objects according to the object path data from the plane into the three dimensional model of the plane, identifying each of the objects in the video images using the object path data with respect to the three dimensional model of the plane on which the objects are disposed, generating a graphical representation of an identifier which identifies each of the objects in the video images, and overlaying the identifier on the objects in the video images to the effect that one or more of the objects is provided with an identifier as the objects move on the plane in the video images.
-
-
18. An image processing apparatus operable to enhance video images of a scene, which includes a plurality of objects disposed on a plane produced from a view of the scene by a video camera, the image processing apparatus including a data processor operable:
-
to process the captured video images so as to extract one or more image features from each object, to compare the one or more image features with sample image features from a predetermined set of possible example objects which the video images may contain, to identify the objects from the comparison of the image features with the stored image features of the possible example objects, to generate object path data, which includes object identification data for each object which identifies the respective object; and
provides a position of the object on a three dimensional model of the plane in the video images with respect to time;to calculate a projection matrix for projecting the position of each of the objects according to the object path data from the plane into the three dimensional model of the plane, to identify each of the objects in the video images using the object path data with respect to the three dimensional model of the plane on which the objects are disposed, to generate a graphical representation of an identifier which identifies each of the objects in the video images, and to overlay the identifier on the objects in the video images to the effect that one or more of the objects is provided with an identifier as the objects move on the plane in the video images.
-
-
19. An apparatus for generating a three dimensional representation of a scene, which includes a plurality of objects disposed on a plane, the three dimensional representation being generated from one or more video images of the scene, which video images include the objects on the plane produced from a view of the scene by a video camera, the apparatus comprising:
-
means for processing the captured video images so as to extract one or more image features from each object, means for comparing the one or more image features with sample image features from a predetermined set of possible example objects which the video images may contain, means for identifying the objects from the comparison of the image features with the stored image features of the possible example objects, means for generating object path data, which includes object identification data for each object which identifies the respective object; and
provides a position of the object on the plane in the video images with respect to time;means for calculating a projection matrix for projecting the position of each of the objects according to the object path data from the plane into a three dimensional model of the plane, for generating the three dimensional representation of the scene, by projecting the position of the objects according to the object path data into the plane of the three dimensional model of the scene using the projection matrix.
-
Specification