Efficient and accurate 3D object tracking
First Claim
Patent Images
1. A method of tracking a location of a face in an input image stream, said method comprising iteratively applying the steps of:
- (a) performing a computerized three-dimensional (3D) to two-dimensional (2D) rendering of a predefined textured 3D face model indicative of at least a portion of a face to produce a 2D rendered image including said face, the 3D-to-2D rendering being performed according to a previously predicted state vector derived from a previous tracking loop or a state vector from an initialisation step, wherein said state vector comprises the pose of said face;
(b) processing said 2D rendered image to extract a series of point features from the portion of the face in said 2D rendered image;
(c) localising corresponding point features in a current 2D image of said input image stream by comparing said 2D rendered image with said current 2D image;
(d) deriving a new state vector from said localised point feature in the input image stream, the new state vector being indicative of a current location of said face within said input image stream.
2 Assignments
0 Petitions
Accused Products
Abstract
A method of tracking an object in an input image stream, the method comprising iteratively applying the steps of: (a) rendering a three-dimensional object model according to a previously predicted state vector from a previous tracking loop or the state vector from an initialization step; (b) extracting a series of point features from the rendered object; (c) localizing corresponding point features in the input image stream; (d) deriving a new state vector from the point feature locations in the input image stream.
-
Citations
15 Claims
-
1. A method of tracking a location of a face in an input image stream, said method comprising iteratively applying the steps of:
-
(a) performing a computerized three-dimensional (3D) to two-dimensional (2D) rendering of a predefined textured 3D face model indicative of at least a portion of a face to produce a 2D rendered image including said face, the 3D-to-2D rendering being performed according to a previously predicted state vector derived from a previous tracking loop or a state vector from an initialisation step, wherein said state vector comprises the pose of said face; (b) processing said 2D rendered image to extract a series of point features from the portion of the face in said 2D rendered image; (c) localising corresponding point features in a current 2D image of said input image stream by comparing said 2D rendered image with said current 2D image; (d) deriving a new state vector from said localised point feature in the input image stream, the new state vector being indicative of a current location of said face within said input image stream. - View Dependent Claims (2, 3, 4, 5, 6)
-
-
7. A method of tracking an object in an input image stream, the method comprising steps of:
-
(i) creating a three-dimensional model of said object to be tracked; (ii) localising initial features points in an initial said input image stream; (iii) calculating an initial state vector indicative of a location of said object within said input image stream, wherein said initial state vector is calculated by minimising the square error between localised said initial features points and corresponding initial features points of said three-dimensional model projected into an image plane; (a) rendering said three-dimensional object model, wherein said object model accords with either a predicted state vector calculated in step (d) of a previous iteration or said initial state vector calculated in step (iii), wherein the rendering includes calculating a mask for said input image stream to distinguish between background and foreground pixels; (b) calculating a predefined number of point features from said object, wherein a corresponding predefined number of locations having highest edginess is selected as features from an image of said input image stream corresponding to the previous iteration for the following localisation step; (c) localising corresponding point features in said input image stream; (d) calculating a new state vector from localised said features points in said input image stream; and (e) iteratively performing steps (a) though (d) for providing at each iteration updated said new state vector from localised said features points. - View Dependent Claims (8, 9, 10)
-
-
11. A computer system including a non-transitory medium programmed with a set of executable instructions for carrying out a method of tracking a location of a face in an input image stream, said method comprising iteratively applying the steps of:
-
(a) performing a computerized three-dimensional (3D) to two-dimensional (2D) rendering of a predefined textured 3D face model indicative of at least a portion of a face to produce 2D a rendered image including said face, the 3D-to-2D rendering being performed according to a previously predicted state vector derived from a previous tracking loop or a state vector from an initialisation step, wherein said state vector comprises the pose of said face; (b) processing said 2D rendered image to extract series of point features from said face in said rendered 2D image; (c) localising corresponding point features in a current 2D image of said input image stream by comparing said 2D rendered image with said current 2D image; (d) deriving a new state vector from said localised point feature in the input image stream, the new state vector being indicative of a current location of said face within said input image stream.
-
-
12. A non-transitory, tangible computer-readable carrier medium carrying a set of instructions that when executed by one or more processors cause one or more processors to carry out a method of tracking a location of a face in an input image stream, said method comprising iteratively applying the steps of:
-
(a) performing a computerized three-dimensional (3D) to two-dimensional (2D) rendering of a textured 3D face model indicative of at least a portion of a face to produce a 2D rendered image including said face, the 3D-to-2D rendering being performed according to a previously predicted state vector derived from a previous tracking loop or a state vector from an initialisation step, wherein said state vector comprises the pose of said face; (b) processing said 2D rendered image to extract series of point features from said face in said rendered 2D image; (c) localising corresponding point features in a current 2D image of said input image stream by comparing the 2D rendered image with said current 2D image; (d) deriving a new state vector from said localised point feature in the input image stream, the new state vector being indicative of a current location of said face within said input image stream.
-
-
13. A system for tracking an object in an input image stream, the system comprising a processor adapted to receive an input image stream, said processor is further adapted to perform the steps of:
-
(i) creating a three-dimensional model of said object to be tracked; (ii) localising initial features points in an initial said input image stream; (iii) calculating an initial state vector indicative of a location of said object within said input image stream, wherein said initial state vector is calculated by minimising the square error between localised said initial features points and corresponding initial feature points of said three-dimensional model projected into an image plane; (a) rendering said three-dimensional object model, wherein said object model accords with either a predicted state vector calculated in step (d) of a previous iteration or said initial state vector calculated in step (iii), wherein the rendering includes calculating a mask for said input image stream to distinguish between background and foreground pixels; (b) calculating a predefined number of features points from said object, wherein a corresponding predefined number of locations having highest edginess is selected as features from an image of said input image stream corresponding to the previous iteration for the following localisation step; (c) localising corresponding point features in said input image stream; (d) calculating a new state vector from localised said features points in said input image stream; and (e) iteratively performing steps (a) though (d) for providing at each iteration updated said new state vector from localised said features points. - View Dependent Claims (14, 15)
-
Specification