Efficient And Accurate 3D Object Tracking

• US 20090324018A1
• Filed: 09/04/2009
• Published: 12/31/2009
• Est. Priority Date: 03/05/2007
• Status: Active Grant

• Current Assignee: Seeing Machines LTD
• Original Assignee: Seeing Machines Pty Limited
• Inventors: Tell, Dennis
• Granted Patent: US 8,848,975 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships

Abstract

A method of tracking an object in an input image stream, the method comprising iteratively applying the steps of: (a) rendering a three-dimensional object model according to a previously predicted state vector from a previous tracking loop or the state vector from an initialisation step; (b) extracting a series of point features from the rendered object; (c) localising corresponding point features in the input image stream; (d) deriving a new state vector from the point feature locations in the input image stream.

Claims

1. A method of tracking an object in an input image stream, said method comprising iteratively applying the steps of:

(a) rendering a three-dimensional object model according to a previously predicted state vector from a previous tracking loop or a state vector from an initialisation step;

(b) extracting a series of point features from said rendered object;

(c) localising corresponding point features in said input image stream;

(d) deriving a new state vector from said localised point feature in the input image stream.

2. A method as claimed in claim 1 further comprising an initialisation step of

(i) creation of a three-dimensional model of said object to be tracked;

(ii) initial detection of said objects position within said input image stream.

3. A method as claimed in claim 1 wherein said step (d) further includes the step of selecting a set of consistent features followed by Kalman filtering of said features for deriving said new state vector.

4. A method as claimed in claim 1 wherein said object comprises a face.

5. A method as claimed in claim 4 wherein said state vector comprises the three-dimensional pose of said face.

6. A method as claimed in claim 3 wherein the selecting of a set of features comprises random sample consensus (RANSAC) selection of said features.

7. A method as claimed in claim 1 wherein said localising step includes utilising a normalised cross-correlation process to localise said point features.

12. A computer program or computer program product comprising a set of instructions for carrying out a method according to claim 1.

13. A computer-readable carrier medium carrying a set of instructions that when executed by one or more processors cause one or more processors to carry out a method according to claim 1.

8. A method of tracking an object in an input image stream, the method comprising steps of:

(i) creating a three-dimensional model of said object to be tracked;

(ii) localising initial features points in an initial said input image stream;

(iii) calculating an initial state vector indicative of said object location within said input image stream, wherein said initial state vector is calculated by minimising the square error between said initial localised feature points and corresponding initial feature points of said three-dimensional model projected into the image plane;

(a) rendering a three-dimensional object model, wherein said object model accords with either said predicted state vector calculated in step (d) of a previous iteration or said initial state vector calculated in step (ii), wherein said rendering includes calculating a mask for said input image stream to distinguishing between background and foreground pixels;

(b) calculating a predefined number of point features from said rendered object, wherein said predefined number of locations having highest edginess are selected as features from said rendered image of the previous iteration for the following localisation step;

(c) localising corresponding point features in said input image stream;

(d) calculating a new state vector from said localised point features in said input image stream; and

(e) iteratively performing steps (a) though (d) for providing at each iteration updated said new state vector from said localised point features.

9. A method according to claim 8, wherein said localisation step uses a masked normalised template correlation to calculate new said localised point features.

10. A method according to claim 8, wherein said new state vector is calculated by using Random Sample Consensus selection and followed by a Kalman filter for estimating said new state vector.

11. A method according to claim 10, wherein said Kalman filter utilises a constant acceleration motion model of said object to predict said new state vector of said object from previously calculated state vectors and said new localised point features.

14. A system for tracking an object in an input image stream, the system comprising a processor adapted to receive an input image stream, said processor is further adapted to perform the steps of:

(i) creating a three-dimensional model of said object to be tracked;

(ii) localising initial features points in an initial said input image stream;

(iii) calculating an initial state vector indicative of said object location within said input image stream, wherein said initial state vector is calculated by minimising the square error between said initial localised feature points and corresponding initial feature points of said three-dimensional model projected into the image plane;

(a) rendering a three-dimensional object model, wherein said object model accords with either said predicted state vector calculated in step (d) of a previous iteration or said initial state vector calculated in step (ii), wherein said rendering includes calculating a mask for said input image stream to distinguishing between background and foreground pixels;

(b) calculating a predefined number of point features from said rendered object, wherein said predefined number of locations having highest edginess are selected as features from said rendered image of the previous iteration for the following localisation step;

(c) localising corresponding point features in said input image stream;

(d) calculating a new state vector from said localised point features in said input image stream; and

(e) iteratively performing steps (a) though (d) for providing at each iteration updated said new state vector from said localised point features.

15. A system according to claim 14, wherein said processor is adapted to perform a masked normalised template correlation for calculating new said localised point features.

16. A system according to claim 14, wherein said processor is adapted to apply said new localised point features into a Kalman filter for calculating said new state vector.