Method, computer program and device for hybrid tracking of real-time representations of objects in a sequence

• US 20120201469A1
• Filed: 10/12/2010
• Published: 08/09/2012
• Est. Priority Date: 10/20/2009
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Total Immersion
• Inventors: Livet, Nicolas, Pasquier, Thomas, Chamoux, Jérémy
• Granted Patent: US 8,903,177 B2
• Time in Patent Office: Days
• US Class Current: 382/201
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/751 Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching

Abstract

The invention in particular relates to the hybrid tracking of representations of objects in a sequence of images using at least one key image. After acquiring a first and second images including a representation of the tracked object, a first image portion is identified in the first image, and a second image portion is retrieved from the key image. A relative pose of a first image portion of said second image, similar to the first image portion of the first image, is estimated. A second image portion of the first or second image, similar to the second image portion of the key image, is sought. The relative pose of the object is then estimated according to the relative poses of the first image portions and the second image portions.

Claims

1. A method of hybrid tracking a representation of at least one object in a sequence of images using at least one key image, the method being implemented on a computer, and comprising the following steps:

acquisition of a first and second images of said sequence of images including a representation of said at least one object;

identification of at least one first image portion of said first image, said at least one first image portion of said first image including at least one part of a representation of said at least one object;

retrieval of at least one second image portion from said at least one key image, said at least one second image portion of said key image including at least one part of a representation of said at least one object;

estimation of a relative pose of at least one first image portion of said second image, similar to said at least one first image portion of said first image, according to said at least one first image portion of said first image;

search (320, 760, 770) for at least one second image portion of said first or second image, similar to said at least one second image portion of said key image, and estimation (330, 725, 775) of a relative pose of said at least one object according to the relative poses of said first image portions and said second image portions.

2. The method according to claim 1 whereby said step of identifying said at least one first image portion of said first image includes a step retrieving (510) said at least one first image portion of said first image and whereby said step of estimating said relative pose of said at least one first image portion of said second image includes a step of searching (310, 710, 710′

), in said second image, for said at least one first image portion of said second image, similar to said at least one first image portion of said first image.

3. The according to claim 2 wherein said step of searching for said at least one first image portion of said second image similar to said at least one first image portion of said first image implements a search algorithm of the KLT type.

4. The method according to claim 1 wherein said at least one second image portion is retrieved from said at least one key image according to at least one predetermined point of interest of said key image.

5. The method according to claim 4 also including a step of selecting (740, 740′

) said at least one point of interest.

6. The method according to claim 5 also including a step of validating (760) said at least one selected point of interest.

7. The method according to claim 1 wherein said step of searching for at least one second image portion of said first or second image, similar to said at least one second image portion of said key image, implements an algorithm of the KLT type.

8. The method according to claim 1 wherein said step of searching for at least one second image portion of said first or second image, similar to said at least one second image portion of said key image, includes a step of re-projecting said at least one second image portion of said key image according to an estimated pose of said at least one object.

9. The method according to claim 1 also including a step of determining at least one point of interest in said at least one second image.

10. The method according claim 9 wherein said step of searching for said at least one first image portion of said second image, similar to said at least one first image portion of said first image, implements at least two different algorithms to search for the image portion by similarity.

11. The method according to claim 1 in which said step of estimating a relative pose of at least one first image portion of said second image and/or said step of searching for at least one second image portion of said first or second image include a step of predicting a pose using at least one result of the relative pose estimation of said at least one object carried out on preceding images.

12. A computer program stored on a computer readable storage medium, said computer program including instructions suitable for the implementation of each of the steps of the method according to claim 1 when said program is executed on a computer.

13. A device including means suitable for the implementation of each of the steps of the method according to claim 1.