Pose estimating device and pose estimating method

• US 7,940,960 B2
• Filed: 10/25/2007
• Issued: 05/10/2011
• Est. Priority Date: 10/27/2006
• Status: Active Grant

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation), Toshiba Digital Solutions Corporation (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Okada, Ryuzo
• Primary Examiner(s): Carter; Aaron W
• Application Number:
  US11/924,194
• Publication Number:
  US 20080152218A1
• Time in Patent Office: 1,293 Days
• Field of Search: 382/103, 382154-161, 706/12
• US Class Current: 382/103
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 18/24323 Tree-organised classifiers
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

A pose estimating device includes: a pose dictionary; an image feature extracting unit configured to extract observed image feature information; a past information storing unit configured to store past pose estimating information of the articulated object; a pose predicting unit configured to predict a present pose; a node predicting unit configured to calculate a prior probability as to whether each nodes includes a present pose; an identifying unit configured to calculate a likelihood of the observed image feature information for each node; a node probability calculating unit configured to calculate a probability in which the present pose belongs to the node in the upper layer; and a pose estimating unit configured to calculate pose information.

Claims

1. A pose estimating device comprising:

a dictionary memory configured to store a pose dictionary that includes;

training pose information related to various poses of an articulated object,

a tree structure of poses that consists of layered nodes, wherein similarity among the training pose information belonging to a lower node is higher than similarity among the training pose information belonging to an upper node,

a two class discriminative function for each node in upper layers that is upper than the lowermost layer in the tree structure, which determines on the basis of the training pose information and training image feature information whether or not a pose represented by image feature information belongs to the respective node, where the training image feature information is obtained by capturing virtual images of the articulated object of various sizes and various surface shapes each relating to the training pose information, and

a mapping function for each node of the lowermost layer in the tree structure, which transforms image feature information into pose information on the basis of the training image feature information and the training pose information belonging to the nodes;

an image feature extracting unit configured to extract observed image feature information from an image of the articulated object;

a past information storing unit configured to store past pose estimating information of the articulated object;

a pose predicting unit configured to predict a present pose on the basis of the past pose estimating information and a predetermined motion model;

a node predicting unit configured to calculate a prior probability as to whether or not each nodes in respective layers of the tree structure includes a pose of a correct solution corresponding to the present pose based on the predicted pose;

a two class identifying unit configured to calculate a likelihood of the observed image feature information for each node in the upper layers by using the two class discriminative function stored in the node;

a node probability calculating unit configured to calculate a probability in which the pose of the correct solution is in the node in the upper layers from the prior probability and the likelihood; and

a pose estimating unit configured to calculate pose information with respect to the lowermost layer by transforming the observed image feature information by the mapping functions stored in the pose dictionary.

2. The device according to claim 1, further comprising:

a two-class discriminative function learning unit configured to learn the two class discriminative function for each node in the upper layers using a set of sample data;

wherein the set of sample data consists of positive samples of the training image feature information whose corresponding training pose information belongs to the node of the two-class discriminative function to be learned and negative samples of the training image feature information whose corresponding training pose information belongs to the other nodes in the same layer of the tree structure.

3. The device according to claim 1, further comprising:

a mapping function learning unit configured to learn the mapping function;

wherein the mapping function learning unit learns the mapping function in accordance with a regression analysis by considering the training pose information included in a certain node and the training image feature information as a set of correct sample data.

4. The device according to claim 3,

wherein the set of correct sample data includes plural sets of the training pose information belonging to the respective nodes of the lowermost layer and the training image feature information;

and

wherein the training pose information of the respective nodes and the training image feature information are configured so that the plural sets of training pose information and the training image feature information do not include any two sets of the training pose information and the training image feature in which the training image feature information is similar to each other and the training pose information is greatly different from each other.

5. The device according to claim 3,

wherein the set of correct sample data includes plural sets of the training pose information belonging to the respective nodes of the lowermost layer and the training image feature information;

and

wherein the training pose information of the respective nodes and the training image feature information are configured so as to enable a relation between the training pose information and the training image feature information in the set of the correct sample data to be approximated by a linear function.

6. The device according to claim 1, further comprising:

a node reducing unit configured to determine a node for carrying out a calculation by the identifying unit and the pose estimating unit on the basis of the prior probability in the respective nodes and a probability that the pose of the correct solution is included in the respective nodes in the upper layer.

7. The device according to claim 1,

wherein the upper layer includes one or a plurality of layers.

8. The device according to claim 1,

wherein the tree structure is configured by the nodes corresponding to a set of the poses in which the similarity of the poses of the articulated object is larger than a threshold value, the threshold value set smaller as the layers get lower and set constant in the same layer;

and

wherein the nodes of the respective layers are connected to the nodes in the upper layers having the largest similarity.

9. The device according to claim 1, wherein the pose information includes an articulate angle of each part of the articulated object.

10. A pose estimating method using a pose dictionary, wherein the pose dictionary includes:

training pose information related to various poses of an articulated object,

a tree structure of poses that consists of layered nodes, wherein similarity among the training pose information belonging to a lower node is higher than similarity among the training pose information belonging to an upper node,

a two-class discriminative function for each node in upper layers that is upper than the lowermost layer in the tree structure, which determines on the basis of the training pose information and training image feature information whether or not a pose represented by image feature information belongs to the respective node, where the training image feature information is obtained by capturing virtual images of the articulated object of various sizes and various surface shapes each relating to the learning pose information, and a mapping function for each node of the lowermost layer in the tree structure, which transforms image feature information into pose information on the basis of the training image feature information and the training pose information belonging to the nodes;

the method comprising;

setting up the pose dictionary;

extracting observed image feature information from an image of the articulated object;

storing past pose estimating information of the articulated object;

predicting a present pose on the basis of the past pose estimating information and a predetermined motion model;

calculating a prior probability as to whether or not each nodes in respective layers of the tree structure includes a pose of a correct solution corresponding to the present pose based on the predicted pose;

calculating a likelihood of the observed image feature information for each node in the upper layer by using the two class discriminative function stored in the node;

calculating a probability in which the pose of the correct solution is in the node in the upper layer from the prior probability and the likelihood; and

calculating pose information with respect to the lowermost layer by transforming the observed image feature information by the mapping function stored in the pose dictionary.