Foreground action estimating apparatus and foreground action estimating method

• US 8,351,650 B2
• Filed: 10/08/2009
• Issued: 01/08/2013
• Est. Priority Date: 10/10/2008
• Status: Expired due to Fees

• Current Assignee: Sony Limited (Sony Group Corp.)
• Original Assignee: Sony Limited (Sony Group Corp.)
• Inventors: Wu, Weiguo, Liang, Li
• Primary Examiner(s): DULANEY, KATHLEEN YUAN
• Application Number:
  US12/575,902
• Publication Number:
  US 20100092074A1
• Time in Patent Office: 1,188 Days
• Field of Search: 382/103, 382/159, 382/173, 382/260, 715/863, 345/156, 725/12
• US Class Current: 382/103
• CPC Class Codes: G06V 40/107 Static hand or arm

Abstract

The present invention provides a foreground action estimating apparatus and a foreground action estimating method, wherein the foreground action estimating apparatus includes: a training image inputting means for inputting a foreground image, a background image and an image having the foreground and background images as training images; a basis matrix calculating means for calculating a foreground basis matrix and a background basis matrix by respectively extracting a foreground feature and a background feature from the foreground image and the background image, respectively, and combining the foreground basis matrix and the background basis matrix to obtain a combined basis matrix; a feature suppressing means for calculating the feature coefficients of the training images in accordance with the combined basis matrix obtained by the basis matrix calculating means so as to obtain image features of the background-feature-suppressed training images; and a foreground action information acquiring means for estimating foreground action information in accordance with a feature mapping matrix from the image feature to an action information set, by using the background-feature-suppressed image features.

Claims

1. A foreground action estimating apparatus for estimating foreground actions in a picture comprising:

a training image inputting means for inputting a foreground image, a background image and an image having the foreground and background images as training images;

a basis matrix calculating means for calculating a foreground basis matrix and a background basis matrix by respectively extracting a foreground feature and a background feature from the foreground image and the background image, respectively, and combining the foreground basis matrix and the background basis matrix to obtain a combined basis matrix;

a feature suppressing means for calculating feature coefficients of the training images in accordance with the combined basis matrix obtained by the basis matrix calculating means to obtain background-feature-suppressed image features of the training images; and

a foreground action information acquiring means for estimating foreground action information in accordance with a feature mapping matrix from the image features to an action information set, by using the background-feature-suppressed image features.

2. The foreground action estimating apparatus according to claim 1, wherein sizes of the foreground image, the background image, and the training images are uniform.

3. The foreground action estimating apparatus according to claim 1 further comprising a storage unit for storing the action information set,

wherein the foreground image and the training images correspond to the same action in the action information set.

4. The foreground action estimating apparatus according to claim 3, wherein the feature suppressing means comprises:

a feature extracting unit for extracting the image features of the training images;

a calculating unit for calculating the feature coefficients of the image features in the combined basis matrix;

a separating unit for separating a background basis coefficient from the feature coefficients so as to obtain a foreground basis coefficient; and

a reestablishing unit for reestablishing the background-feature-suppressed image features of the training images by a predetermined algorithm by using the foreground basis coefficient and the foreground basis matrix.

5. The foreground action estimating apparatus according to claim 4, wherein in the reestablishing unit, the predetermined algorithm is Universal Matrix Multiplication.

6. The foreground action estimating apparatus according to claim 1, wherein the foreground basis matrix and the background basis matrix in the basis matrix calculating means are obtained at least via non-negative matrix factorization.

7. The foreground action estimating apparatus according to claim 1, wherein the foreground image includes at least a person and the foreground actions includes at least a posture.

8. The foreground action estimating apparatus according to any one of the preceding claims, wherein the foreground feature, the background feature, and the image features are statistical features of the foreground image, the background image, and training images in an edge direction and include at least a HoG feature and a SIFT feature.

9. A foreground action estimating method for estimating foreground actions in a picture comprising the following steps:

a training image inputting step for inputting a foreground image, a background image and an image having the foreground and background images as training images;

a basis matrix calculating step for calculating a foreground basis matrix and a background basis matrix by respectively extracting a foreground feature and a background feature from the foreground image and the background image, respectively, and combining the foreground basis matrix and the background basis matrix to obtain a combined basis matrix;

a feature suppressing step for calculating the feature coefficients of the training images in accordance with the combined basis matrix obtained in the basis matrix calculating step to obtain image features of the background-feature-suppressed training images; and

a foreground action information acquiring step for estimating foreground action information in accordance with a feature mapping matrix from the image features to an action information set, by using the background-feature-suppressed image features.

10. The foreground action estimating method according to claim 9, wherein sizes of the foreground image, the background image, and the training images are uniform.

11. The foreground action estimating method according to claim 9 further comprising an action information set storing step,

wherein the foreground image and the training images correspond to the same action in the action information set.

12. The foreground action estimating method according to claim 11, wherein the feature suppressing step executes the processing of:

extracting the image features of the training images;

calculating the feature coefficients of the image features in the combined basis matrix;

separating a background basis coefficient from the feature coefficients so as to obtain a foreground basis coefficient; and

reestablishing the background-feature-suppressed image features of the training images by a predetermined algorithm via the foreground basis coefficient and the foreground basis matrix.

13. The foreground action estimating method according to claim 12, wherein the predetermined algorithm is Universal Matrix Multiplication.

14. The foreground action estimating method according to claim 9, wherein in the basis matrix calculating step, the foreground basis matrix and the background basis matrix are obtained at least via non-negative matrix factorization.

15. The foreground action estimating method according to claim 9, wherein the foreground image includes at least a person and the foreground actions includes at least a posture.

16. The foreground action estimating method according to any one of claims 9-15, wherein the foreground feature, the background feature, and the image features are statistical features of the foreground image, the background image, and training images in an edge direction and include at least a HoG feature and a SIFT feature.