Image Processor Comprising Gesture Recognition System with Static Hand Pose Recognition Based on First and Second Sets of Features

• US 20150253863A1
• Filed: 03/06/2015
• Published: 09/10/2015
• Est. Priority Date: 03/06/2014
• Status: Abandoned Application

• Current Assignee: Avago Technologies General IP PTE Limited (Broadcom, Inc.)
• Original Assignee: Avago Technologies General IP PTE Limited (Broadcom, Inc.)
• Inventors: Babin, Dmitry Nicolaevich, Mazurenko, Ivan Leonidovich, Petyushko, Alexander Alexandrovich, Letunovskiy, Aleksey Alexandrovich, Zaytsev, Denis Vladimirovich
• Application Number:
  US14/640,492
• Publication Number:
  US 20150253863A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 30/248 involving plural approaches, e.g. verification by template match; Resolving confusion among similar patterns, e.g. "O" versus "Q" G06V30/242 takes precedence
• CPC Class Codes: G06V 40/113 Recognition of static hand signs
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

An image processing system comprises an image processor having image processing circuitry and an associated memory. The image processor is configured to implement a gesture recognition system comprising a static pose recognition module. The static pose recognition module is configured to identify a hand region of interest in at least one image, to obtain a vocabulary of hand poses, to estimate a plurality of hand features based on the hand region of interest, the plurality of hand features comprising a first set of features estimated from the hand region of interest and a second set of features comprising at least one feature estimated using a transform on a contour of the hand region of interest, and to recognize a static pose of the hand region of interest based on the first set of features and the second set of features, wherein respective numbers of features in the first set of features and the second set of features are based at least in part on a size of the vocabulary of hand poses.

Claims

1. A method comprising steps of:

identifying a hand region of interest in at least one image;

obtaining a vocabulary of hand poses;

estimating a plurality of hand features based on the hand region of interest, the plurality of hand features comprising a first set of features estimated from the hand region of interest and a second set of features comprising at least one feature estimated using a transform on a contour of the hand region of interest; and

recognizing a static pose of the hand region of interest based on the first set of features and the second set of features, wherein respective numbers of features in the first set of features and the second set of features are based at least in part on a size of the vocabulary of hand poses;

wherein the steps are implemented in an image processor comprising a processor coupled to a memory.

2. The method of claim 1 wherein identifying the hand region of interest comprises generating a hand image comprising a binary region of interest mask in which pixels within the hand region of interest all have a first binary value and pixels outside the hand region of interest all have a second binary value complementary to the first binary value.

3. The method of claim 1 wherein recognizing the static pose of the hand region of interest comprises:

determining a set of candidate hand poses from a subset of the vocabulary of hand poses based on the first set of features; and

recognizing the static pose based on the set of candidate hand poses, the first set of features and the second set of features.

4. The method of claim 1 wherein the first set of features comprises two or more of an area of the hand region of interest, a perimeter of the hand region of interest, a width of the hand region of interest and a height of the hand region of interest.

5. The method of claim 4 wherein the first set of features further comprises one or more of a forefinger area of the hand region of interest, a wrist width of the hand region of interest and a forefinger width of the hand region of interest.

6. The method of claim 4 wherein the first set of features further comprises one or more of second-order centered moments or functions thereof for coordinates of pixels of the hand region of interest.

7. The method of claim 1 wherein the transform on the contour of the hand region of interest comprises one of a discrete cosine transform and a wavelet transform.

8. The method of claim 1 wherein said at least one feature estimated using the transform on the contour of the hand region of interest is estimated by:

identifying a center of the hand region of interest;

obtaining a vector by estimating respective distances from a subset of contour points of the hand region of interest to the center of the hand region of interest; and

transforming the vector to obtain a set of coefficients.

9. The method of claim 8 wherein identifying the center of the hand region of interest comprises one of:

identifying a center of mass of mask points in the hand region of interest;

identifying a center of a largest inscribed circle in the hand region of interest; and

identifying a pair of points (m_x, m_y) in a Cartesian coordinate system comprising an x-axis and a y-axis wherein m_x corresponds to an x-coordinate center of the largest inscribed circle in the hand region of interest and m_y is calculated by subtracting a constant value dy from ybottom, where dy corresponds to a fixed constant based on a distance between the hand and the camera capturing said at least one image and ybottom is a y-coordinate of a lowest row of pixels in the hand region of interest.

10. The method of claim 8 further comprising determining the subset of contour points by one of:

estimating a contour perimeter of the hand region of interest and selecting points on the contour such that respective distances between adjacent points in the subset is based on the contour perimeter and a predetermined number of contour points to be included in the subset; and

tracking points of the contour and selecting ones of the points of the contour to include in the subset such that respective distances between adjacent points in the subset are approximately equal to a predefined constant step.

11. The method of claim 10 wherein tracking points of the contour comprises tracking a circuit of the contour of the hand region of interest and, if a full circuit gives less than the predetermined number of contour points, appending the subset with (0, 0) points until the number of contour points in the subset is equal to the predetermined number.

12. The method of claim 8 wherein the second set of features further comprises a residual feature determined by processing the set of coefficients obtained by transforming the vector.

13. The method of claim 12 wherein the set of coefficients are associated with respective indices and wherein processing the set of coefficients comprises, for each index:

replacing a tail of the set of coefficients with zeros to obtain a truncated vector of the set of coefficients;

applying an inverse transform to the truncated vector; and

estimating a difference between the vector and the truncated vector using a distance metric.

14. The method of claim 13 wherein the distance metric comprises one of a Euclidean distance metric, a Manhattan distance metric and a maximum absolute difference metric.

15. (canceled)

16. An apparatus comprising:

an image processor comprising image processing circuitry and an associated memory;

wherein the image processor is configured to implement a gesture recognition system utilizing the image processing circuitry and the memory, the gesture recognition system comprising a static pose recognition module; and

wherein the static pose recognition module is configured;

to identify a hand region of interest in at least one image;

to obtain a vocabulary of hand poses;

to estimate a plurality of hand features based on the hand region of interest, the plurality of hand features comprising a first set of features estimated from the hand region of interest and a second set of features comprising at least one feature estimated using a transform on a contour of the hand region of interest; and

to recognize a static pose of the hand region of interest based on the first set of features and the second set of features, wherein respective numbers of features in the first set of features and the second set of features are based at least in part on a size of the vocabulary of hand poses.

17. The apparatus of claim 16 wherein the static pose recognition module is configured to recognize the static pose of the hand region of interest by determining a set of candidate hand poses from a subset of the vocabulary of hand poses based on the first set of features and to recognize the static pose based on the set of candidate hand poses, the first set of features and the second set of features.

18. The apparatus of claim 16 wherein the transform on the contour of the hand region of interest comprises one of a discrete cosine transform and a wavelet transform.

21. The apparatus of claim 16 wherein the first set of features comprises two or more of an area of the hand region of interest, a perimeter of the hand region of interest, a width of the hand region of interest and a height of the hand region of interest.

22. The apparatus of claim 21 wherein the first set of features further comprises one or more of a forefinger area of the hand region of interest, a wrist width of the hand region of interest and a forefinger width of the hand region of interest.

23. The apparatus of claim 21, wherein the first set of features further comprises one or more of second-order centered moments or functions thereof for coordinates of pixels of the hand region of interest.

19. (canceled)

20. (canceled)