Hand detection, location, and/or tracking

• US 9,208,580 B2
• Filed: 02/26/2013
• Issued: 12/08/2015
• Est. Priority Date: 08/23/2012
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Gu, Jin
• Primary Examiner(s): NAKHJAVAN, SHERVIN K
• Application Number:
  US13/777,928
• Publication Number:
  US 20140056472A1
• Time in Patent Office: 1,015 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 18/21342 using statistical independence, i.e. minimising mutual information or maximising non-gaussianity
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters
• CPC Class Codes: G06V 10/26 Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
• CPC Class Codes: G06V 10/7715 Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

Various arrangements for identifying a location of a hand of a person are presented. A group of pixels may be identified in an image of a scene as including the person. A reference point may be set for the group of pixels identified as the person. The hand may be identified as using a local distance maximum from the reference point. An indication, such as coordinates, of the location of the hand may be output based on the local distance.

Claims

1. A method for identifying a location of a hand of a person, comprising:

identifying a group of pixels in an image of a scene as representing the person;

defining a plane positioned and oriented based on three-dimensional coordinates of the group of pixels identified as representing the person;

setting a reference point for the group of pixels identified as representing the person;

identifying a local distance maximum from the reference point within the group of pixels identified as representing the person based on the position and orientation of the plane; and

outputting an indication of the location of the hand of the person based on the identified local distance maximum.

2. The method for identifying the location of the hand of the person of claim 1, wherein identifying the location of the hand of the person based on the local distance maximum from the reference point within the group of pixels identified as representing the person comprises ignoring one or more additional local distance maximums that are within a threshold distance of the plane.

3. The method for identifying the location of the hand of the person of claim 1, wherein identifying the group of pixels in the image as including the person comprises:

performing a principal component analysis on the group of pixels to identify the group of pixels as representing the person based on a presence of a shape resembling a head and shoulders in the group of pixels.

4. The method for identifying the location of the hand of the person of claim 1, further comprising:

prior to identifying the group of pixels in the image as representing the person, identifying a plurality of pixels of the image of the scene as background, wherein the plurality of pixels of the image are not used when identifying the group of pixels as representing the person.

5. The method for identifying the location of the hand of the person of claim 1, further comprising:

creating a foreground model for a pixel based on the pixel being in the group of pixels, wherein the foreground model indicates a depth and intensity.

6. The method for identifying the location of the hand of the person of claim 1, further comprising:

identifying a second group of pixels in the image of the scene; and

excluding the second group of pixels from being identified as representing any person based on a size of the second group of pixels.

7. The method for identifying the location of the hand of the person of claim 1, wherein the indication of the location of the hand comprises three dimensional coordinates.

8. The method for identifying the location of the hand of the person of claim 1, wherein identifying the group of pixels in the image of the scene as representing the person comprises:

analyzing a history of pixel groups to determine that a first group of pixels and a second group of pixels are to be treated as the group of pixels.

9. The method for identifying the location of the hand of the person of claim 1, further comprising:

determining a second group of pixels in the image of the scene does not correspond to any person based on the second group of pixels being smaller than a predefined minimum size threshold.

10. The method for identifying the location of the hand of the person of claim 1, further comprising:

prior to identifying the group of pixels in the image as representing the person, receiving the image of the scene, wherein each pixel of the image has depth data and intensity data.

11. A system for identifying a location of a hand of a person, the system comprising:

a processor; and

a memory communicatively coupled with and readable by the processor and having stored therein processor-readable instructions which, when executed by the processor, cause the processor to;

identify a group of pixels in an image of a scene as representing the person;

set a reference point for the group of pixels identified as representing the person, wherein the reference point lies within a plane that is definable based on three-dimensional coordinates of the group of pixels identified as representing the person;

identify a local distance maximum from the reference point within the group of pixels identified as representing the person; and

output an indication of the location of the hand of the person based on the identified local distance maximum.

12. The system for identifying the location of the hand of the person of claim 11, wherein the processor-readable instructions which, when executed by the processor, cause the processor to identify the location of the hand of the person based on the local distance maximum from the reference point within the group of pixels identified as representing the person comprises processor-readable instructions which, when executed by the processor, cause the processor to:

ignore one or more additional local distance maximums that are within a threshold distance of the plane.

13. The system for identifying the location of the hand of the person of claim 11, wherein the processor-readable instructions which, when executed by the processor, cause the processor to identify the group of pixels in the image as including the person comprises processor-readable instructions which, when executed by the processor, cause the processor to:

perform a principal component analysis on the group of pixels to identify the group of pixels as representing the person based on a presence of a shape resembling a head and shoulders in the group of pixels.

14. The system for identifying the location of the hand of the person of claim 11, wherein the processor-readable instructions further comprise processor-readable instructions which, when executed by the processor, cause the processor to:

prior to identifying the group of pixels in the image as representing the person, identify a plurality of pixels of the image of the scene as background, wherein;

the plurality of pixels of the image are not used when identifying the group of pixels as representing the person.

15. The system for identifying the location of the hand of the person of claim 11, wherein the processor-readable instructions further comprise processor-readable instructions which, when executed by the processor, cause the processor to:

identify a second group of pixels in the image of the scene; and

exclude the second group of pixels from being identified as representing any person based on a size of the second group of pixels.

16. The system for identifying the location of the hand of the person of claim 11, wherein the indication of the location of the hand comprises three dimensional coordinates.

17. The system for identifying the location of the hand of the person of claim 11, wherein the processor-readable instructions further comprise processor-readable instructions which, when executed by the processor, cause the processor to:

prior to identifying the group of pixels in the image as representing the person, receive the image of the scene, wherein each pixel of the image has depth data and intensity data.

18. A non-transitory computer-readable medium having computer-readable instructions stored thereon, the computer-readable instructions being configured to cause a computer to:

identify a group of pixels in an image of a scene as representing a person;

set a reference point for the group of pixels identified as representing the person;

identify a local distance maximum from the reference point within the group of pixels identified as representing the person, wherein the reference point lies within a plane that is definable based on three-dimensional coordinates of the group of pixels identified as representing the person; and

output an indication of a location of a hand of the person based on the identified local distance maximum.

19. The non-transitory computer-readable medium of claim 18, wherein the computer-readable instructions configured to cause the computer to identify the location of the hand of the person based on the local distance maximum from the reference point within the group of pixels identified as representing the person comprises computer-readable instructions configured to cause the computer to:

ignore one or more additional local distance maximums that are within a threshold distance of the plane.

20. The non-transitory computer-readable medium of claim 18, wherein the computer-readable instructions configured to cause the computer to identify the group of pixels in the image as including the person comprises computer-readable instructions configured to cause the computer to:

perform a principal component analysis on the group of pixels to identify the group of pixels as representing the person based on a presence of a shape resembling a head and shoulders in the group of pixels.

21. The non-transitory computer-readable medium of claim 18, wherein the computer-readable instructions further comprise computer-readable instructions configured to cause the computer to:

prior to identifying the group of pixels in the image as representing the person, identify a plurality of pixels of the image of the scene as background, wherein;

the plurality of pixels of the image are not used when identifying the group of pixels as representing the person.

22. The non-transitory computer-readable medium of claim 18, wherein the computer-readable instructions further comprise computer-readable instructions configured to cause the computer to:

identify a second group of pixels in the image of the scene; and

exclude the second group of pixels from being identified as representing any person based on a size of the second group of pixels.

23. The non-transitory computer-readable medium of claim 18, wherein the indication of the location of the hand comprises three dimensional coordinates.

24. The non-transitory computer-readable medium of claim 18, wherein the computer-readable instructions further comprise computer-readable instructions configured to cause the computer to:

prior to identifying the group of pixels in the image as representing the person, receive the image of the scene, wherein each pixel of the image has depth data and intensity data.

25. An apparatus for identifying a location of a hand of a person, the apparatus comprising:

means for identifying a group of pixels in an image of a scene as representing the person;

means for setting a reference point for the group of pixels identified as representing the person, wherein the reference point lies within a plane that is definable based on three-dimensional coordinates of the group of pixels identified as representing the person;

means for identifying a local distance maximum from the reference point within the group of pixels identified as representing the person; and

means for outputting an indication of the location of the hand of the person based on the identified local distance maximum.

26. The apparatus for identifying the location of the hand of the person of claim 25, wherein the means for identifying the location of the hand of the person based on the local distance maximum from the reference point within the group of pixels identified as representing the person comprises:

means for ignoring one or more additional local distance maximums that are within a threshold distance of the plane.

27. The apparatus for identifying the location of the hand of the person of claim 25, wherein the means for identifying the group of pixels in the image as including the person comprises:

means for performing a principal component analysis on the group of pixels to identify the group of pixels as representing the person based on a presence of a shape resembling a head and shoulders in the group of pixels.

28. The apparatus for identifying the location of the hand of the person of claim 25, further comprising:

means for identifying a plurality of pixels of the image of the scene as background prior to identifying the group of pixels in the image as representing the person, wherein;

the plurality of pixels of the image are not used when identifying the group of pixels as representing the person.

29. The apparatus for identifying the location of the hand of the person of claim 25, further comprising:

means for identifying a second group of pixels in the image of the scene; and

means for excluding the second group of pixels from being identified as representing any person based on a size of the second group of pixels.

30. The apparatus for identifying the location of the hand of the person of claim 25, wherein the indication of the location of the hand comprises three dimensional coordinates.

31. The apparatus for identifying the location of the hand of the person of claim 25, further comprising:

means for receiving the image of the scene prior to identifying the group of pixels in the image as representing the person, wherein each pixel of the image has depth data and intensity data.

32. The apparatus for identifying the location of the hand of the person of claim 25, further comprising:

means for creating a foreground model for a pixel based on the pixel being in the group of pixels, wherein the foreground model indicates a depth and intensity.

33. The apparatus for identifying the location of the hand of the person of claim 25, further comprising:

means for determining a second group of pixels in the image of the scene does not correspond to any person based on the second group of pixels being smaller than a predefined minimum size threshold.

34. The apparatus for identifying the location of the hand of the person of claim 25, wherein the means for identifying the group of pixels in the image of the scene as representing the person comprises:

means for analyzing a history of pixel groups to determine that a first group of pixels and a second group of pixels are to be treated as the group of pixels.