System and method for gesture interface control

• US 9,111,138 B2
• Filed: 11/30/2010
• Issued: 08/18/2015
• Est. Priority Date: 11/30/2010
• Status: Active Grant

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Inventors: Kirby, Shaun K., Evans, David K.
• Primary Examiner(s): Lao, Lun-Yi
• Assistant Examiner(s): KHAN, IBRAHIM A
• Application Number:
  US12/957,116
• Publication Number:
  US 20120133580A1
• Time in Patent Office: 1,722 Days
• Field of Search: 345156-168, 382/103, 382/170, 715/863
• US Class Current: 1/1
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06V 40/113 Recognition of static hand signs

Abstract

A method is provided in one example and includes generating a histogram associated with at least one object; receiving image data; comparing the image data to the histogram in order to determine if at least a portion of the image data corresponds to the histogram; identifying a pose associated with the object; and triggering an electronic command associated with the pose. In more particular embodiments, the image data is evaluated in order to analyze sequences of poses associated with a gesture that signals the electronic command to be performed.

Claims

1. A method, comprising:

generating a histogram of a hand associated with a user of a system, wherein the system responds to system interface electronic commands issued by the user using the hand;

receiving image data;

comparing the image data to the histogram using a tunable process to identify a cluster of hand colored pixels that are most likely a hand region in the image data;

computing an outer contour of the cluster of hand colored pixels to identify a portion of the image data that reflects the hand;

computing a convex hull for the hand reflected in the identified portion of the image data, wherein the convex hull shows a convexity of contours of the hand;

computing a foreshortened bounding box of the convex hull that includes the hand reflected in the identified portion of the image data;

analyzing convexity defects associated with the hand within the foreshortened bounding box to identify a pose associated with the hand reflected in the portion of the image data; and

if the pose is associated with a system interface electronic command, triggering the system interface electronic command associated with the pose, such that the system performs a function specified by the user.

2. The method of claim 1, wherein the image data is evaluated in order to analyze sequences of poses associated with a gesture that signals the system interface electronic command to be performed.

3. The method of claim 1, wherein analyzing the convexity defects includes designating a center of the foreshortened bounding box as a position of a pose of the hand.

4. The method of claim 1, wherein computing the foreshortened bounding box includes foreshortening a bounding box along an axis of a forearm to focus on the hand reflected in the identified portion of the image data, wherein a polar angle with a long axis of the bounding box is evaluated to determine which end of the bounding box represents the hand.

5. The method of claim 1, wherein comparing the image data to the histogram using a tunable process to identify a cluster of hand colored pixels that are most likely the hand region includes:

translating the image data to Hue-Saturation-Value (HSV) color space;

determining backspace of the image data in HSV color space based on the captured histogram to identify likely hand regions in the image data; and

performing a tunable filtering process, a tunable erosion process, and a tunable dilation process to identify the most likely hand region in the image data from the likely hand regions.

6. The method of claim 1, wherein generating includes prompting the user to position the hand such that a stencil coincides with the hand, and adding Hue-Saturation-Value (HSV) tuples of stencil regions to the histogram.

7. The method of claim 1, wherein analyzing the convexity defects associated with the hand within the foreshortened bounding box to identify the pose associated with the hand reflected in the portion of the image data includes comparing the convexity defects to a tunable parameter.

8. Logic encoded in one or more non-transitory tangible media that includes code for execution and when executed by a processor is operable to perform operations comprising:

generating a histogram of a hand associated with a user of a system, wherein the system responds to system interface electronic commands issued by the user using the hand;

receiving image data;

comparing the image data to the histogram using a tunable process to identify a cluster of hand colored pixels that are most likely a hand region in the image data;

computing an outer contour of the cluster of hand colored pixels to identify a portion of the image data that reflects the hand;

computing a convex hull for the hand reflected in the identified portion of the image data, wherein the convex hull shows a convexity of contours of the hand;

computing a foreshortened bounding box of the convex hull that includes the hand reflected in the identified portion of the image data;

analyzing convexity defects associated with the hand within the foreshortened bounding box to identify a pose associated with the hand reflected in the portion of the image data; and

if the pose is associated with a system interface electronic command, triggering the system interface electronic command associated with the pose, such that the system performs a function specified by the user.

9. The logic of claim 8, wherein the image data is evaluated in order to analyze sequences of poses associated with a gesture that signals the system interface electronic command to be performed.

10. The logic of claim 8, wherein analyzing the convexity defects includes designating a center of the foreshortened bounding box as a position of a pose of the hand.

11. The logic of claim 8, wherein computing the foreshortened bounding box includes foreshortening a bounding box along an axis of a forearm to focus on the hand reflected in the identified portion of the image data, wherein a polar angle with a long axis of the bounding box is evaluated to determine which end of the bounding box represents the hand.

12. The logic of claim 11, wherein comparing the image data to the histogram using a tunable process to identify a cluster of hand colored pixels that are most likely the hand region includes:

translating the image data to Hue-Saturation-Value (HSV) color space;

determining backspace of the image data in HSV color space based on the captured histogram to identify likely hand regions in the image data; and

performing a tunable filtering process, a tunable erosion process, and a tunable dilation process to identify the most likely hand region in the image data from the likely hand regions.

13. The logic of claim 8, wherein the generating includes prompting the user to position the hand such that a stencil coincides with the hand, and adding Hue-Saturation-Value (HSV) tuples of stencil regions to the histogram.

14. The logic of claim 8, analyzing the convexity defects associated with the hand within the foreshortened bounding box to identify the pose associated with the hand reflected in the portion of the image data includes comparing the convexity defects to a tunable parameter.

15. An apparatus, comprising:

a memory element configured to store data; and

a processor operable to execute instructions associated with the data, wherein the processor and the memory element cooperate such that the apparatus is configured to;

generate a histogram of a hand associated with a user of a system, wherein the system responds to system interface electronic commands issued by the user using the hand;

receive image data;

compare the image data to the histogram using a tunable process to identify a cluster of hand colored pixels that are most likely a hand region in the image data;

compute an outer contour of the cluster of hand colored pixels to identify a portion of the image data that reflects the hand;

compute a convex hull for the hand reflected in the identified portion of the image data, wherein the convex hull shows a convexity of contours of the hand;

compute a foreshortened bounding box of the convex hull that includes the hand reflected in the identified portion of the image data;

analyze convexity defects associated with the hand within the foreshortened bounding box to identify a pose associated with the hand reflected in the portion of the image data; and

if the pose is associated with a system interface electronic command, trigger the system interface electronic command associated with the pose, such that the system performs a function specified by the user.

16. The apparatus of claim 15, wherein the image data is evaluated in order to analyze sequences of poses associated with a gesture that signals the system interface electronic command to be performed.

17. The apparatus of claim 15, wherein analyzing the convexity defects includes designating a center of the foreshortened bounding box as a position of a pose of the hand.

18. The apparatus of claim 15, wherein computing the foreshortened bounding box includes foreshortening a bounding box along an axis of a forearm to focus on the hand reflected in the identified portion of the image data, wherein a polar angle with a long axis of the bounding box is evaluated to determine which end of the bounding box represents the hand.

19. The apparatus of claim 18, wherein comparing the image data to the histogram using a tunable process to identify a cluster of hand colored pixels that are most likely the hand region includes:

translating the image data to Hue-Saturation-Value (HSV) color space;

determining backspace of the image data in HSV color space based on the captured histogram to identify likely hand regions in the image data; and

performing a tunable filtering process, a tunable erosion process, and a tunable dilation process to identify the most likely hand region in the image data from the likely hand regions.

20. The apparatus of claim 15, further comprising:

a camera element configured to receive the image data, wherein the camera element is coupled to a display configured to render images during a communication session.