Gestural interaction identification

• US 9,171,200 B2
• Filed: 03/04/2011
• Issued: 10/27/2015
• Est. Priority Date: 03/04/2011
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Subramanian, Anbumani, Hedge, Guruprasad, Madhvanath, Sriganesh
• Primary Examiner(s): Liew, Alex
• Application Number:
  US14/003,212
• Publication Number:
  US 20130343611A1
• Time in Patent Office: 1,698 Days
• Field of Search: 382/103, 382115-118
• US Class Current: 1/1
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

A method of identifying gestural interaction comprises detecting a user with an imaging device, detecting with the imaging device the depth value at the centroid of the user with respect to the imaging device, detecting with the imaging device the closest distance of the user with respect to the imaging device, and, with a processor, identifying the initiation of a gestural interaction based on the ratio of the closest distance and the depth value at the centroid of the user is above a predetermined threshold. A computer program product for identifying initiation and termination of gestural interaction within a gestural interaction system comprises a computer readable storage medium having computer usable program code embodied therewith, the computer usable program code comprising computer usable program code that identifies the initiation of a gestural interaction by a user depending on whether a virtual bubble around the user has been broken.

Claims

1. A method of identifying gestural interaction comprising:

detecting a user with an imaging device;

detecting with the imaging device the depth value at a centroid of the user with respect to the imaging device, comprising;

determining the centroid location of the user and the depth value of the centroid location;

detecting with the imaging device a closest distance of the user with respect to the imaging device, comprising;

determining a closest user feature within the image of the user; and

with a processor, identifying an initiation of a gestural interaction based on a ratio of the closest distance and the depth value at the centroid of the user is above a predetermined threshold.

2. The method of claim 1, in which detecting a user with an imaging device comprises:

imaging a scene comprising the user and images of objects to be removed with the imaging device, in which the imaging device comprises a color imaging device and a depth sensor;

sending a color image and a depth image captured by the imaging device to the processor;

converting the color image obtained from the imaging device into a grayscale image; and

with the processor, subtracting the background using the grayscale image and the depth image.

3. The method of claim 1, in which, if the ratio of the closest distance and the depth value at the centroid of the user is above the predetermined threshold, then, with the processor, initiating recognition of the user'"'"'s gestures as gestural interaction.

4. The method of claim 1, in which, if the ratio of the closest distance and the depth value at the centroid of the user is below the predetermined threshold, then not initiating recognition of the user'"'"'s gestures as gestural interaction.

5. The method of claim 1, in which, if the ratio of the closest distance and the depth value at the centroid of the user falls below the predetermined threshold, then terminating recognition of the user'"'"'s gestures as gestural interaction.

6. The method of claim 1, in which identifying the initiation of a gestural interaction is individually performed for each of a number of users detected by the imaging device.

7. A gestural interaction system comprising:

an imaging device that detects a depth value at a centroid of a user and a closest distance of the user relative to the imaging device, including determining the centroid location of the user, a depth value of the centroid location, and a closest user feature within the image of the user; and

a processor that identifies an initiation of a gestural interaction based on whether a ratio of the closest distance and a depth value at the centroid of the user is above a predetermined threshold.

8. The system of claim 7, in which the imaging device comprises:

a color imaging device; and

a depth sensor that determines the depth value of objects within the field of view of the imaging device.

9. The system of claim 7, further comprising a display device for displaying feedback to the user based on the user'"'"'s gestural interaction.

10. The system of claim 7, further comprising a data storage device communicatively coupled to the processor, in which the data storage device stores executable code that, when executed by the processor, causes the gestural interaction system to interact with the user.

11. A computer program product for identifying initiation and termination of gestural interaction within a gestural interaction system, the computer program product comprising:

a non-transitory computer readable storage medium having computer usable program code embodied therewith, the computer usable program code comprising;

computer usable program code that identifies an initiation of a gestural interaction by a user depending on whether a virtual bubble around the user has been broken;

computer usable program code that detects with an image device a user, a depth value at a centroid of the user with respect to the imaging device, and a closest distance of the user with respect to the imaging device, including determining the centroid location of the user, a depth value of the centroid location, and a closest user feature within the image of the user;

computer usable program code that determines whether to initiate recognition of the user'"'"'s gestures as gesture interaction based on a ratio of a closest distance of the user with respect to an imaging device and a depth value at a centroid of the user compared to a predetermined threshold value.

12. The computer program product of claim 11, further comprising computer usable program code that initiates the recognition of the user'"'"'s gestures as gestural interaction if the ratio is above the predetermined threshold.

13. The computer program product of claim 11, further comprising computer usable program code that does not initiate recognition of the user'"'"'s gestures as gestural interaction if the ratio is below the predetermined threshold.