Distinguishing between one-handed and two-handed gesture sequences in virtual, augmented, and mixed reality (xR) applications

• US 10,642,369 B2
• Filed: 06/14/2018
• Issued: 05/05/2020
• Est. Priority Date: 06/14/2018
• Status: Active Grant

• Current Assignee: Dell Products LP (Dell Technologies Inc.)
• Original Assignee: Dell Products LP (Dell Technologies Inc.)
• Inventors: Iyer, Vivek Viswanathan, Smith, Hana Schuster
• Primary Examiner(s): Azongha, Sardis F
• Application Number:
  US16/008,982
• Publication Number:
  US 20190384405A1
• Time in Patent Office: 691 Days
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06V 40/11 Hand-related biometrics; Hand pose recognition
• CPC Class Codes: G06V 40/113 Recognition of static hand signs
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

Systems and methods for distinguishing between one-handed and two-handed gesture sequences in virtual, augmented, and mixed reality (xR) applications are described. In an illustrative, non-limiting embodiment, an Information Handling System (IHS) may include a processor and a memory coupled to the processor, the memory having program instructions stored thereon that, upon execution, cause the IHS to: receive a gesture sequence from a user wearing a Head-Mounted Device (HMD) coupled to the IHS, where the HMD is configured to display an xR application, and identify the gesture sequence as: (i) a one-handed gesture sequence, or (ii) a two-handed gesture sequence.

Claims

1. An Information Handling System (IHS), comprising:

a processor; and

a memory coupled to the processor, the memory having program instructions stored thereon that, upon execution, cause the IHS to;

receive a gesture sequence from a user wearing a Head-Mounted Device (HMD) coupled to the IHS in a plurality of video frames captured via a camera mounted on the HMD, wherein the HMD is configured to display a virtual, augmented, or mixed reality (xR) application;

detect a most frequent gesture and a second most frequent gesture in the plurality of video frames, wherein a number of video frames with the most frequent gesture is greater than a number of frames with the second most frequent gesture by a selected amount; and

identify the gesture sequence as;

(i) a one-handed gesture sequence, or (ii) a two-handed gesture sequence.

2. The IHS of claim 1, wherein to receive the gesture sequence, the program instructions, upon execution, further cause the IHS to detect a change in lighting using an Automatic Light Sensor (ALS) mounted on the HMD.

3. The IHS of claim 1, wherein to identify the gesture sequence, the program instructions, upon execution, further cause the IHS to:

extract a first set of features from each of a first set of video frames as part of a start phase of a gesture recognition component; and

extract a second set of features from each of a second set of video frames as part of a motion phase of the gesture recognition component.

4. The IHS of claim 3, wherein the program instructions, upon execution, further cause the IHS to:

detect two hands in at least one of the second set of video frames; and

in response to a comparison between the first and second sets of features, identify the gesture sequence as a first one-handed gesture sequence made with one hand followed by a second one-handed gesture sequence made with another hand.

5. The IHS of claim 4, wherein to identify the gesture sequence, the program instructions, upon execution, further cause the IHS to perform a table look-up operation using calibration data that includes an indication of the user'"'"'s motion velocity.

6. The IHS of claim 4, wherein the program instructions, upon execution, further cause the IHS to:

detect only one hand in at least one of the second set of video frames; and

in response to a comparison between the first and second sets of features, identify the gesture sequence as a two-handed gesture sequence.

7. The IHS of claim 6, wherein to identify the gesture sequence, the program instructions, upon execution, further cause the IHS to perform a table look-up operation using calibration data that includes an indication of the user'"'"'s asynchronicity.

8. The IHS of claim 1, wherein the program instructions, upon execution, further cause the IHS to map the identified gesture sequence to a user interface command.

9. A method, comprising:

receiving a plurality of video frames captured via a gesture camera mounted on a Head-Mounted Device (HMD), wherein the HMD is configured to display a virtual, augmented, or mixed reality (xR) application;

extracting a first set of features from each of a first set of video frames as part of a start phase of a gesture recognition component of the xR application;

extracting a second set of features from each of a second set of video frames as part of a motion phase of the gesture recognition component;

detecting only one hand in at least one of the second set of video frames; and

in response to a comparison between the first and second sets of features, identifying a gesture sequence as a two-handed gesture sequence.

10. The method of claim 9, wherein identifying the gesture sequence further comprises:

detecting two hands in at least one of the second set of video frames; and

in response to a comparison between the first and second sets of features, identifying the gesture sequence as a first one-handed gesture sequence made with one hand followed by a second one-handed gesture sequence made with another hand.

11. The method of claim 10, wherein identifying the gesture sequence further comprises performing a table look-up operation using calibration data that includes an indication of a user'"'"'s motion velocity.

12. The method of claim 9, wherein identifying the gesture sequence further comprises performing a table look-up operation using calibration data that includes an indication of a user'"'"'s asynchronicity.

13. A hardware memory device of an Information Handling System (IHS) having program instructions stored thereon that, upon execution by a hardware processor, cause the IHS to:

receive a plurality of video frames captured via a gesture camera mounted on a Head-Mounted Device (HMD), wherein the HMD is configured to display a virtual, augmented, or mixed reality (xR) application;

extract a first set of features from each of a first set of video frames as part of a start phase of a gesture recognition component of the xR application;

extract a second set of features from each of a second set of video frames as part of a motion phase of the gesture recognition component;

detect only one hand in at least one of the second set of video frames; and

in response to a comparison between the first and second sets of features, identify a gesture sequence as a two-handed gesture sequence.

14. The hardware memory device of claim 13, wherein the program instructions, upon execution, further cause the IHS to:

detect two hands in at least one of the second set of video frames; and

in response to a comparison between the first and second sets of features, identify the gesture sequence as a first one-handed gesture sequence made with one hand followed by a second one-handed gesture sequence made with another hand.

15. The hardware memory device of claim 14, wherein the program instructions, upon execution, further cause the IHS to perform a table look-up operation using calibration data that includes an indication of a user'"'"'s motion velocity.

16. The hardware memory device of claim 13, wherein the program instructions, upon execution, further cause the IHS to perform a table look-up operation using calibration data that includes an indication of a user'"'"'s asynchronicity.