HUMAN-BODY-GESTURE-BASED REGION AND VOLUME SELECTION FOR HMD

• US 20140225918A1
• Filed: 02/14/2013
• Published: 08/14/2014
• Est. Priority Date: 02/14/2013
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Maciocci, Giuliano, Tunmer, Michael L., Mabbutt, Paul, MITTAL, Arpit
• Granted Patent: US 10,133,342 B2
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

Methods, systems, computer-readable media, and apparatuses for selecting an Augmented Reality (AR) object on a head mounted device (HMD) are presented. In some embodiments, an HMD may define a Region-of-Interest (ROI) based on a gesture formed by at least one hand of a user. Subsequently the HMD may display to the user a shape on the HMD. In at least one arrangement, the shape outlines the ROI. Additionally, the HMD may display to the user a plurality of AR objects, each of the plurality of AR objects being associated with a target inside the ROI. Furthermore, the HMD may reduce the size of the ROI based on a first movement of the at least one hand of the user to form a reduced-sized ROI. In at least one arrangement, the reduced-sized ROI is used to select a specific AR object from the plurality of AR objects.

Claims

1. A method for selecting an Augmented Reality (AR) object on a head mounted device (HMD) comprising:

defining a Region-of-Interest (ROI) based on a gesture formed by at least one hand of a user;

displaying to the user a shape on the HMD, wherein the shape outlines the ROI;

displaying to the user a plurality of AR objects, each of the plurality of AR objects being associated with a target inside the ROI; and

reducing the size of the ROI based on a first movement of the at least one hand of the user to form a reduced-sized ROI, wherein the reduced-sized ROI is used to select a specific AR object from the plurality of AR objects.

2. The method of claim 1, wherein the at least one hand of the user comprises both hands of the user.

3. The method of claim 2, wherein the first movement for reducing the size of the ROI comprises of moving the user'"'"'s hands closer to each other.

4. The method of claim 1, wherein multiple augmentations are associated with the specific AR object, the method further comprising:

displaying to the user a corresponding augmentation from the multiple augmentations associated with the specific AR object based on a second movement of the at least one hand of the user in the direction of the specific AR object.

5. The method of claim 1, further comprising:

capturing text inside the reduced-sized ROI; and

initializing translation based on the captured text.

6. The method of claim 1, further comprising:

performing a visual recognition search inside the reduced-sized ROI.

7. The method of claim 1, the method further comprising:

disengaging the reduced-sized ROI based on a disengagement event.

8. The method of claim 7, wherein the disengagement event occurs when the at least one hand of the user is away from the ROI, when at least one finger and thumb of the user are closed together, or when a voice command by the user.

9. A head mounted device (HMD) for selecting an Augmented Reality (AR) object comprising:

one or more processors; and

memory storing computer-readable instructions that, when executed by the one or more processors, cause the HMD to;

define a Region-of-Interest (ROI) based on a gesture formed by at least one hand of a user;

display to the user a shape on the HMD, wherein the shape outlines the ROI;

display to the user a plurality of AR objects, each of the plurality of AR objects being associated with a target inside the ROI; and

reduce the size of the ROI based on a first movement of the at least one hand of the user to form a reduced-sized ROI, wherein the reduced-sized ROI is used to select a specific AR object from the plurality of AR objects.

10. The HMD of claim 9, wherein the at least one hand of the user comprises both hands of the user.

11. The HMD of claim 10, wherein the first movement for reducing the size of the ROI comprises of moving the user'"'"'s hands closer to each other.

12. The HMD of claim 9, wherein multiple augmentations are associated with the specific AR object, and wherein the computer-readable instructions further cause the HMD to:

display to the user a corresponding augmentation from the multiple augmentations associated with the specific AR object based on a second movement of the at least one hand of the user in the direction of the specific AR object.

13. The HMD of claim 9, wherein the computer-readable instructions further cause the HMD to:

capture text inside the reduced-sized ROI; and

initialize translation based on the captured text.

14. The HMD of claim 9, wherein the computer-readable instructions further cause the HMD to:

perform a visual recognition search inside the reduced-sized ROI.

15. The HMD of claim 9, wherein the computer-readable instructions further cause the HMD to:

disengage the reduced-sized ROI based on a disengagement event.

16. The HMD of claim 15, wherein the disengagement event occurs when the at least one hand of the user is away from the ROI, when at least one finger and thumb of the user are closed together, or when a voice command by the user.

17. One or more computer-readable media storing computer-executable instructions for selecting an Augmented Reality (AR) object on a head mounted device (HMD) that, when executed, cause one or more computing devices included in a HMD to:

define a Region-of-Interest (ROI) based on a gesture formed by at least one hand of a user;

display to the user a shape on the HMD, wherein the shape outlines the ROI;

display to the user a plurality of AR objects, each of the plurality of AR objects being associated with a target inside the ROI; and

reduce the size of the ROI based on a first movement of the at least one hand of the user to form a reduced-sized ROI, wherein the reduced-sized ROI is used to select a specific AR object from the plurality of AR objects.

18. The one or more computer-readable media of claim 17, wherein the at least one hand of the user comprises both hands of the user.

19. The one or more computer-readable media of claim 18, wherein the first movement for reducing the size of the ROI comprises of moving the user'"'"'s hands closer to each other.

20. The one or more computer-readable media of claim 17, wherein multiple augmentations are associated with the specific AR object, and wherein the computer executable instructions further cause the one or more computing devices to:

display to the user a corresponding augmentation from the multiple augmentations associated with the specific AR object based on a second movement of the at least one hand of the user in the direction of the specific AR object.

21. The one or more computer-readable media of claim 17, wherein the computer executable instructions further cause the one or more computing devices to:

capture text inside the reduced-sized ROI; and

initialize translation based on the captured text.

22. The one or more computer-readable media of claim 17, wherein the computer executable instructions further cause the one or more computing devices to:

perform a visual recognition search inside the reduced-sized ROI.

23. The one or more computer-readable media of claim 17, the method further comprising:

disengaging the reduced-sized ROI based on a disengagement event.

24. The one or more computer-readable media of claim 23, wherein the disengagement event occurs when the at least one hand of the user is away from the ROI, when at least one finger and thumb of the user are closed together, or when a voice command by the user.

25. A head mounted device (HMD) for selecting an Augmented Reality (AR) object comprising:

means for defining a Region-of-Interest (ROI) based on a gesture formed by at least one hand of a user;

means for displaying to the user a shape on the HMD, wherein the shape outlines the ROI;

means for displaying to the user a plurality of AR objects, each of the plurality of AR objects being associated with a target inside the ROI; and

means for reducing the size of the ROI based on a first movement of the at least one hand of the user to form a reduced-sized ROI, wherein the reduced-sized ROI is used to select a specific AR object from the plurality of AR objects.

26. The HMD of claim 25, wherein the at least one hand of the user comprises both hands of the user.

27. The HMD of claim 26, wherein the first movement for reducing the size of the ROI comprises of moving the user'"'"'s hands closer to each other.

28. The HMD of claim 25, wherein multiple augmentations are associated with the specific AR object, further comprising:

means for displaying to the user a corresponding augmentation from the multiple augmentations associated with the specific AR object based on a second movement of the at least one hand of the user in the direction of the specific AR object.

29. The HMD of claim 25, further comprising:

means for capturing text inside the reduced-sized ROI; and

means for initializing translation based on the captured text.

30. The HMD of claim 25, further comprising:

means for performing a visual recognition search inside the reduced-sized ROI.

31. The HMD of claim 25, the method further comprising:

disengaging the reduced-sized ROI based on a disengagement event.

32. The HMD of claim 31, wherein the disengagement event occurs when the at least one hand of the user is away from the ROI, when at least one finger and thumb of the user are closed together, or when a voice command by the user.