Processing of gesture-based user interactions activation levels

• US 9,058,058 B2
• Filed: 07/23/2012
• Issued: 06/16/2015
• Est. Priority Date: 09/14/2007
• Status: Active Grant

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Intellectual Ventures Holding 67 LLC (Intellectual Ventures LLC)
• Inventors: Bell, Matthew, Chennavasin, Tipatat, Clanton, Charles H., Hulme, Michael, Ophir, Eyal, Vieta, Matthew
• Primary Examiner(s): Durkin, James T
• Application Number:
  US13/556,090
• Publication Number:
  US 20120287044A1
• Time in Patent Office: 1,058 Days
• Field of Search: 715/772, 715/821, 715/823, 715/824, 715/863, 715/864, 345/419, 463/36, 463/39
• US Class Current: 1/1
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: G01B 11/14 for measuring distance or clearance between spaced objects or spaced apertures G01B11/26 takes precedence; rangefinders G01C3/00
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/04812 Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

Systems and methods for processing gesture-based user interactions within an interactive display area are provided. The display of one or more virtual objects and user interactions with the one or more virtual objects may be further provided. Multiple interactive areas may be created by partitioning an area proximate a display into multiple volumetric spaces or zones. The zones may be associated with respective user interaction capabilities. A representation of a user on the display may change as the ability of the user to interact with one or more virtual object changes.

Claims

1. A method comprising:

receiving information corresponding to a physical space from one or more cameras;

mapping a position of a user in the physical space to a position in a virtual space based at least in part on the received information;

determining an activation level of the user based on a distance from a position of a hand of the user in the physical space to a position of another portion of the user in the physical space, wherein the distance associated with the activation level is increased or decreased using;

a proximity of the position of the hand of the user to a physical space relative to a virtual object during a predetermined time period, and

a velocity of the hand of the user along one or more axes during the predetermined time period;

selecting one of a plurality of visually perceptible indicators based on a comparison of the activation level and activation thresholds associated with respective visually perceptible indicators; and

generating an image comprising;

the virtual object; and

the selected one of the plurality of visually perceptible indicators.

2. The method of claim 1, wherein generating the image further comprises selecting another visually perceptible indicator from the plurality of visually perceptible indicators in response to determining that the activation threshold for the selected one of the plurality of visually perceptible indicators has been surpassed by the activation level.

3. The method of claim 1, wherein generating the image further comprises generating a supplemental indicator in response to determining that the activation threshold has been surpassed by the activation level.

4. The method of claim 1, wherein the image further comprises an icon corresponding to the position of the user in the physical space, and wherein the visually perceptible indicator corresponding to the activation level includes a visual appearance of the icon in the image.

5. The method of claim 1, wherein the selected visually perceptible indicator comprises a representation of a body part.

6. The method of claim 1, wherein the plurality of visually perceptible indicators correspond to different configurations of a representation of a hand.

7. A non-transitory computer readable storage medium having software instructions stored thereon that, in response to execution by a computing device, cause the computing device to perform operations comprising:

receiving information corresponding to a three-dimensional space from one or more cameras;

determining a first position of a hand of a user in the three-dimensional space based at least in part on the received information;

determining a second position of a portion of the user that is not the hand of the user in the three-dimensional space based at least in part on the received information;

determining an activation level of the hand of the user based on a distance between the first position and the second position, wherein the distance associated with the activation level is increased or decreased using;

a proximity of the position of the hand of the user to a physical space relative to a virtual object during a predetermined time period, and

a velocity of the hand of the user along one or more axes during the predetermined time period;

selecting one of a plurality of visually perceptible indicators based on the activation level and activation thresholds associated with respective visually perceptible indicators; and

generating an image of a virtual space comprising;

the virtual object; and

the selected one of the plurality of visually perceptible indicators.

8. The non-transitory computer readable storage medium of claim 7, wherein the generated image further comprises a depiction of an interaction between the user and the virtual object in response to the activation level exceeding the activation threshold, wherein the activation threshold is exceeded in response to the first position remaining at least a predetermined distance from the second position for a predetermined period of time.

9. The non-transitory computer readable storage medium of claim 7, wherein the generated image further comprises a depiction of an interaction between the user and the virtual object in response to the activation level exceeding the activation threshold.

10. The non-transitory computer readable storage medium of claim 7, wherein generating the image further comprises selecting another visual indication from the plurality of visually perceptible indicators in response to determining that the activation threshold for the selected one of the plurality of visually perceptible indicators has been surpassed by the activation level.

11. The non-transitory computer readable storage medium of claim 7, wherein generating the image further comprises generating a supplemental indicator in response to determining that the activation threshold has been surpassed by the activation level.

12. A system, comprising:

one or more cameras; and

one or more hardware processors configured to execute instructions in order to cause the system to;

receive information corresponding to a physical space from the one or more cameras;

determine a first position of a hand of a user in the physical space based at least in part on the received information;

determine a second position of a second body part of the user different from the hand in the physical space based at least in part on the received information;

determine an activation level for an interaction based on a distance between the first position and the second position, wherein the distance associated with the activation level is increased or decreased using;

a proximity of the position of the hand of the user to a physical space relative to a virtual object during a predetermined time period, and

a velocity of the hand of the user along one or more axes during the predetermined time period;

select one of a plurality of visual indications based on the activation level and activation thresholds associated with respective visual indications; and

generate an image of a virtual space comprising;

the virtual object; and

the selected one of the plurality of visual indications.

13. The system of claim 12, wherein generating the image further comprises selecting another visual indication from the plurality of visual indications in response to determining that the activation threshold for the selected one of the plurality of visual indications has been surpassed by the activation level.

14. The system of claim 12, wherein generating the image further comprises generating a supplemental indicator in response to determining that an activation threshold has been surpassed by the activation level.

15. The system of claim 12, wherein the image further comprises an icon corresponding to the position of the user in the physical space, and wherein the visual indication corresponding to the activation level is indicated by a visual appearance of the icon in the image.

16. The system of claim 12, wherein the plurality of visual indications represent different hand configurations.

17. The system of claim 12, wherein the one or more hardware processors are further configured to execute instructions in order to cause the system to:

select the virtual object from within the virtual space when the activation level exceeds an activation threshold.