Processing of Gesture-Based User Interactions

• US 20090077504A1
• Filed: 09/15/2008
• Published: 03/19/2009
• Est. Priority Date: 09/14/2007
• Status: Active Grant

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Intellectual Ventures Holding 67 LLC (Intellectual Ventures LLC)
• Inventors: Bell, Matthew, Vieta, Matthew, Chennavasin, Tipatat, Clanton, Charles H., Hulme, Michael, Ophir, Eyal
• Granted Patent: US 8,230,367 B2
• Time in Patent Office: Days
• US Class Current: 715/863
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: G01B 11/14 for measuring distance or clearance between spaced objects or spaced apertures G01B11/26 takes precedence; rangefinders G01C3/00
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/04812 Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

Systems and methods for processing gesture-based user interactions with an interactive display are provided.

Claims

1. A method for processing gesture-based user interactions with an interactive display, the method comprising:

gathering three-dimensional information about an object in a defined three-dimensional space;

identifying a portion of the three-dimensional information as corresponding to a hand of a user in the defined three-dimensional space;

displaying an on-screen image of a digital object;

displaying an on-screen image associated with and approximating the physical shape of the user, wherein the displayed on-screen image approximating that of the user does not interact with the displayed digital object; and

displaying an icon corresponding to the hand of the user in the defined three-dimensional space, wherein the displayed icon is capable of interacting with the displayed digital object.

2. A method for processing gesture-based user interactions with an interactive display, the method comprising:

gathering three-dimensional information about a plurality of users in a defined three-dimensional space, wherein the defined three-dimensional space includes a plurality of volumetric zones, wherein a portion of each of the plurality of users as defined by the three-dimensional information is located in one or more of the plurality of volumetric zones;

identifying a portion of the three-dimensional information as corresponding to a hand of one of the plurality of users in the defined three-dimensional space;

displaying on-screen digital objects; and

displaying an indication of whether one of the plurality of users is currently capable of interacting with the displayed digital objects based on the location of the one of the plurality of users in one or more of the plurality of volumetric zones.

3. The method of claim 2, wherein the plurality of volumetric zones includes a volumetric zone furthest from the interactive display relative to the other volumetric zones.

4. The method of claim 3, further comprising displaying an on-screen image that does not interact with the displayed digital objects and that is associated with and approximating the physical shape of one of the plurality of users located in the volumetric zone furthest from the interactive display relative to the other volumetric zones.

5. The method of claim 4, wherein the display of the on-screen image of one of the plurality of users in the volumetric zone furthest from the interactive display relative to the other volumetric zones differs from the display of an on-screen image associated with and approximating the physical shape of one of the plurality of users located in a volumetric zone for interacting with the displayed on-screen digital objects.

6. The method of claim 2, wherein the plurality of volumetric zones includes a volumetric zone closest to the interactive display relative to the other volumetric zones.

7. The method of claim 6, wherein the indication of whether one of the plurality of users is currently capable of interacting with the displayed digital objects corresponds to the proximity of the one or more of the plurality of volumetric zones with respect to the interactive display.

8. The method of claim 2, wherein the plurality of volumetric zones includes a volumetric zone for interacting with the displayed digital objects, and an icon corresponding to the hand of the one of the plurality of users is displayed only when the corresponding hand of the one of the plurality of users is in the volumetric zone for interacting with the display objects.

9. A method for processing gesture-based user interactions with an interactive display, the method comprising:

gathering three-dimensional information about an object in a defined three-dimensional space;

identifying a portion of the three-dimensional information as corresponding to a hand of a user in the defined three-dimensional space;

displaying an on-screen image of a digital object, the digital object having a visually perceptible activation state corresponding to whether the user has exceeded an activation level threshold for the digital object; and

displaying an icon for interacting with the on-screen image of the digital object, the icon corresponding to the hand of the user in the defined three-dimensional space, and wherein the activation level corresponds to a position of the hand of the user over time.

10. The method of claim 9, wherein the visually perceptible activation state of the digital object indicates if the activation level threshold for the digital object has been surpassed.

11. The method of claim 10, wherein the visually perceptible activation state of the digital object is indicated by the icon corresponding to the hand of the user.

12. The method of claim 10, wherein the visually perceptible activation state of the digital object is indicated by the on-screen image of the digital object.

13. The method of claim 10, wherein the visually perceptible activation state of the displayed digital object is selected from the group consisting of not active, partially active, and active.

14. A method for processing gesture-based user interactions with an interactive display, the method comprising:

gathering three-dimensional information about an object in a defined three-dimensional space;

identifying a portion of the three-dimensional information as corresponding to a hand of a user in the defined three-dimensional space;

displaying an on-screen image of a digital object, the digital object having a visually perceptible activation state corresponding to whether the user has exceeded an activation level threshold for the digital object; and

displaying an icon for interacting with the on-screen image of the digital object, the icon corresponding to the hand of the user in the defined three-dimensional space, and wherein the activation level threshold corresponds to a position of the displayed icon corresponding to the hand of the user versus the displayed digital object over time.

15. A method for processing gesture-based user interactions with an interactive display, the method comprising:

gathering three-dimensional information about an object in a defined three-dimensional space;

identifying a portion of the three-dimensional information as corresponding to a hand of a user in the defined three-dimensional space;

displaying an on-screen image of a digital object, the digital object having an activation level threshold; and

displaying an icon for interacting with the on-screen image of the digital object, the icon corresponding to the hand of the user in the defined three-dimensional space, and wherein the icon may control an action associated with the digital object only if the activation level threshold of the displayed digital object has been surpassed, and wherein the activation level threshold of the digital object is surpassed as a result of the icon being proximate to a region associated with the digital object over time.

16. The method of claim 15, wherein the time to surpass the activation level threshold of digital object corresponds to the amount of motion of the icon with respect to the region associated with the digital object.

17. The method of claim 15, wherein the time to surpass the activation level threshold of the digital object is inversely proportional to the amount of motion of the icon with respect to the region associated with the digital object.