PROCESSING OF GESTURE-BASED USER INTERACTION USING VOLUMETRIC ZONES

• US 20200133399A1
• Filed: 12/23/2019
• Published: 04/30/2020
• Est. Priority Date: 09/14/2007
• Status: Active Grant

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Inventors: Bell, Matthew, Chennavasin, Tipatat, Clanton, Charles H., Hulme, Michael, Ophir, Eyal, Vieta, Matthew
• Granted Patent: US 10,990,189 B2
• Time in Patent Office: Days
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: G01B 11/14 for measuring distance or clearance between spaced objects or spaced apertures G01B11/26 takes precedence; rangefinders G01C3/00
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/04812 Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

Systems and methods for processing gesture-based user interactions within an interactive display area are provided. The display of one or more virtual objects and user interactions with the one or more virtual objects may be further provided. Multiple interactive areas may be created by partitioning an area proximate a display into multiple volumetric spaces or zones. The zones may be associated with respective user interaction capabilities. A representation of a user on the display may change as the ability of the user to interact with one or more virtual object changes.

Claims

1. A method comprising:

tracking, via a computer vision system, at least part of a hand of a user over time;

rendering, based on the tracking, a visual representation of the at least part of the hand on a display;

rendering a virtual object on the display;

performing selection detection of the virtual object based at least in part on movement of the at least part of the hand; and

based on a particular value of the selection detection, changing the rendering of the visual representation of the at least part of the hand by selecting, from an animation sequence, an animation frame of the at least part of the hand.

2. The method of claim 1, wherein the particular value corresponds to a distance value representing a detected distance of the hand of the user from at least part of a body of the user.

3. The method of claim 2, wherein the at least part of the body of the user comprises at least one of:

a head of the user;

a torso of the user;

a shoulder of the user;

or

a center of gravity of the user.

4. The method of claim 1, wherein the particular value corresponds to a distance value representing a detected distance of the hand of the user from the virtual object.

5. The method of claim 1, wherein the animation sequence is configured to cause the visual representation to depict that the at least part of the hand gradually opens or closes as the particular value at least one of increases or decreases.

6. The method of claim 1, wherein the animation frame depicts one or more fingers of the hand in a partially closed position.

7. The method of claim 1, wherein the particular value represents an amount by which a detected distance of the hand of the user increases or decreases within a specified period of time.

8. The method of claim 1, wherein the particular value represents a speed of the hand relative to the display along an axis perpendicular to the display.

9. A system comprising:

a physical processor;

a memory device;

a display device;

a computer vision system that has at least one camera and that is coupled to the display device, wherein the memory device stores instructions that, when executed by the processor, cause the system to;

track at least part of a hand of a user over time;

render, based on the tracking, a visual representation of the at least part of the hand on the display device;

render a virtual object on the display device;

perform selection detection of the virtual object based at least in part on movement of the at least part of the hand; and

based on a particular value of the selection detection, change the rendering of the visual representation of the at least part of the hand by selecting, from an animation sequence, an animation frame of the at least part of the hand.

10. The system of claim 9, wherein the particular value corresponds to a distance value representing a detected distance of the hand of the user from at least part of a body of the user.

11. The system of claim 10, wherein the at least part of the body of the user comprises at least one of:

a head of the user;

a torso of the user;

a shoulder of the user;

or

a center of gravity of the user.

12. The system of claim 9, wherein the particular value corresponds to a distance value representing a detected distance of the hand of the user from the virtual object.

13. The system of claim 9, wherein the animation sequence is configured to cause the visual representation to depict that the at least part of the hand gradually opens or closes as the particular value at least one of increases or decreases.

14. The system of claim 9, wherein the animation frame depicts one or more fingers of the hand in a partially closed position.

15. The system of claim 9, wherein the particular value represents an amount by which a detected distance of the hand of the user increases or decreases within a specified period of time.

16. The system of claim 9, wherein the particular value represents a speed of the hand relative to the display along an axis perpendicular to the display.

17. A non-transitory computer-readable medium having instructions recorded thereon that, when enacted by a computer processor, cause the computer processor to:

track at least part of a hand of a user over time;

render, based on the tracking, a visual representation of the at least part of the hand on a display;

render a virtual object on the display;

perform selection detection of the virtual object based at least in part on movement of the at least part of the hand; and

based on a particular value of the selection detection, change the rendering of the visual representation of the at least part of the hand by selecting, from an animation sequence, an animation frame of the at least part of the hand.

18. The non-transitory computer-readable medium of claim 17, wherein the particular value corresponds to a distance value representing a detected distance of the hand of the user from at least part of a body of the user.

19. The non-transitory computer-readable medium of claim 17, wherein the particular value corresponds to a distance value representing a detected distance of the hand of the user from the virtual object.

20. The non-transitory computer-readable medium of claim 17, wherein the particular value represents an amount by which a detected distance of the hand of the user increases or decreases within a specified period of time.