Method, system and device for navigating in a virtual reality environment

• US 9,972,136 B2
• Filed: 02/15/2017
• Issued: 05/15/2018
• Est. Priority Date: 12/18/2014
• Status: Active Grant

• Current Assignee: META Platforms Technologies LLC (Meta Platforms, Inc. (f/k/a Facebook, Inc.))
• Original Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Inventors: Grossinger, Nadav, Alon, Emil, Bar David, Iddo, Solodnik, Efrat
• Primary Examiner(s): Nguyen, Phu K
• Application Number:
  US15/433,952
• Publication Number:
  US 20170169618A1
• Time in Patent Office: 454 Days
• Field of Search: 345419
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

A method, a system, and a device for navigating in a virtual reality scene, using body parts gesturing and posturing are provided herein. The method may include: projecting a synthetic 3D scene, into both eyes of a user, via a near eye display, so as to provide a virtual reality view to the user; identifying at least one gesture or posture carried out by at least one body part of said user; measuring at least one metric of a vector associated with the detected gesture or posture; applying a movement or action of said user in virtual reality environment, based on the measured metrics; and modifying the virtual reality view so as to reflect the movement or action of said user in the virtual reality environment.

Claims

1. A method comprising:

displaying a synthetic 3D scene;

identifying at least one gesture or posture carried out by at least one body part of the user;

deriving a vector that spatially represents the identified gesture or posture;

transforming the vector into a continuous movement or action in the synthetic 3D scene, based on at least one of an angle of the vector and a length of the vector, wherein a speed of the movement is based on a value of the angle; and

modifying the synthetic 3D scene to reflect the movement or action in the synthetic 3D scene.

2. The method of claim 1, wherein the identifying the at least one gesture or posture comprises applying a classifier to a database of predefined postures and gestures.

3. The method of claim 1, wherein the movement or action in the synthetic 3D scene is carried out continuously as long as the at least one of the angle of the vector and the length of the vector maintains a value beyond a predefined threshold.

4. The method of claim 1, wherein the movement or action in the synthetic 3D scene is terminated responsive to detecting a predefined termination act.

5. The method of claim 1, wherein the movement or action in the synthetic 3D scene is terminated responsive to detecting a return of the at least one body part to a basic posture position.

6. The method of claim 1, wherein the identified posture is leaning forward a torso of the user or raising shoulders of the user.

7. The method of claim 1, wherein the identified posture is defined by a spatial relationship between at least two body parts of the user, and the method further comprises determining angles between the at least two body parts and transforming the determined angles into the continuous movement or action in the synthetic 3D scene.

8. The method of claim 1, wherein the vector associated with the identified gesture or posture further comprises a spatial direction angle.

9. The method of claim 1, wherein identifying the least one gesture or posture further comprises subtracting movement components affiliated with head movements of the user to retrieve postures and gestures of the at least one body part that are relative to the head of the user.

10. The method of claim 1, further comprising superimposing a virtual user-interface object into the synthetic 3D scene enabling the user to apply gestures and postures relative to the virtual user-interface object.

11. A system comprising:

a device configured to display a synthetic 3D scene; and

a computer processor configured to;

identify at least one gesture or posture carried out by at least one body part of the user;

derive a vector that spatially represents the identified gesture or posture;

transform the vector into a continuous movement or action in the synthetic 3D scene, based on at least one of an angle of the vector and a length of the vector, wherein a speed of the movement is based on a value of the angle; and

modify the synthetic 3D scene to reflect the movement or action in the synthetic 3D scene.

12. The system of claim 11, wherein the computer processor is further configured to apply a classifier to a database of predefined postures and gestures to identify the at least one gesture or posture.

13. The system of claim 11, wherein the computer processor is further configured to perform the movement or action in the synthetic 3D scene continuously as long as the at least one of the angle of the vector and the length of the vector maintains a value beyond a predefined threshold.

14. The system of claim 11, wherein the computer processor is further configured to terminate the movement or action in the synthetic 3D scene responsive to detection of a predefined termination act.

15. The system of claim 11, wherein the computer processor is further configured to terminate the movement or action in the synthetic 3D scene responsive to detecting a return of the at least one body part to a basic posture position.

16. The system of claim 11, wherein the identified posture is leaning forward a torso of the user or raising shoulders of the user.

17. The system of claim 11, wherein the identified posture is defined by a spatial relationship between at least two body parts of the user, and the computer processor is further configured to determine angles between the at least two body parts and transform the determined angles into the continuous movement or action in the synthetic 3D scene.

18. The system of claim 11, wherein the vector associated with the identified gesture or posture further comprises a spatial direction angle.

19. The system of claim 11, wherein the computer processor is further configured to subtract movement components affiliated with head movements of the user to retrieve postures and gestures of the at least one body part that are relative to the head of the user.

20. The system of claim 11, wherein the computer processor is further configured to superimpose a virtual user-interface object into the synthetic 3D scene enabling the user to apply gestures and postures relative to the virtual user-interface object.