Methods and apparatus to determine objects to present in virtual reality environments

• US 10,514,752 B2
• Filed: 07/07/2017
• Issued: 12/24/2019
• Est. Priority Date: 07/07/2016
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Clement, Manuel Christian, Welker, Stefan
• Primary Examiner(s): Balaoing, Ariel A
• Application Number:
  US15/644,300
• Publication Number:
  US 20180011531A1
• Time in Patent Office: 900 Days
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L

Abstract

In at least one general aspect, a method can include detecting an interaction with a first object in a virtual reality (VR) environment, producing first contextual information based on user interaction with the first object, determining a second object to display in the VR environment based on the first contextual information, and presenting the second object in the VR environment. The interaction of the user may include at least one of a movement, a speech, or a physiological trait of the user.

Claims

1. A method comprising:

detecting an interaction with a first object in a virtual reality (VR) environment;

producing first contextual information based on user interaction with the first object within the VR environment, the interaction includes a physical movement being represented by movement data, the movement data being used to determine a direction in which the user'"'"'s entire body moves toward the first object, the first contextual information being produced based on the determined direction;

determining a second object to display in the VR environment based on the first contextual information; and

presenting the second object in the VR environment.

2. The method of claim 1, wherein the movement of the user is obtained via at least one of an eye-tracking device, a head-tracking device, a hand-tracking device, or a body-tracking device.

3. The method of claim 1, wherein the interaction further includes speech, the speech of the user is obtained via a microphone.

4. The method of claim 1, wherein the interaction further includes physiological trait of the user, the physiological trait of the user is obtained via a biosensor.

5. The method of claim 1, wherein the first contextual information is produced when the user focuses on the first object in the VR environment.

6. The method of claim 1, wherein producing the first contextual information includes estimating a result to determine whether to display the second object.

7. The method of claim 1, further comprising:

detecting an interaction with a third object in the VR environment;

obtaining second contextual information associated with the interaction with the third object; and

determining the second object to display in the VR environment based on the first contextual information and the second contextual information.

8. The method of claim 7, wherein the first contextual information and the second contextual information are weighted when determining the second object based on which of the interaction with the first object or the interaction with the third object occurs first.

9. The method of claim 1, wherein determining the second object includes selecting the second object from a database of objects.

10. The method of claim 1, wherein the first object is a real object.

11. The method of claim 1, wherein the first object is a virtual object.

12. The method of claim 1, further comprising:

determining a gaze direction in accordance with the interaction of the user with the first object; and

determining the second object to display in the VR environment based on the determined gaze direction.

13. The method of claim 12, wherein the determining the gaze direction includes determining a first intensity level and a second intensity level, wherein:

the first intensity level corresponds to the user focusing on the first object, and

the second intensity level corresponds to the user glancing at the first object.

14. The method of claim 13, wherein when the user is at the first intensity level, the second object is presented to the user.

15. The method of claim 13, wherein when the user is at the second intensity level, the second object is not presented to the user.

17. A non-transitory machine-readable media stores machine-readable instructions that, when executed, cause a machine to at least perform the method of claim 1.

18. The method of claim 1, further comprising determining a duration in which the user moves toward the first object.

19. The method of claim 1, wherein the first contextual information is produced based on whether the user moves towards the first object at a faster speed than an average walking speed.

16. A system, comprising:

a computing device configured to generate a virtual environment, the computing device including;

a memory storing executable instructions; and

a processor configured to execute the instructions, to cause the computing device to;

detecting an interaction with a first object in a virtual reality (VR) environment;

obtaining first contextual information based on user interaction with the first object, the interaction includes a physical movement being represented by movement data, the movement data being used to determine a direction and duration in which the user moves toward the first object, the first contextual information being produced based on the determined direction toward the first object and based on a speed the user moves towards the first object;

determining a second object to display in the VR environment based on the first contextual information; and

presenting the second object in the VR environment.

20. A method comprising:

detecting an interaction with a first object in a virtual reality (VR) environment;

producing first contextual information based on user interaction with the first object within the VR environment, the interaction includes a physical movement being represented by movement data, the movement data being used to determine a direction and duration in which the user walks toward the first object, the first contextual information being produced based on the determined direction;

determining a second object to display in the VR environment based on the first contextual information; and

presenting the second object in the VR environment.