Virtual reality presentation of eye movement and eye contact

• US 10,489,960 B2
• Filed: 05/08/2019
• Issued: 11/26/2019
• Est. Priority Date: 02/14/2017
• Status: Active Grant

• Current Assignee: Linden Research Incorporated
• Original Assignee: Linden Research Incorporated
• Inventors: Grant, Jeremiah Arthur
• Primary Examiner(s): Hajnik, Daniel F
• Application Number:
  US16/406,670
• Publication Number:
  US 20190266776A1
• Time in Patent Office: 202 Days
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings

Abstract

A computing system and method to implement a three-dimensional virtual reality world with avatar eye movements without user eye tracking. A position and orientation of a respective avatar in the virtual reality world is tracked to generate a view of the virtual world for the avatar and to present the avatar to others. In response to detecting a predetermined event, the computing system predicts a point (e.g., the eye of another avatar) that is of interest to the respective avatar responsive to the event, and computes, according to an eye movement model, an animation of the eyes of the respective avatar where the gaze of the avatar moves from an initial point to the predicted point and/or its vicinity for a period of time and back to the initial point.

Claims

1. A method implemented in a three-dimensional virtual reality world, the method comprising:

detecting an event related to an avatar having a position and orientation in the virtual reality world, the avatar having at least one eye; and

in response to the event,

predicting a point of interest to the avatar without tracking eyes of a user of the avatar;

computing an animation of the eye of the avatar, wherein the animation is configured to move a gaze of the avatar from an initial point at a time of the event to the point of interest; and

switching from a first input mode before the animation to a second input mode during the animation, wherein;

when in the first input mode, inputs from one or more devices control the position and orientation of the avatar; and

when in the second input mode, inputs from the one or more devices controls the eye movements;

wherein the animation is computed according to a predetermined eye movement model, comprising;

determining, using a machine learning technique, personalization parameters of the eye movement model based on inputs received in the second input mode.

2. The method of claim 1, further comprising:

adjusting a view of the virtual world computed for the avatar according to the eye movements of the animation.

3. The method of claim 2, further comprising:

presenting eye movements of the animation to avatars that have visibility into the animated eye.

4. The method of claim 1, wherein the animation moves the gaze of the avatar to the point of interest for a period of time and turns the gaze back to the initial point.

5. The method of claim 4, wherein the animation is computed without using user inputs during the animation.

6. The method of claim 5, wherein the point of interest includes an eye of a second avatar.

7. The method of claim 6, wherein the event corresponds to a real-time communication to or from the second avatar.

8. The method of claim 7, wherein the animation is computed according to a predetermined eye movement model.

9. The method of claim 8, wherein the animation is computed based on a context determined from the event.

10. The method of claim 9, wherein the animation is computed based on personalization parameters of the avatar for which the animation is computed.

11. The method of claim 1, wherein the animation is computed according to a predetermined eye movement model, and the method further comprises:

training the eye movement model via machine learning using eye movements captured in video images of people engaging in social activities.

12. The method of claim 1, wherein the animation is computed according to a predetermined eye movement model, and the method further comprises:

training the eye movement model via machine learning using eye movements captured using eye tracking devices of users using the virtual reality world.

13. A non-transitory computer storage medium storing instructions configured to instruct a computer device to perform a method implemented in a three-dimensional virtual reality world, the method comprising:

detecting an event related to an avatar having a position and orientation in the virtual reality world, the avatar having at least one eye; and

in response to the event,

predicting a point of interest to the avatar without tracking eyes of a user of the avatar;

computing an animation of the eye of the avatar, wherein the animation is configured to move a gaze of the avatar from an initial point at a time of the event to the point of interest; and

switching from a first input mode before the animation to a second input mode during the animation, wherein;

when in the first input mode, inputs from one or more devices control the position and orientation of the avatar; and

when in the second input mode, inputs from the one or more devices controls the eye movements;

wherein the animation is computed according to a predetermined eye movement model, comprising;

determining, using a machine learning technique, personalization parameters of the eye movement model based on inputs received in the second input mode.

14. A computing system to implement a three-dimensional virtual reality world, the system comprising:

a server system; and

a data storage device storing;

a three-dimensional model of the virtual reality world; and

avatar models representing residences of the virtual reality world;

wherein the server system generates, from the three-dimensional model of the virtual reality world and the avatar models, data stream to provide views of the virtual reality world to client devices that are connected to the server system via a computer network;

wherein the computing system tracks a position and orientation of a respective avatar in the virtual reality world; and

wherein, in response to detecting a predetermined event related to the respective avatar, the computing system is configured to;

predict a point that is of interest to the respective avatar;

compute an animation of one or more eyes of the respective avatar by moving a gaze of the respective avatar from an initial point at a time of the event to the predicted point of interest to the avatar; and

switch from a first input mode before the animation to a second input mode during the animation, wherein;

when in the first input mode, inputs from one or more devices control the position and orientation of the avatar; and

when in the second input mode, inputs from the one or more devices controls the eye movements;

wherein the animation is computed according to a predetermined eye movement model, comprising;

determine, using a machine learning technique, personalization parameters of the eye movement model based on inputs received in the second input mode.

15. The computing system of claim 14, wherein the animation is computed without input from a client device tracking eyes of a user represented by the respective avatar in the virtual reality world;

the computing system is further configured to present the animation to at least one avatar that has visibility of the one or more eyes of the respective avatar.

16. The computing system of claim 14, further configured to adjust a view of the virtual world generated for the respective avatar in accordance with a swift of the gaze of the respective avatar in the computed animation.

17. The computing system of claim 14, wherein during a period of the animation, a user input device that controls the position and orientation of the respective avatar before the period of the animation is automatically reconfigured to control the movement of the gaze during the period of the animation.