Methods and systems for gaze-based control of virtual reality media content

• US 10,401,960 B2
• Filed: 07/16/2018
• Issued: 09/03/2019
• Est. Priority Date: 03/31/2016
• Status: Active Grant

• Current Assignee: Verizon Patent and Licensing Incorporated (Verizon Communications Inc.)
• Original Assignee: Verizon Patent and Licensing Incorporated (Verizon Communications Inc.)
• Inventors: Khalid, Mohammad Raheel, Jaafar, Ali
• Primary Examiner(s): Mushambo, Martin
• Application Number:
  US16/036,793
• Publication Number:
  US 20180321742A1
• Time in Patent Office: 414 Days
• Field of Search: 345633
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/70 of video data
• CPC Class Codes: G06F 16/75 Clustering; Classification
• CPC Class Codes: G06F 16/78 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2219/2024 Style variation

Abstract

An exemplary virtual reality media system presents a field of view of an immersive virtual reality world on a display screen of a media player device associated with a user. The field of view includes content of the immersive virtual reality world and dynamically changes in response to user input provided by the user as the user experiences the immersive virtual reality world. Additionally, the virtual reality media system detects that a gaze of the user is directed for a predetermined amount of time at a gaze target included within the field of view. In response to the detection, the virtual reality media system presents an interactive user interface associated with the gaze target. The interactive user interface is presented within the field of view together with the content of the immersive virtual reality world. Corresponding methods and systems are also described.

Claims

1. A method comprising:

presenting, by a virtual reality media system on a display screen of a media player device associated with a user, a field of view of an immersive virtual reality world, the field of view including content of the immersive virtual reality world and dynamically changing in response to user input provided by the user as the user experiences the immersive virtual reality world;

detecting, by the virtual reality media system, that a gaze of the user is directed for a predetermined amount of time at a gaze target included within the field of view, the detecting including displaying, at a location within the field of view associated with the gaze of the user and while the gaze of the user is directed at the gaze target, a graphical indicator that indicates

that the gaze of the user is currently directed at the gaze target,

a first length of time that has elapsed toward the predetermined amount of time, and

a second length of time that remains before the predetermined amount of time will elapse; and

presenting, by the virtual reality media system in response to the detecting that the gaze of the user is directed for the predetermined amount of time at the gaze target, an interactive user interface associated with the gaze target, the interactive user interface presented within the field of view together with the content of the immersive virtual reality world.

2. The method of claim 1, wherein:

the detecting that the gaze of the user is directed for the predetermined amount of time at the gaze target further includes

tracking a gaze reticle persistently centered within the field of view with respect to at least one of a vertical dimension of the field of view and a horizontal dimension of the field of view,

determining, based on the tracking, that the gaze reticle is directed at the gaze target, and

determining, based on the tracking, that the gaze reticle remains persistently directed at the gaze target for the predetermined amount of time; and

the displaying of the graphical indicator is performed

in response to the determining that the gaze reticle is directed at the gaze target, and

concurrently with the gaze reticle remaining persistently directed at the gaze target for the predetermined amount of time.

3. The method of claim 2, wherein the gaze reticle persistently centered within the field of view is invisible to the user as the user experiences the immersive virtual reality world through the field of view.

4. The method of claim 2, wherein the gaze reticle persistently centered within the field of view is:

invisible to the user as the user experiences the immersive virtual reality world through the field of view when the gaze reticle is greater than a predefined distance from the gaze target; and

displayed to the user as the user experiences the immersive virtual reality world through the field of view when the gaze reticle is within the predefined distance of the gaze target.

5. The method of claim 1, wherein:

the detecting that the gaze of the user is directed for the predetermined amount of time at the gaze target further includes

tracking eye movements of the user as the user experiences the immersive virtual reality world,

determining, based on the eye movements of the user, that the user is looking at the gaze target, and

determining, based on the eye movements of the user, that the user continues to look at the gaze target for the predetermined amount of time; and

the displaying of the graphical indicator is performed

in response to the determining that the user is looking at the gaze target based on the eye movements of the user, and

concurrently with the user continuing to look at the gaze target for the predetermined amount of time.

6. The method of claim 5, wherein an indicator associated with an eye tracking area representing where the user is detected to be looking is invisible to the user as the user experiences the immersive virtual reality world through the field of view.

7. The method of claim 5, wherein an indicator associated with an eye tracking area representing where the user is detected to be looking is:

invisible to the user as the user experiences the immersive virtual reality world through the field of view when the indicator is greater than a predefined distance from the gaze target; and

displayed to the user as the user experiences the immersive virtual reality world through the field of view when the indicator is within the predefined distance of the gaze target.

8. The method of claim 1, wherein the location at which the graphical indicator is displayed is a persistent location within the field of view, the persistent location associated with the gaze of the user.

9. The method of claim 1, wherein:

the graphical indicator includes a cursor and a percentage indicator that gradually rotates around the cursor during the predetermined amount of time until the percentage indicator rotates all the way around the cursor when the predetermined amount of time has elapsed;

the graphical indicator indicates the first length of time that has elapsed toward the predetermined amount of time by indicating how far the percentage indicator has rotated around the cursor; and

the graphical indicator indicates the second length of time that remains before the predetermined amount of time will elapse by indicating how far the percentage indicator still has to rotate around the cursor before rotating all the way around the cursor.

10. The method of claim 1, wherein:

the graphical indicator includes a cursor displayed within an outer circle that gradually grows during the predetermined amount of time until the cursor entirely fills the outer circle when the predetermined amount of time has elapsed;

the graphical indicator indicates the first length of time that has elapsed toward the predetermined amount of time by indicating how much the cursor has grown to fill the outer circle; and

the graphical indicator indicates the second length of time that remains before the predetermined amount of time will elapse by indicating how much the cursor still has to grow before entirely filling the outer circle.

11. The method of claim 1, wherein the gaze target includes a virtual object integrated into the content of the immersive virtual reality world.

12. The method of claim 1, wherein the gaze target is invisible to the user as the user views the content of the immersive virtual reality world within the field of view.

13. The method of claim 1, wherein:

the immersive virtual reality world is associated with a virtual reality media program, the virtual reality media program including a beginning and an end;

the virtual reality media program is presented to the user over a period of time as a current playback point of the virtual reality media program progresses from the beginning of the virtual reality media program to the end of the virtual reality media program; and

the interactive user interface associated with the gaze target includes a gaze-selectable control that, when selected by the user, adjusts the current playback point of the virtual reality media program as the current playback point progresses from the beginning of the virtual reality media program to the end of the virtual reality media program.

14. The method of claim 1, wherein:

the interactive user interface is semi-transparent; and

the presenting of the interactive user interface within the field of view together with the content of the immersive virtual reality world includes superimposing the semi-transparent interactive user interface over the content of the immersive virtual reality world such that the content of the immersive virtual reality world behind the interactive user interface can be at least partially viewed by the user.

15. The method of claim 1, further comprising:

receiving, by the virtual reality media system, data representative of camera-captured real-world scenery that is separate from real-world scenery in an immediate vicinity of the user, the data representative of the camera-captured real-world scenery captured by at least one video camera arranged to capture a 360-degree image of the real-world scenery around a center point corresponding to the video camera; and

generating, by the virtual reality media system based on the received data representative of the camera-captured real-world scenery, the immersive virtual reality world.

16. The method of claim 1, embodied as computer-executable instructions on at least one non-transitory computer-readable medium.

17. A method comprising:

presenting, by a virtual reality media system on a display screen of a media player device associated with a user, a field of view of an immersive virtual reality world, the field of view including content of the immersive virtual reality world and dynamically changing in response to user input provided by the user as the user experiences the immersive virtual reality world;

detecting, by the virtual reality media system, that a gaze of the user is directed for a first predetermined amount of time at a gaze target included within the field of view, the detecting including displaying, at a location within the field of view associated with the gaze of the user and while the gaze of the user is directed at the gaze target, a graphical indicator that indicates

that the gaze of the user is currently directed at the gaze target,

a first length of time that has elapsed toward the predetermined amount of time, and

a second length of time that remains before the predetermined amount of time will elapse;

presenting, by the virtual reality media system in response to the detecting that the gaze of the user is directed for the first predetermined amount of time at the gaze target, an interactive user interface associated with the gaze target, the interactive user interface presented within the field of view together with the content of the immersive virtual reality world and including a gaze-selectable control;

detecting, by the virtual reality media system while the interactive user interface is being presented within the field of view, that the gaze of the user is directed at the gaze-selectable control for a second predetermined amount of time; and

performing, by the virtual reality media system based on the detecting that the gaze of the user is directed at the gaze-selectable control for the second predetermined amount of time, an operation associated with the gaze-selectable control.

18. A system comprising:

at least one physical computing device that;

presents, on a display screen of a media player device associated with a user, a field of view of an immersive virtual reality world, the field of view including content of the immersive virtual reality world and dynamically changing in response to user input provided by the user as the user experiences the immersive virtual reality world;

detects that a gaze of the user is directed for a predetermined amount of time at a gaze target included within the field of view by performing operations including displaying, at a location within the field of view associated with the gaze of the user and while the gaze of the user is directed at the gaze target, a graphical indicator that indicates

that the gaze of the user is currently directed at the gaze target,

a first length of time that has elapsed toward the predetermined amount of time, and

a second length of time that remains before the predetermined amount of time will elapse; and

presents, in response to the detection that the gaze of the user is directed for the predetermined amount of time at the gaze target, an interactive user interface associated with the gaze target, the interactive user interface presented within the field of view together with the content of the immersive virtual reality world.

19. The system of claim 18, wherein:

the operations performed by the at least one physical computing device to detect that the gaze of the user is directed for the predetermined amount of time at the gaze target further include

tracking a gaze reticle persistently centered within the field of view with respect to at least one of a vertical dimension of the field of view and a horizontal dimension of the field of view,

determining, based on the tracking, that the gaze reticle is directed at the gaze target, and

determining, based on the tracking, that the gaze reticle remains persistently directed at the gaze target for the predetermined amount of time; and

the operation of displaying the graphical indicator is performed

in response to the determining that the gaze reticle is directed at the gaze target, and

concurrently with the gaze reticle remaining persistently directed at the gaze target for the predetermined amount of time.

20. The system of claim 18, wherein:

the operations performed by the at least one physical computing device to detect that the gaze of the user is directed for the predetermined amount of time at the gaze target further include

tracking eye movements of the user as the user experiences the immersive virtual reality world,

determining, based on the eye movements of the user, that the user is looking at the gaze target, and

determining, based on the eye movements of the user, that the user continues to look at the gaze target for the predetermined amount of time; and

the operation of displaying the graphical indicator is performed

in response to the determining that the user is looking at the gaze target based on the eye movements of the user, and

concurrently with the user continuing to look at the gaze target for the predetermined amount of time.