Methods and systems for gaze-based control of virtual reality media content

• US 10,048,751 B2
• Filed: 03/31/2016
• Issued: 08/14/2018
• Est. Priority Date: 03/31/2016
• Status: Active Grant

• Current Assignee: Verizon Patent and Licensing Incorporated (Verizon Communications Inc.)
• Original Assignee: Verizon Patent and Licensing Incorporated (Verizon Communications Inc.)
• Inventors: Jaafar, Ali, Khalid, Mohammad Raheel
• Primary Examiner(s): Williams, Kimberly A
• Application Number:
  US15/087,831
• Publication Number:
  US 20170285737A1
• Time in Patent Office: 866 Days
• Field of Search: 345633
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/70 of video data
• CPC Class Codes: G06F 16/75 Clustering; Classification
• CPC Class Codes: G06F 16/78 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2219/2024 Style variation

Abstract

An exemplary virtual reality media system presents a field of view of an immersive virtual reality world on a display screen of a media player device associated with a user. The field of view includes content of the immersive virtual reality world and dynamically changes in response to user input provided by the user as the user experiences the immersive virtual reality world. Additionally, the virtual reality media system detects that a gaze of the user is directed for a predetermined amount of time at a gaze target included within the field of view. In response to the detection, the virtual reality media system presents an interactive user interface associated with the gaze target. In some examples, the interactive user interface is presented within the field of view together with the content of the immersive virtual reality world. Corresponding methods and systems are also described.

Claims

1. A method comprising:

presenting, by a virtual reality media system on a display screen of a media player device associated with a user, a field of view of an immersive virtual reality world, wherein the field of view includes content of the immersive virtual reality world and dynamically changes in response to user input provided by the user as the user experiences the immersive virtual reality world;

detecting, by the virtual reality media system, that a gaze of the user is directed for a predetermined amount of time at a gaze target included within the field of view, the gaze target comprising an object integrated into the content of the immersive virtual reality world and the detecting including

tracking a gaze reticle within the field of view that is invisible to the user as the user experiences the immersive virtual reality world through the field of view,

determining, based on the tracking, that the gaze reticle is directed at the gaze target,

determining, based on the tracking, that the gaze reticle remains persistently directed at the gaze target for the predetermined amount of time,

displaying, within the field of view based on the determining that the gaze reticle is directed at the gaze target, a graphical indicator indicating that the gaze of the user is currently directed at the gaze target, the graphical indicator presented at a persistent location in the field of view associated with the gaze reticle, and

indicating, within the field of view and by way of the graphical indicator concurrently with the determining that the gaze reticle remains persistently directed at the gaze target for the predetermined amount of time, a running time that has elapsed toward the predetermined amount of time; and

presenting, by the virtual reality media system in response to the detecting that the gaze of the user is directed for the predetermined amount of time at the gaze target, an interactive user interface associated with the gaze target, the interactive user interface presented within the field of view together with the content of the immersive virtual reality world.

2. The method of claim 1, wherein the object comprised in the gaze target is a virtual object.

3. The method of claim 2, wherein the interactive user interface associated with the gaze target includes information not presented to the user within the field of view outside of the interactive user interface, the information associated with the virtual object.

4. The method of claim 1, wherein the gaze target is graphically highlighted to stand out from other content of the immersive virtual reality world surrounding the gaze target.

5. The method of claim 1, wherein the gaze reticle is persistently centered within the field of view with respect to at least one of a vertical dimension of the field of view and a horizontal dimension of the field of view.

6. The method of claim 1, wherein:

the immersive virtual reality world is associated with a virtual reality media program, the virtual reality media program including a beginning and an end;

the virtual reality media program is presented to the user over a period of time as a current playback point of the virtual reality media program progresses from the beginning of the virtual reality media program to the end of the virtual reality media program; and

the interactive user interface associated with the gaze target includes a gaze-selectable control that, when selected by the user, adjusts the current playback point of the virtual reality media program as the current playback point progresses from the beginning of the virtual reality media program to the end of the virtual reality media program.

7. The method of claim 1, wherein the interactive user interface associated with the gaze target includes a gaze-selectable control, the method further comprising:

detecting, by the virtual reality media system in response to the presenting of the interactive user interface within the field of view, that the gaze of the user is directed at the gaze-selectable control for a second predetermined amount of time; and

performing, by the virtual reality media system based on the detecting that the gaze of the user is directed at the gaze-selectable control for the second predetermined amount of time, an operation associated with the gaze-selectable control.

8. The method of claim 1, wherein the interactive user interface associated with the gaze target includes a game that is controlled exclusively by one or more gaze-based gestures.

9. The method of claim 1, wherein:

the interactive user interface is semi-transparent; and

the presenting of the interactive user interface within the field of view together with the content of the immersive virtual reality world includes superimposing the semi-transparent interactive user interface over the content of the immersive virtual reality world such that the content of the immersive virtual reality world behind the interactive user interface can be at least partially viewed by the user.

10. The method of claim 1, further comprising:

receiving, by the virtual reality media system, data representative of camera-captured real-world scenery, the data representative of the camera-captured real-world scenery captured by at least one video camera arranged to capture a 360-degree image of the real-world scenery around a center point corresponding to the video camera; and

generating, by the virtual reality media system based on the received data representative of the camera-captured real-world scenery, the immersive virtual reality world.

11. The method of claim 1, wherein:

the user input provided by the user includes a change to an orientation of the display screen of the media player device with respect to at least one axis of at least two orthogonal axes;

the media player device is configured to detect the change to the orientation of the display screen as the user experiences the immersive virtual reality world; and

the dynamic changing of the content included within the field of view includes changing the content to represent a view of the immersive virtual reality world that is determined to be visible from a viewpoint of the user within the immersive virtual reality world according to the change to the orientation of the display screen with respect to the at least one axis.

12. The method of claim 1, embodied as computer-executable instructions on at least one non-transitory computer-readable medium.

13. A method comprising:

presenting, by a virtual reality media system on a display screen of a media player device associated with a user, a field of view of an immersive virtual reality world, wherein the field of view includes content of the immersive virtual reality world and dynamically changes in response to user input provided by the user as the user experiences the immersive virtual reality world;

detecting, by the virtual reality media system, that a gaze of the user is directed for a first predetermined amount of time at a gaze target included within the field of view, the gaze target comprising an object integrated into the content of the immersive virtual reality world and the detecting including

tracking a gaze reticle within the field of view that is invisible to the user as the user experiences the immersive virtual reality world through the field of view,

determining, based on the tracking, that the gaze reticle is directed at the gaze target,

determining, based on the tracking, that the gaze reticle remains persistently directed at the gaze target for the predetermined amount of time,

displaying, within the field of view based on the determining that the gaze reticle is directed at the gaze target, a graphical indicator indicating that the gaze of the user is currently directed at the gaze target, the graphical indicator presented at a persistent location in the field of view associated with the gaze reticle, and

indicating, within the field of view and by way of the graphical indicator concurrently with the determining that the gaze reticle remains persistently directed at the gaze target for the predetermined amount of time, a running time that has elapsed toward the predetermined amount of time;

presenting, by the virtual reality media system in response to the detecting that the gaze of the user is directed for the first predetermined amount of time at the gaze target, an interactive user interface associated with the gaze target, the interactive user interface presented within the field of view together with the content of the immersive virtual reality world and including a gaze-selectable control;

detecting, by the virtual reality media system while the interactive user interface is being presented within the field of view, that the gaze of the user is directed at the gaze-selectable control for a second predetermined amount of time; and

performing, by the virtual reality media system based on the detecting that the gaze of the user is directed at the gaze-selectable control for the second predetermined amount of time, an operation associated with the gaze-selectable control.

14. The method of claim 13, embodied as computer-executable instructions on at least one non-transitory computer-readable medium.

15. A system comprising:

at least one physical computing device that;

presents, on a display screen of a media player device associated with a user, a field of view of an immersive virtual reality world, wherein the field of view includes content of the immersive virtual reality world and dynamically changes in response to user input provided by the user as the user experiences the immersive virtual reality world;

detects that a gaze of the user is directed for a predetermined amount of time at a gaze target included within the field of view, the gaze target comprising an object integrated into the content of the immersive virtual reality world and the detection performed by

tracking a gaze reticle within the field of view that is invisible to the user as the user experiences the immersive virtual reality world through the field of view,

determining, based on the tracking, that the gaze reticle is directed at the gaze target,

determining, based on the tracking, that the gaze reticle remains persistently directed at the gaze target for the predetermined amount of time,

displaying, within the field of view based on the determining that the gaze reticle is directed at the gaze target, a graphical indicator indicating that the gaze of the user is currently directed at the gaze target, the graphical indicator presented at a persistent location in the field of view associated with the gaze reticle, and

indicating, within the field of view and by way of the graphical indicator concurrently with the determining that the gaze reticle remains persistently directed at the gaze target for the predetermined amount of time, a running time that has elapsed toward the predetermined amount of time; and

presents, in response to the detection that the gaze of the user is directed for the predetermined amount of time at the gaze target, an interactive user interface associated with the gaze target, the interactive user interface presented within the field of view together with the content of the immersive virtual reality world.

16. The system of claim 15, wherein the gaze reticle is persistently centered within the field of view with respect to at least one of a vertical dimension of the field of view and a horizontal dimension of the field of view.

17. The system of claim 15, wherein the object comprised in the gaze target is a virtual object.

18. The system of claim 17, wherein the interactive user interface associated with the gaze target includes information not presented to the user within the field of view outside of the interactive user interface, the information associated with the virtual object.

19. The system of claim 15, wherein the gaze target is graphically highlighted to stand out from other content of the immersive virtual reality world surrounding the gaze target.

20. The system of claim 15, wherein the at least one physical computing device further:

receives data representative of camera-captured real-world scenery, the data representative of the camera-captured real-world scenery captured by at least one video camera arranged to capture a 360-degree image of the real-world scenery around a center point corresponding to the video camera; and

generates, based on the received data representative of the camera-captured real-world scenery, the immersive virtual reality world.