Virtual reality distraction monitor

• US 10,139,901 B2
• Filed: 07/05/2016
• Issued: 11/27/2018
• Est. Priority Date: 07/05/2016
• Status: Active Grant

• Current Assignee: Immersv, Inc.
• Original Assignee: Immersv, Inc.
• Inventors: Gentilin, John Charles, Shah, Mihir, Buck, Andrew Raymond, Linsner, Jonathan
• Primary Examiner(s): Chen, Frank
• Application Number:
  US15/202,375
• Publication Number:
  US 20180012408A1
• Time in Patent Office: 875 Days
• Field of Search: None
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 9/445 Program loading or initiating bootstrapping G06F9/4401; security arrangements for program loading or initiating G06F21/57
• CPC Class Codes: H04L 67/141 Setup of application sessions admission control or resource allocation in data switching networks H04L47/70
• CPC Class Codes: H04L 67/52 specially adapted for the location of the user terminal

Abstract

A method and apparatus for tracking distractions in a virtual reality environment is provided. In an embodiment, data identifying an area of interest is stored with a particular virtual reality application on a client computing device. The client computing device tracks a point of focus during execution of the application on the client computing device. In response to determining that the point of focus differs from the area of interest, the client computing device executes an application event.

Claims

1. A computer-implemented method comprising:

storing on a client computing device, with a particular digital graphical virtual reality application, digital data identifying one or more particular areas of interest within the particular digital graphical virtual reality application;

tracking, through the particular digital graphical virtual reality application executing on the client computing device, a point of focus of a video display of the client computing device executing the particular digital graphical virtual reality application;

determining that the point of focus of the video display of the client computing device executing the particular digital graphical virtual reality application is different than any of the one or more particular areas of interest;

in response to the determining, causing execution of an application event through the particular digital graphical virtual reality application or a client computing device event through the client computing device, the application event comprising storing application metadata that identifies a virtual location of the client computing device in the virtual reality application.

2. The computer-implemented method of claim 1 wherein the application metadata further identifies one or more of a time within the application, one or more previous events within the application, or one or more prior actions of the particular user.

3. The computer-implemented method of claim 1, wherein the digital data identifying one or more particular areas of interest is generated automatically by a processor of a server computer from heat map data that has been received from a plurality of client computing devices and which identifies points of focus of video displays of the plurality of client computing devices.

4. The computer-implemented method of claim 1, further comprising:

tracking the point of focus of the video display at a first point of time in a reference digital graphical virtual reality application;

tracking the point of focus of the video display at a second point of time in the particular digital graphical virtual reality application;

identifying the second point of time as similar to the first point of time;

determining that the point of focus of the video display of the client computing device executing the particular digital graphical virtual reality application is different than the one or more particular areas of interest by determining that the point of focus of the video display at the second point of time is different than the point of focus of the video display at the first point of time.

5. The computer-implemented method of claim 4 wherein the reference digital graphical virtual reality application is a different application than the particular digital graphical virtual reality application.

6. The computer-implemented method of claim 1 wherein the application event comprises an audiovisual event in the particular digital graphical virtual reality application to attract the particular user to shift the point of focus to the particular area of interest.

7. The computer-implemented method of claim 1, further comprising:

identifying a plurality of times in which the point of focus of the video display is different from a stored area of interest;

in response to the identifying, identifying a state of a particular user associated with the client computing device;

in response to identifying the state of the particular user, modifying execution of the particular digital graphical virtual reality application based, at least in part, on the state of the particular user.

8. The computer-implemented method of claim 1, wherein the client computing device comprises a head mounted display.

9. The computer-implemented method of claim 1, wherein the client computing device event comprises the client computing device vibrating, playing a particular sound, or altering a visual display of the client computing device.

10. The computer-implemented method of claim 1, wherein the point of focus of the video display changes based, at least in part, on the client computing device detecting motion of the client computing device.

11. A computer-implemented method comprising:

receiving, at a server computer, a particular digital graphical virtual reality application;

generating, at the server computer, instructions which, when executed by a client computing device, cause;

tracking, through the particular digital graphical virtual reality application executing on the client computing device, a point of focus of a video display of the client computing device executing the particular digital graphical virtual reality application;

sending, to the server computer, first digital data identifying the point of focus of the video display at one or more times during execution of the particular digital graphical virtual reality application;

receiving, from a plurality of client computing devices, second digital data identifying points of focus for a plurality of users of the particular digital graphical virtual reality application at the one or more times;

generating, from the second digital data, one or more heat maps for the particular digital graphical virtual reality application at the one or more times;

identifying, from the one or more heat maps, an area of interest for the particular digital graphical virtual reality application;

generating instructions which, when executed by a particular client computing device, cause;

tracking, through the particular digital graphical virtual reality application executing on the client computing device, a point of focus of a video display of the particular client computing device executing the particular digital graphical virtual reality application;

determining, at a particular time of the one or more times, that the point of focus of the video display is different from the area of interest;

in response to determining, causing execution of an application event through the particular digital graphical virtual reality application or a client computing device event through the client computing device.

12. The computer-implemented method of claim 11, further comprising:

receiving, at the server computer, second digital data identifying an area of interest for the particular digital graphical virtual reality application;

wherein the instructions which, when executed by a client computing device, further cause;

determining that, at the one or more times during execution of the particular digital graphical virtual reality application, the point of focus of the video display is different than the area of interest for the particular digital graphical virtual reality application;

sending, to the server computer, the first digital data in response to the determining.

13. The computer-implemented method of claim 12, further comprising:

receiving, at the server computer, along with the first digital data, third digital data identifying an application state of the particular digital graphical virtual reality application at the one or more times;

determining, based, at least in part, on the third digital data, a state of a particular user associated with the client computing device.

14. The computer-implemented method of claim 13, wherein the application state comprises one or more of a virtual location of the video display, a time within the application, one or more previous events within the application, or one or more prior actions of a user within the particular digital graphical virtual reality application.

15. The computer-implemented method of claim 12, further comprising:

receiving, from a plurality of client computing devices, third digital data identifying points of focus for a plurality of users of the particular digital graphical virtual reality application at the one or more times;

generating, based on the third digital data, a reverse heat map identifying variations of points of focus from the area of interest.

16. The computer-implemented method of claim 11, further comprising:

receiving, at the server computer, second digital data identifying an area of interest for the particular digital graphical virtual reality application;

receiving, at the server computer from a plurality of client computing devices, third digital data identifying points of focus of video displays of the plurality of client computing devices executing the particular digital graphical virtual reality application;

computing, based on the second digital data and the third digital data, a relative length of focus on the area of interest of the video displays of the plurality of client computing devices executing the particular digital graphical virtual reality application.

17. A head mounted display system comprising:

a memory;

an inertial measurement unit configured to measure movement of the head mounted display system;

one or more processors communicatively coupled to the memory and configured to execute one or more instructions to cause the head mounted display system to perform;

executing, on the head mounted display system, a particular virtual reality application;

while executing the particular virtual reality application, receiving inertial measurement unit data from the inertial measurement unit;

based on the inertial measurement unit data, computing a distraction score;

storing, in the memory, a distraction score threshold;

determining that the computed distraction score at a particular time is greater than the distraction score threshold and, in response, storing application metadata that identifies a virtual location of the client computing device in the virtual reality application.

18. The head mounted display system of claim 17, wherein the one or more processors are further configured to perform:

computing, from the inertial measurement unit data, for a particular period of time, a rotation value indicating radians of movement of the head mounted display system over the particular period of time;

computing the distraction score by normalizing the rotation value for the particular period of time.