Virtual reality system including social graph

• US 10,691,202 B2
• Filed: 11/03/2017
• Issued: 06/23/2020
• Est. Priority Date: 07/28/2014
• Status: Active Grant

• Current Assignee: Verizon Patent and Licensing Incorporated (Verizon Communications Inc.)
• Original Assignee: Verizon Patent and Licensing Incorporated (Verizon Communications Inc.)
• Inventors: van Hoff, Arthur, Annau, Thomas M., Christensen, Jens
• Primary Examiner(s): Tran, Thai Q
• Assistant Examiner(s): Smith, Stephen R
• Application Number:
  US15/803,599
• Publication Number:
  US 20180059783A1
• Time in Patent Office: 963 Days
• Field of Search: 386201, 386223, 386224, 386341, 348 36- 39, 348 42- 60, 3482181, 345 8, 725 10
• CPC Class Codes: G06F 18/2431 Multiple classes
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06Q 30/0251 Targeted advertisements
• CPC Class Codes: G06Q 30/0261 based on user location
• CPC Class Codes: G06Q 30/0263 based upon Internet or website rating
• CPC Class Codes: G06Q 30/0273 Determination of fees for advertising
• CPC Class Codes: G06Q 50/01 Social networking
• CPC Class Codes: G06T 11/206 Drawing of charts or graphs
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: H04N 13/106 Processing image signals for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/111 Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 21/2668 Creating a channel for a dedicated end-user group, e.g. insertion of targeted commercials based on end-user profiles information retrieval from the Internet by querying with filtering and personalisation G06F16/9535; arrangements for replacing or switching information during the broadcast H04H20/10; push services over packet-switching network H04L12/1859; adaptation of message content in packet-switching networks H04L51/063
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/44218 : Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/4788 : communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 21/812 : involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 21/816 : involving special video data, e.g 3D video
• CPC Class Codes: H04N 23/698 : for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 7/181 : for receiving images from a plurality of remote sources

Abstract

The disclosure includes a system and method for receiving viewing data that describes a location of a first user'"'"'s gaze while viewing virtual reality content. The method also determining an object of interest in the virtual reality content based on the location of the first user'"'"'s gaze. The method also includes generating a social network that includes the first user as a member of the social network. The method also includes performing an action in the social network related to the object of interest.

Claims

1. A method comprising:

recording a compressed stream of video data describing an environment using a camera array, the camera array including a set of camera modules that are configured to provide at least a 3×

field of view overlap such that each pixel included in the environment is described by at least three camera modules included in the set of camera modules;

generating three-dimensional content based on the compressed stream of video data, wherein the three-dimensional content depicts the environment;

receiving viewing data that describes a location of a gaze of a user;

providing the three-dimensional content to the user based on the viewing data;

receiving sensor data that describes behavior of the user while the user is viewing the three-dimensional content, wherein the sensor data includes words spoken by the user;

determining locations of the gaze of the user with respect to the three-dimensional content;

generating, based on the locations of the gaze of the user, a gaze map that determines areas of interest of the three-dimensional content based on a number of user gazes for each location included in the determined locations;

determining, based on the words spoken by the user and the gaze map, that the user is interested in a first object in the three-dimensional content; and

providing the user with a first advertisement that is related to the first object.

2. The method of claim 1, wherein the sensor data further includes at least one of an indication that eyes of the user were darting, a pupil dilation of the eyes of the user, a heart rate of the user, and a level of perspiration of the user during viewing of the three-dimensional content.

3. The method of claim 1, wherein the first advertisement is provided as a graphical object and further comprising:

receiving a selection of the graphical object from the user; and

responsive to receiving the selection, providing the user with a third-party website.

4. The method of claim 2, further comprising:

receiving a voice input from the user; and

determining, based on the voice input, that the user is searching for the three-dimensional content.

5. The method of claim 1, further comprising:

determining that the user views a second object in the three-dimensional content for a threshold amount of time; and

responsive to the user viewing the second object for the threshold amount of time, providing the user with a second advertisement associated with the second object.

6. The method of claim 1, wherein the three-dimensional content is displayed on a first device and further comprising:

determining a location within the three-dimensional content when the user stops viewing the three-dimensional content;

determining that the user is viewing the three-dimensional content on a second device; and

providing the user with the three-dimensional content in a substantially same location.

7. The method of claim 1, further comprising determining a fee for displaying a second advertisement to the user at each area of interest represented in the gaze map.

8. A method comprising:

recording a compressed stream of video data describing an environment using a camera array, the camera array including a set of camera modules that are configured to provide at least a 3×

field of view overlap such that each pixel included in the environment is described by at least three camera modules included in the set of camera modules;

generating three-dimensional content based on the compressed stream of video data, wherein the three-dimensional content depicts the environment;

receiving viewing data that describes a location of a gaze of a user;

providing the three-dimensional content to a first user based on the viewing data;

determining that the first user is interested in an object in the three-dimensional content;

providing the first user with an advertisement that is related to the object;

generating a social graph that includes the first user and associates the first user with information about the three-dimensional content;

receiving information from a social network that indicates one or more geographic locations that the first user visited;

providing the first user with the three-dimensional content, wherein the three-dimensional content includes the one or more geographic locations;

determining that a second user from the social graph visited the one or more geographic locations that the first user visited; and

providing the second user with a synchronized three-dimensional experience that includes the one or more geographic locations and that is synchronized with the three-dimensional content viewed by the first user.

9. The method of claim 8, further comprising:

determining a type of three-dimensional content viewed by the first user; and

associating the first user with the type of three-dimensional content and the social graph.

10. The method of claim 9, further comprising:

identifying the second user from the social graph as being associated with the type of three-dimensional content; and

providing a recommendation to the first user to connect with the second user within the social network.

11. The method of claim 9, further comprising:

identifying the second user from the social graph as being associated with the type of three-dimensional content; and

providing a recommendation to the first user to view other three-dimensional content that was viewed by the second user.

12. A non-transitory computer storage medium encoded with a computer program, the computer program comprising instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

recording a compressed stream of video data describing an environment using a camera array, the camera array including a set of camera modules that are configured to provide at least a 3×

field of view overlap such that each pixel included in the environment is described by at least three camera modules included in the set of camera modules;

generating three-dimensional content based on the compressed stream of video data, wherein the three-dimensional content depicts the environment;

receiving viewing data that describes a location of a gaze of a user;

providing the three-dimensional content to a user based on the viewing data;

receiving sensor data that describes behavior of the user while the user is viewing the three-dimensional content, wherein the sensor data includes words spoken by the user;

determining locations of the gaze of the user with respect to the three-dimensional content;

generating, based on the locations of the gaze of the user, a gaze map that determines areas of interest of the three-dimensional content based on a number of user gazes for each location included in the determined locations;

determining, based on the words spoken by the user and the gaze map, that the user is interested in a first object in the three-dimensional content; and

providing the user with a first advertisement that is related to the first object.

13. The computer storage medium of claim 12, wherein the sensor data further includes at least one of an indication that eyes of the user were darting, a pupil dilation of the eyes of the user, a heart rate of the user, and a level of perspiration of the user during viewing of the three-dimensional content.

14. The computer storage medium of claim 13, wherein the first advertisement is provided as a graphical object and the instructions are further operable to perform operations comprising:

receiving a selection of the graphical object from the user; and

responsive to receiving the selection, providing the user with a third-party website.

15. The computer storage medium of claim 13, wherein the instructions are further operable to perform operations comprising:

receiving a voice input from the user; and

determining, based on the voice input, that the user is searching for the three-dimensional content.

16. The computer storage medium of claim 12, wherein the instructions are further operable to perform operations comprising:

determining that the user views a second object in the three-dimensional content for a threshold amount of time; and

responsive to the user viewing the second object for the threshold amount of time, providing the user with an advertisement associated with the object.

17. The computer storage medium of claim 12, wherein the three-dimensional content is displayed on a first device and the instructions are further operable to perform operations comprising:

determining a location within the three-dimensional content when the user stops viewing the three-dimensional content;

determining that the user is viewing the three-dimensional content on a second device; and

providing the user with the three-dimensional content in a substantially same location.

18. The computer storage medium of claim 12, wherein the first advertisement is a pop-up window that appears above the first object.