Delivery of Spectator Feedback Content to Virtual Reality Environments Provided by Head Mounted Display

• US 20180093185A1
• Filed: 02/08/2017
• Published: 04/05/2018
• Est. Priority Date: 09/30/2016
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Inventors: Black, Glenn T., Taylor, Michael G., Tokubo, Todd
• Granted Patent: US 11,071,915 B2
• Time in Patent Office: Days
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/26 having at least one additional display device, e.g. on the game controller or outside a game booth
• CPC Class Codes: A63F 13/537 using indicators, e.g. showing the condition of a game character on screen
• CPC Class Codes: A63F 13/5372 for tagging characters, objects or locations in the game scene, e.g. displaying a circle under the character controlled by the player
• CPC Class Codes: A63F 13/5375 for graphically or textually suggesting an action, e.g. by displaying an arrow indicating a turn in a driving game
• CPC Class Codes: A63F 13/5378 for displaying an additional top view, e.g. radar screens or maps using two or more virtual cameras concurrently A63F13/5252
• CPC Class Codes: A63F 13/56 Computing the motion of game characters with respect to other game characters, game objects or elements of the game scene, e.g. for simulating the behaviour of a group of virtual soldiers or for path finding
• CPC Class Codes: A63F 13/58 by computing conditions of game characters, e.g. stamina, strength, motivation or energy level
• CPC Class Codes: A63F 13/86 Watching games played by other players
• CPC Class Codes: A63F 2300/8082 Virtual reality
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 40/176 Dynamic expression
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces

Abstract

Systems and methods for processing operations for delivering spectator feedback content to a head mounted display (HMD) of an HMD user are provided. A computer-implemented method includes providing a VR scene to an HMD user and capturing images of one or more spectators usable to detect a presence and emotional state of the one or more spectators. The method also includes displaying within the VR scene of the HMD user one or more avatars representing the presence of the one or more spectators. Further, the method includes delivering an indication of the detected emotional state of the one or more spectators via the one or more avatars, respectively.

Claims

1. A computer-implemented method for processing operations for delivering spectator feedback content to a head mounted display (HMD) of an HMD user, comprising:

providing a virtual reality (VR) scene to the HMD of the HMD user;

capturing images of one or more spectators, the images usable to detect a presence of one or more spectators and an emotional state of the one or more spectators;

providing for display within the VR scene one or more avatars, the one or more avatars representing the presence of the one or more spectators, respectively; and

delivering an indication of a detected emotional state of the one or more spectators to the HMD for display within the VR scene, the indication being represented by a respective avatar within the VR scene.

2. The computer-implemented method of claim 1, wherein said delivering the indication of the detected emotional state includes changing a facial feature of the one or more avatars to at least partially correspond to the detected emotional state.

3. The computer-implemented method of claim 1, wherein the indication of the detected emotional state includes a text that at least partially describes the detected emotional state.

4. The computer-implemented method of claim 1, further comprising:

tracking an amount of time the one or more spectators have been waiting; and

providing, to the HMD for display within the VR scene, a representation of the amount of time the one or more spectators have been waiting.

5. The computer-implemented method of claim 1, further comprising:

providing, to the HMD for display within the VR scene, a player avatar within the VR scene, the player avatar representing the HMD user;

tracking an amount of time the HMD user has been using the HMD; and

providing, to the HMD for display within the VR scene, a representation of the amount of time the HMD user has been using the HMD, the representation including updating a game age of the player avatar.

6. The computer-implemented method of claim 1, further comprising:

capturing sound data of the one or more spectators, the sound data usable to determine that a spectator is providing a communication to the HMD user;

processing the sound data for identifying the spectator providing the communication to the HMD user; and

delivering, to the HMD, a representation of the communication, the representation indicating the spectator who provided the communication to the HMD user.

7. A computer-implemented method for processing operations for delivering information on a positional state of one or more spectators within a real world setting to a head mounted display (HMD) of a HMD user, comprising:

providing a VR scene to the HMD of the HMD user;

obtaining positional data on the one or more spectators, the positional data usable to determine a location of the one or more spectators relative to the HMD user in the real world setting; and

providing, to the HMD for display within the VR scene, an overhead view of the real world setting, the overhead view including a first indication of the location of the one or more spectators relative to the HMD.

8. The computer-implemented method of claim 7, wherein the overhead view includes a bird'"'"'s-eye-view.

9. The computer-implemented method of claim 7, further comprising:

determining that a personal space of a spectator is being encroached by the HMD user; and

providing, to the HMD for display within the overhead view of the VR scene, a second indication, the second indication for alerting the HMD user that the personal space of the spectator is being encroached on.

10. The computer-implemented method of claim 7, further comprising:

determining that a personal space of a spectator is being encroached by the HMD user;

detecting, within the VR scene, that one or more points of interest within the VR scene are in a direction or proximity that corresponds to the personal space of the spectator in the real world setting; and

moving, within the VR scene, at least one of the one or more points of interest of the VR scene to a different location within the VR scene, the different location not being in the direction or proximity that corresponds to the personal space of the spectator in the real world setting, wherein said moving at least one of the one or more points of interest enable the HMD user to discontinue from encroaching the personal space of the spectator while still being able to continue to interact with the VR scene.

11. The computer-implemented method of claim 9, wherein the second indication that the personal space of a spectator is being encroached includes an audible alert.

12. The computer-implemented method of claim 9, wherein the second indication includes an animation, the animation being within the overhead view of the VR scene.

13. The computer-implemented method of claim 9, further comprising:

providing, to the HMD for presentation within the VR scene, instruction for repositioning, the instruction for reposition including a visual or audible prompt indicating how the HMD user is to move in order to discontinue from encroaching on the personal space of the spectator.

14. A computer-implemented method for processing operations for delivering information on a positional state of one or more spectators within a real world setting to a head mounted display (HMD) of a HMD user, comprising:

providing a VR scene to the HMD of the HMD user;

obtaining positional data on the one or more spectators, the positional data usable to determine locations of the one or more spectators relative to the HMD user in the real world setting;

determining, based on the positional data, that a personal space of a spectator is being encroached; and

providing for display, within the VR scene, a representation of the spectator whose personal space is being encroached.

15. The computer-implemented method of claim 14, wherein the representation is based on images captured of the spectator.

16. The computer-implemented method of claim 14, wherein the representation is displayed at a display location within the VR scene that corresponds to a real world location of the spectator.

17. The computer-implemented method of claim 14, wherein the images captured of the spectator are at least partially obtained by one or more front facing cameras on the HMD.

18. The computer-implemented method of claim 14, further comprising:

providing, to the HMD for presentation, instructions for producing an audible alert when it is determined that a personal space of a spectator is being encroached.

19. The computer-implemented method of claim 14, further comprising:

providing, to the HMD for display within the VR scene, instruction for repositioning to avoid further encroaching on the personal space of the spectator.

20. The computer-implemented method of claim 14, further comprising:

detecting, within the VR scene, that one or more points of interest of the HMD user within the VR scene have a first display location that corresponds to the personal space of the spectator; and

moving, within the VR scene, at least one of the one or more points of interest within the VR scene to a second display location within the VR scene, the second display location not corresponding to the personal space of the spectator.