Systems and methods for reducing an effect of occlusion of a tracker by people

• US 10,440,240 B2
• Filed: 09/08/2017
• Issued: 10/08/2019
• Est. Priority Date: 09/30/2016
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Inventors: Osman, Steven
• Primary Examiner(s): Lim, Seng Heng
• Application Number:
  US15/699,844
• Publication Number:
  US 20180097975A1
• Time in Patent Office: 760 Days
• Field of Search: None
• CPC Class Codes: A61B 2034/2057 Details of tracking cameras
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/533 for prompting the player, e.g. by displaying a game menu
• CPC Class Codes: A63F 13/54 involving acoustic signals, e.g. for simulating revolutions per minute [RPM] dependent engine sounds in a driving game or reverberation against a virtual wall
• CPC Class Codes: A63F 2300/8082 Virtual reality
• CPC Class Codes: B25J 9/1602 characterised by the control system, structure, architecture
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G01S 5/16 using electromagnetic waves other than radio waves
• CPC Class Codes: G03B 9/60 by varying speed of movement of obturating members
• CPC Class Codes: H04N 13/106 Processing image signals for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 5/2226 Determination of depth image, e.g. for foreground/background separation determining depth by image analysis in general G06T7/50; segmentation by image analysis in general G06T7/10

Abstract

Systems and methods for reducing an effect of occlusion of a tracker by people are described. When a user is playing a video game using a head-mounted display, a field-of-view for tracking the user is obstructed by a spectator. The video game cannot be displayed at a normal frame rate to the user as a result of the obstruction. The systems and methods described herein determine and identify a source of the obstruction, and implement operations to remove the obstruction.

Claims

1. A method for identifying an obstruction to a trackable field-of-view (FOV) of a tracker, comprising:

executing interactive content for display of a plurality of images on a head-mounted display, wherein the head-mounted display is configured to be worn by a user, wherein the execution of the interactive content is changed based on changes in a position and orientation of the head-mounted display when tracked by the tracker;

determining that the trackable FOV is obstructed by a spectator during the execution of the interactive content when the spectator is not visible via the head-mounted display to the user, wherein the obstruction by the spectator reduces a quality factor associated with tracking of the position and orientation used for generating the plurality of images displayed on the head-mounted display; and

notifying the user that the quality factor has reduced as a result of the obstruction by the spectator, wherein said notifying is provided by a message rendered by the head-mounted display.

2. The method of claim 1, wherein determining that the trackable FOV is obstructed by the spectator comprises determining that the head-mounted display is blocked for a period of time, wherein the block reduces continuous tracking of the position and orientation during the period of time used to achieve a frame rate.

3. The method of claim 1, further comprising determining an identity of the spectator upon determining that the trackable FOV is obstructed by the spectator, wherein determining the identity of the spectator includes detecting, by the tracker, a human face proximate to the head-mounted display.

4. The method of claim 1, wherein the message includes an image notification for display on the head-mounted display to indicate to the user that the trackable FOV is obstructed by the spectator.

5. The method of claim 1, further comprising generating an audio notification for outputting a sound via the head-mounted display indicating that the trackable FOV is obstructed by the spectator.

6. The method of claim 1, wherein the tracker is a camera placed to have a view of the head-mounted display, or a combination of a projector and photosensors, or a camera of the head-mounted display.

7. The method of claim 1, further comprising determining an identity of the spectator upon determining that the trackable FOV is obstructed by the spectator, wherein determining the identity of the spectator comprises controlling a shutter speed of a camera and an aperture diaphragm of the camera to focus on the spectator instead of the head-mounted display.

8. The method of claim 1, wherein the reduction in the quality factor is associated with a drop in presentation quality of the plurality of images as displayed on the head-mounted display.

9. The method of claim 1, further comprising notifying the spectator that the spectator is obstructing the trackable FOV.

23. The method of claim 1, wherein when the quality factor is reduced, one of the plurality of images is not displayed.

24. The method of claim 1, wherein the reduction in the quality factor includes a decrease in a frame rate at which the plurality of images are displayed on the head-mounted display or a reduction in speed of generation of one of more game states based on the position and orientation.

10. A method for identifying a spectator to a user during interaction with interactive content on a head-mounted display, comprising:

determining that the spectator is within a trackable field-of-view (FOV) of a tracker, wherein said determining that the spectator is within the trackable FOV is performed when interactive content is executed to display the interactive content on the head-mounted display, wherein the head-mounted display is configured to be worn by the user during the execution of the interactive content;

determining that the spectator is identifiable upon determining that the spectator is within the trackable FOV;

determining that the spectator is obstructing the trackable FOV when the spectator is not visible via the head-mounted display to the user; and

notifying the user that the spectator is obstructing the trackable FOV in response to determining that the spectator is obstructing the trackable FOV.

11. The method of claim 10, wherein determining that the spectator is obstructing the trackable FOV comprises determining that the head-mounted display is blocked for a period of time, wherein the block reduces continuous tracking of a position and orientation of the head-mounted display during the period of time used to achieve a frame rate.

12. The method of claim 10, wherein notifying the user includes generating an image notification for display on the head-mounted display to indicate to the user that the trackable FOV is obstructed by the spectator.

13. The method of claim 10, wherein notifying the user includes generating an audio notification for outputting a sound via the head-mounted display indicating that the trackable FOV is obstructed by the spectator.

14. The method of claim 10, wherein the tracker is a camera placed to have a view of the head-mounted display, or is a combination of a projector and photosensors, or is a camera of the head-mounted display.

15. The method of claim 10, further comprising identifying the spectator upon determining that the spectator is obstructing the trackable FOV.

16. A system for identifying an obstruction to a trackable field-of-view (FOV) of a tracker, comprising:

a head-mounted display configured to be worn by a user to display interactive content; and

a processor coupled to the head-mounted display, wherein the processor is configured to;

execute the interactive content for display of a plurality of images on the head-mounted display, wherein the execution of the interactive content is changed based on changes in a position and orientation of the head-mounted display when tracked by the tracker;

determine that the trackable FOV is obstructed by a spectator during the execution of the interactive content when the spectator is not visible via the head-mounted display to the user, wherein the obstruction by the spectator reduces a quality factor associated with a determination of the position and orientation used to generate the plurality of images displayed on the head-mounted display;

identify the spectator upon determining that the trackable FOV is obstructed by the spectator; and

generate a notification to be output via the head-mounted display that the quality factor is reduced as a result of the obstruction by the spectator, wherein the notification is provided by a message rendered by the head-mounted display.

17. The system of claim 16, wherein to determine that the trackable FOV is obstructed by the spectator, the processor is configured to determine that the head-mounted display is blocked for a period of time, wherein the block of the head-mounted display affects continuous tracking of the position and orientation during the period of time used to achieve a frame rate.

18. The system of claim 16, wherein to identify the spectator, the processor is configured to change a focus of the tracker to the spectator from a focus on the head-mounted display.

19. The system of claim 16, wherein the message includes an image notification for display on the head-mounted display to indicate to the user that the trackable FOV is obstructed by the spectator.

20. The system of claim 16, wherein to generate the notification, the processor is configured to generate an audio notification for outputting a sound via the head-mounted display to indicate to the user that the trackable FOV is obstructed by the spectator.

21. The system of claim 16, wherein the tracker is a camera placed to have a view of the head-mounted display, or a combination of a projector and photosensors, or a camera of the head-mounted display.

22. The system of claim 16, wherein to identify the spectator, the processor is configured to control a shutter speed of a camera and an aperture diaphragm of the camera to focus on the spectator instead of the head-mounted display.