Camera based safety mechanisms for users of head mounted displays

• US 9,630,105 B2
• Filed: 07/16/2014
• Issued: 04/25/2017
• Est. Priority Date: 09/30/2013
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Inventors: Stafford, Jeffrey Roger
• Primary Examiner(s): D'Agostino, Paul A
• Application Number:
  US14/333,500
• Publication Number:
  US 20150094142A1
• Time in Patent Office: 1,014 Days
• Field of Search: None
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/212 using sensors worn by the player, e.g. for measuring heart beat or leg activity
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/25 Output arrangements for video game devices
• CPC Class Codes: A63F 13/26 having at least one additional display device, e.g. on the game controller or outside a game booth
• CPC Class Codes: A63F 13/49 Saving the game status; Pausing or ending the game
• CPC Class Codes: A63F 13/5255 according to dedicated instructions from a player, e.g. using a secondary joystick to rotate the camera around a player's character
• CPC Class Codes: A63F 13/533 for prompting the player, e.g. by displaying a game menu
• CPC Class Codes: A63F 13/537 using indicators, e.g. showing the condition of a game character on screen
• CPC Class Codes: A63F 2300/306 for displaying a marker associated to an object or location in the game field
• CPC Class Codes: A63F 2300/538 for performing operations on behalf of the game client, e.g. rendering
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L

Abstract

The disclosure provides methods and systems for warning a user of a head mounted display during gameplay of a video game. A game is executed causing interactive scenes of the game to be rendered on a display portion of a head mounted display (HMD) worn by a user. A change in position of the HMD worn by the user, while the user is interacting with the game, is detected. The change in position is evaluated, the evaluation causing a signal to be generated when the change exceeds a pre-defined threshold value. When the signal is generated, content is sent to interrupt the interactive scenes being rendered on the display portion of the HMD. The data sent provides descriptive context for the signal.

Claims

1. A method, comprising:

executing a game, the execution causing interactive scenes of the game to be rendered on a display portion of a head mounted display (HMD) worn by a user, the HMD when worn by the user is configured to block a view to a real-world environment;

determining coordinates of the HMD in a three-dimensional space of the real-world environment, the coordinates used to identify a current position of the user wearing the HMD;

monitoring a movement of the HMD during execution of the game, the movement of the HMD determined by identifying a change in one or more coordinates of the HMD in the three-dimensional space;

evaluating the change in the one or more coordinates of the HMD worn by the user to determine when the user is approaching a physical object present in the real-world environment;

blending in an image of the physical object with the interactive scenes when the evaluating determines that the user continues to approach the physical object; and

fading out the image of the physical object when the evaluating determines that the user is moving away from the physical object,

wherein the method operations are performed by a processor.

2. The method of claim 1, further comprising,

determining a position of the physical object in the real-world environment using a depth sensing device that collects data usable to map depth information of the real-world environment.

3. The method of claim 2, wherein the depth sensing device is one of a depth sensor, or an image sensor, or a stereo three-dimensional camera, or an infrared camera, or an ultraviolet camera, or a video camera, or a radar device, or a laser radar device, or a depth camera, or combinations of any two or more thereof.

4. The method of claim 1, further comprising,

performing a calibration operation to identify locations of the physical object and other physical objects in the real-world environment;

and

using the locations of the physical object or other physical objects in the real-world environment and the change in the one or more coordinates of the HMD to assist in determining when the user is approaching or moving away from the physical object.

5. The method of claim 1, further comprising,

providing a warning on the display portion of the HMD that includes instructions on how the user is to move away from the physical object.

6. The method of claim 1, wherein blending further includes increasing a brightness of the image of the physical object as the user approaches closer to the physical object.

7. A method, comprising:

executing an interactive application, the execution causing interactive content of the interactive application to be rendered on a display portion of a head mounted display (HMD) worn by a user, the HMD when worn by the user is configured to block a view to a real-world environment;

determining coordinates of the HMD in a three-dimensional space of the real-world environment, the coordinates used to identify a current position of the user wearing the HMD;

monitoring a movement of the HMD during execution of the interactive application, the movement of the HMD determined by identifying a change in one or more coordinates of the HMD in the three-dimensional space;

evaluating the change in the one or more coordinates of the HMD worn by the user to determine when the user is approaching a physical object present in the real-world environment;

blending in an image of the physical object with the interactive content when the evaluating determines that the user continues to approach the physical object; and

fading out the image of the physical object when the evaluating determines that the user is moving away from the physical object,

wherein the method operations are performed by a processor.

8. The method of claim 7, further comprising,

performing a calibration operation to identify locations of the physical object and other physical objects in the real-world environment;

and

using the locations of the physical object or other physical objects in the real-world environment and the change in the one or more coordinates of the HMD to assist in determining when the user is approaching or moving away from the physical object.