SYSTEMS AND METHODS FOR TRANSITIONING BETWEEN TRANSPARENT MODE AND NON-TRANSPARENT MODE IN A HEAD MOUNTED DIPSLAY

• US 20140364212A1
• Filed: 04/16/2014
• Published: 12/11/2014
• Est. Priority Date: 06/08/2013
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment Incorporated (Sony Group Corp.)
• Inventors: Osman, Steven, Mao, Xiaodong
• Granted Patent: US 9,908,048 B2
• Time in Patent Office: Days
• US Class Current: 463/31
• CPC Class Codes: A63F 13/00 Video games, i.e. games using an electronically generated display having two or more dimensions gaming systems which provide a financial reward G07F17/32
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/212 using sensors worn by the player, e.g. for measuring heart beat or leg activity
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/26 having at least one additional display device, e.g. on the game controller or outside a game booth
• CPC Class Codes: A63F 13/5255 according to dedicated instructions from a player, e.g. using a secondary joystick to rotate the camera around a player's character
• CPC Class Codes: A63F 13/537 using indicators, e.g. showing the condition of a game character on screen
• CPC Class Codes: A63F 2300/6653 for altering the visibility of an object, e.g. preventing the occlusion of an object, partially hiding an object
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays

Abstract

Systems and methods for executing a game presented on a screen of a head mounted display include executing a game. The execution of the game renders interactive scenes of the game on the screen of the HMD. Images identifying a shift in gaze direction of the user wearing the HMD, are received. The gaze shift is detected during viewing of the interactive scenes presented on the HMD screen. Real-world images that are in line with the gaze direction of the user, are captured from a forward-facing camera of the HMD. A portion of the screen is transitioned from a non-transparent mode to a semi-transparent mode in response to the shift in the gaze direction such that at least part of the real world images are presented in the portion of the screen rendering the interactive scenes of the game. The transparent mode is discontinued after a period of time.

Claims

1. A method for executing a game presented on a screen of a head mounted display (HMD), comprising,

executing a game, the execution rendering interactive scenes of the game on the screen;

receiving images of the HMD worn by a user to identify a first spatial position of the HMD relative to a capture location that is directed toward the user;

receiving images of a controller held by the user to identify a second spatial position of the controller relative to the capture location, the controller providing input to at least partially drive interaction with the game being executed;

receiving images that identify a shift in gaze direction of the user viewing the interactive scenes of the game presented on the screen of the HMD;

receiving real-world images in line with the gaze direction of the user, captured from a forward direction of the HMD;

transitioning a portion of the screen to a transparent mode, in response to detection of the shift in the gaze direction, such that at least a portion of the captured real-world images is presented in the portion of the screen rendering the interactive scenes; and

discontinuing the transparent mode after a period of time,

wherein the method operations are performed by a processor.

2. The method of claim 1, wherein the game is executed on a computer that is communicatively connected to the head mounted display.

3. The method of claim 1, wherein the game is executed on a game cloud and the head mounted display is communicatively connected to the game cloud.

4. The method of claim 1, wherein the first spatial position and the second spatial position are identified to provide six-axis data.

5. The method of claim 1, wherein the transitioning is to a semi-transparent mode and is performed in response to a trigger event detected at the HMD.

6. The method of claim 5, wherein the trigger event is one of a sound or a movement detected in a real-world vicinity of the user wearing the HMD, an audio or a visual command provided by a user of the HMD, an audio or input detected at the HMD, a command provided from the HMD or the controller, or any combinations thereof.

7. The method of claim 1, wherein the presenting includes rendering the interactive scenes of the game in background and the captured real-world images in foreground of the screen.

8. The method of claim 1, wherein the presenting includes rendering the interactive scenes of the game in foreground and the captured real-world images in background of the screen.

9. A method for executing a game presented on a screen of a head mounted display (HMD), comprising,

executing a game, the execution rendering interactive scenes of the game on the screen of the HMD;

receiving images that identify a shift in gaze direction of a user wearing the HMD when viewing the interactive scenes presented on the screen of the HMD;

receiving real-world images that are in line with the gaze direction of the user, the real-world images being captured from a forward-facing camera of the HMD;

transitioning a portion of the screen to a semi-transparent mode in response to the shift in the gaze direction, such that at least part of the captured real-world images is presented in the portion of the screen rendering the interactive scenes; and

discontinuing the semi-transparent mode after a period of time,

wherein the method operations are performed by a processor.

10. The method of claim 9, further comprising,

receiving images of the HMD to identify a first spatial position of the HMD relative to a capture location that is directed toward the user;

and

receiving images of a controller held by the user to identify a second spatial position of the controller relative to the capture location, the controller providing input to at least partially drive interaction with the game being executed.

11. A method for executing a game presented on a screen of a head mounted display (HMD), comprising,

executing a game;

rendering interactive scenes of the game on the screen of the HMD as the game is executed;

detecting a trigger event occurring within a vicinity of the HMD during the execution of the game, the trigger event causing a transition of the screen from a non-transparent mode to a fully transparent mode, the transitioning includes adjusting one or more optic characteristics of the screen to switch the screen to the fully transparent mode, wherein the fully transparent mode enables a user to view real-world environment in the vicinity of the HMD through the screen; and

discontinuing the fully transparent mode after a period of time,

wherein the method operations are performed by a processor.

12. The method of claim 11, further includes,

examining the trigger event to determine if the transition of the screen has to be effectuated;

and

performing the transition based on the determination.

14. The method of claim 11, wherein the event is detected to occur when a gaze condition of the user wearing the HMD is identified.

15. The method of claim 14, wherein the gaze condition is one of,

(a) detecting a gaze direction for a period time;

(b) detecting a gaze pattern over a period of time; and

(c) detecting changes in gaze directions.

16. The method of claim 11, wherein the transitioning is performed in linear or non-linear gradual increments.

17. The method of claim 11, wherein discontinuing further includes adjusting optic characteristics of the screen to switch to a non-transparent mode, the switch to non-transparent mode causes rendering of the interactive scenes of the game on the screen.

18. The method of claim 11, wherein the screen is a liquid crystal display screen.

19. The method of claim 11, wherein the game is executed on a computer that is communicatively connected to the HMD, on a game cloud, wherein the HMD is communicatively connected to the game cloud or partly on the game cloud and partly on the computer that are communicatively connected to the HMD, the interactive scenes of the game being streamed from the computer or the game cloud.

20. The method of claim 11, wherein the trigger event is a sound or a movement detected near a real-world vicinity of the user wearing the HMD, an audio command or a visual command provided by a user, an input detected at the HMD, a command provided from a controller, a command provided from the HMD or any combinations thereof.

13. The method of claim 13, wherein the determination includes,

verifying a cause of the trigger event;

determining if the cause lasts for at least a pre-defined threshold period; and

effectuating the transition when the cause lasts for at least the pre-defined threshold period.

21. A method for executing a game presented on a screen of a head mounted display (HMD), comprising,

executing a game, the execution rendering interactive scenes of the game on the screen;

receiving images of the HMD worn by a user to identify a first spatial position of the HMD relative to a capture location that is directed toward the user;

receiving images of a controller held by the user to identify a second spatial position of the controller relative to the capture location, the controller providing input to at least partially drive interaction with the game being executed;

modifying a direction of view into the interactive scenes rendered on the screen when the HMD changes an orientation at the first spatial position; and

receiving input from the controller, the input including changes in orientation at the second spatial position;

wherein the first spatial position and the second spatial position are correlated during the rendering of the interactive scenes and wherein the method operations are performed by a processor.

22. The method of claim 21, wherein the changes in orientation include changes to at least one of position or motion.

23. The method of claim 21, further comprising,

detecting an occurrence of a trigger event within a vicinity of the HMD during the execution of the game, the trigger event causing a transition of the screen from a non-transparent mode to a fully transparent mode, the transitioning includes adjusting one or more optic characteristics of the screen to switch the screen to the fully transparent mode, wherein the fully transparent mode enables a user to view real-world environment in the vicinity of the HMD through the screen.

24. The method of claim 23, further comprising,

discontinuing the fully transparent mode after a period of time.

25. The method of claim 23, wherein the screen is a liquid crystal display screen.

26. A method, comprising:

executing a game to enable gameplay, the execution causing rendering of interactive scenes of the game on a screen of a head mounted display (HMD);

detecting real-world object in proximity to a user wearing the HMD during active gameplay; and

providing a notification to the screen or an audio output of the HMD regarding proximity of the real-world object to the user,

wherein the method operations are performed by a processor.

27. The method of claim 26, wherein detecting and providing notification are performed in response to a trigger event detected at the HMD.

28. The method of claim 27, wherein the trigger event is detected in response to the user'"'"'s actions in real-world environment.

29. The method of claim 26, wherein the notification is provided when the proximity of the real-world object is within a pre-defined threshold distance.

30. The method of claim 26, wherein the notification is provided at the HMD in one of textual format, graphical format, audio format, haptic format or any combinations thereof.

31. The method of claim 26, wherein providing notification further includes automatically transitioning the screen of the HMD to a semi-transparent mode or a transparent mode, the transitioning causes rendering of the real-world object on the screen of the HMD.