Methods for Rendering Interactive Content to a Head Mounted Display

• US 20140364215A1
• Filed: 06/04/2014
• Published: 12/11/2014
• Est. Priority Date: 06/09/2013
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment Incorporated (Sony Group Corp.)
• Inventors: Mikhailov, Anton, Mao, Xiadong
• Granted Patent: US 10,173,129 B2
• Time in Patent Office: Days
• US Class Current: 463/31
• CPC Class Codes: A63F 13/00 Video games, i.e. games using an electronically generated display having two or more dimensions gaming systems which provide a financial reward G07F17/32
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/212 using sensors worn by the player, e.g. for measuring heart beat or leg activity
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/26 having at least one additional display device, e.g. on the game controller or outside a game booth
• CPC Class Codes: A63F 13/285 Generating tactile feedback signals via the game input device, e.g. force feedback
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/5255 according to dedicated instructions from a player, e.g. using a secondary joystick to rotate the camera around a player's character
• CPC Class Codes: A63F 13/803 Driving vehicles or craft, e.g. cars, airplanes, ships, robots or tanks
• CPC Class Codes: A63F 2300/301 using an additional display connected to the game console, e.g. on the controller
• CPC Class Codes: A63F 2300/8082 Virtual reality
• CPC Class Codes: G02B 2027/0112 comprising device for genereting colour display
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0176 characterised by mechanical features
• CPC Class Codes: G02B 27/0179 Display position adjusting means not related to the information to be displayed
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G09G 2320/0233 : Improving the luminance or brightness uniformity across the screen
• CPC Class Codes: G09G 2320/0626 : for control of overall brightness
• CPC Class Codes: G09G 3/001 : using specific devices not provided for in groups G09G3/02 - G09G3/36, e.g. using an intermediate record carrier such as a film slide; Projection systems; Display of non-alphanumerical information, solely or in combination with alphanumerical information, e.g. digital display on projected diapositive as background slide projectors per se G03B23/00 = 42 HP
• CPC Class Codes: G09G 3/3406 : Control of illumination source illumination devices structurally associated with liquid crystal cells G02F1/1336
• CPC Class Codes: G09G 3/3413 : Details of control of colour illumination sources
• CPC Class Codes: H04N 13/254 : in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 13/271 : wherein the generated image signals comprise depth maps or disparity maps

Abstract

A method for executing a game presented on a screen of a head mounted display (HMD) is provided. Execution renders interactive scenes of the game on the screen. The method includes receiving data of the HMD worn by a user to identify an HMD spatial position. The method also includes receiving data of a controller held by the user to identify a controller spatial position. During game play, the method includes rendering a point of view into the interactive scenes of the game on the screen, where the point of view is a volume of space rendered in three-dimensions. The method includes adjusting the point of view as the HMD spatial position is detected to change. The controller can drive interactivity in the game and changes in position of the controller and the HMD are correlated to provide an immersive three-dimensional game play experience.

Claims

1. A method for executing a game to be presented on a screen of a head mounted display (HMD), comprising,

executing a game, the execution rendering interactive scenes of the game on the screen;

receiving data of the HMD worn by a user to identify an HMD spatial position;

receiving data of a controller held by the user to identify a controller spatial position;

sending data to the HMD for rendering a point of view into the interactive scenes of the game on the screen of the HMD; and

sending data to the HMD, the data is for adjusting the point of view as the HMD spatial position is detected to change, the method being executed by a processor.

2. The method of claim 1, wherein the controller is represented in the interactive scenes as a virtual object, the virtual object rendered in the interactive scenes in virtual spatial position that maps to the controller spatial position, and changes in the controller spatial position change the virtual spatial position of the virtual object.

3. The method of claim 1, wherein the interactive scenes are rendered in a three-dimension format, such that objects in the interactive scenes appear in front of the HMD and extend a distance into the interactive scenes.

4. The method of claim 1, wherein the HMD and controller spatial positions are tracked using one or more of image data, inertial data, wireless data, infrared data, or combinations thereof.

5. The method of claim 4, wherein tracking using image data includes detecting one or both of the HMD and controller spatial positions using a stereo camera that provides depth data.

6. A method for executing a game presented on a screen of a head mounted display (HMD), comprising,

executing a game, the execution rendering interactive scenes of the game on the screen;

receiving images of the HMD worn by a user to identify an HMD spatial position relative to a capture location that is directed toward the user;

receiving images of a controller held by the user to identify a controller spatial position relative to the capture location, the controller providing input to at least partially drive interaction with the game being executed; and

rendering a point of view into the interactive scenes of the game and sending to the screen of the HMD;

adjusting the point of view as the HMD spatial position is detected to change in the received images of the HMD and sending to the screen of the HMD.

7. The method of claim 6, wherein the controller is represented in the interactive scenes as a virtual object, the virtual object rendered in the interactive scenes in a virtual spatial position that maps to the controller spatial position, and changes in the controller spatial position change the virtual spatial position of the virtual object.

8. The method of claim 6, wherein the game is executed on a computer that is communicative connected to the head mounted display.

9. The method of claim 6, wherein the game is executed on a game cloud and the head mounted display is communicatively connected to the game cloud.

10. The method of claim 6, wherein each of the HMD spatial position and the controller spatial position is additionally identifiable by inertial data.

11. A method for executing a game presented on a screen of a head mounted display (HMD), the execution rendering interactive scenes of the game on the screen, comprising,

receiving data of the HMD worn by a user to identify an HMD spatial position;

receiving data of a controller held by the user to identify a controller spatial position;

sending data to the HMD for rendering a point of view into the interactive scenes of the game on the screen, the point of view being a volume of space rendered in three-dimensions; and

sending data to the HMD for adjusting the point of view as the HMD spatial position is detected to change, the method being executed by a processor.

12. The method of claim 11, wherein the controller provides input that drives interactivity.

13. The method of claim 12, wherein changes in the controller spatial position provide additional input.

14. The method of claim 13, wherein the additional input includes one of lights or inertial data.

15. The method of claim 14, wherein the lights provide location data when viewed by a camera and a color of the lights provide additional data.

16. The method of claim 15, wherein the color indicates player number or mode or setting based on room conditions.

17. The method of claim 11, wherein the controller is represented in the interactive scenes as a virtual object, the virtual object rendered in the interactive scenes in a virtual spatial position that substantially maps to the controller spatial position, and changes in the controller spatial position change the virtual spatial position of the virtual object as shown on the screen.