SYSTEMS AND METHODS FOR USING REDUCED HOPS TO GENERATE AN AUGMENTED VIRTUAL REALITY SCENE WITHIN A HEAD MOUNTED SYSTEM

• US 20190070498A1
• Filed: 11/05/2018
• Published: 03/07/2019
• Est. Priority Date: 06/07/2013
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment America LLC (Sony Group Corp.)
• Inventors: Perry, David
• Granted Patent: US 10,974,136 B2
• Time in Patent Office: Days
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/25 Output arrangements for video game devices
• CPC Class Codes: A63F 13/33 using wide area network [WAN] connections
• CPC Class Codes: A63F 13/355 Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
• CPC Class Codes: A63F 13/40 Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment
• CPC Class Codes: A63F 13/50 Controlling the output signals based on the game progress
• CPC Class Codes: A63F 13/52 involving aspects of the displayed game scene
• CPC Class Codes: A63F 13/71 using secure communication between game devices and game servers, e.g. by encrypting game data or authenticating players
• CPC Class Codes: A63F 13/90 Constructional details or arrangements of video game devices not provided for in groups A63F13/20 or A63F13/25, e.g. housing, wiring, connections or cabinets
• CPC Class Codes: A63F 13/98 Accessories, i.e. detachable arrangements optional for the use of the video game device, e.g. grip supports of game controllers
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/30 characterized by output arrangements for receiving control signals generated by the game device
• CPC Class Codes: A63F 2300/303 for displaying additional data, e.g. simulating a Head Up Display
• CPC Class Codes: A63F 2300/401 Secure communication, e.g. using encryption or authentication
• CPC Class Codes: A63F 2300/532 using secure communication, e.g. by encryption, authentication
• CPC Class Codes: A63F 2300/538 for performing operations on behalf of the game client, e.g. rendering
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 27/01 Head-up displays
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 : Head tracking input arrangements
• CPC Class Codes: G06F 3/013 : Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/014 : Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/0346 : with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/04815 : Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object

Abstract

Systems and methods for using reduced hops to generate an augmented virtual reality scene in a head mounted display. The head mounted display is used for game play via a game cloud system. The head mounted display includes a communications circuit for sending real-world media associated with a game program via a network. The real-world media is processed by the game cloud system and streamed directly from the communications circuit to the game cloud system. The head mounted display further includes a game processing circuit coupled to the communications circuit. The game processing circuit is used for decoding computer-generated interactive media received from the game cloud system via the network. The game processing circuit drives a portion of interactivity associated with the game program by superimposing the computer-generated interactive media on the real-world media. The computer-generated interactive media is generated based on the real-world media.

Claims

1. A video game controller comprising:

an input control configured to receive a selection from a user for play of a video game, the input control configured to generate input data for the selection; and

a communications circuit interfaced with the input control, wherein the communications circuit is configured to generate data packets having the input data, wherein the communications circuit is configured to communicate the data packets via a router and a computer network to a game cloud system without using a game console between the video game controller and the computer network, wherein the game cloud system is configured to process the input data to generate interaction control for a video output that is encoded to produce a plurality of compressed frames, wherein the plurality of compressed frames are configured to be streamed via the computer network to a display device for displaying the video output that is responsive to the input data.

2. The video game controller of claim 1, wherein the input control includes one or more buttons for selection by the user to generate the input data, wherein the input control includes a joystick to generate the input data, wherein the input data indicates that the selection of the input control is made.

3. The video game controller of claim 1, wherein the input control includes one or more inertial sensors, wherein the inertial sensors include a magnetometer, an accelerometer, and a gyroscope to detect a movement of the video game controller, wherein the input data indicates that the video game controller performed the movement.

4. The video game controller of claim 1, wherein the communications circuit is coupled to the input control to receive the input data from the input control.

5. The video game controller of claim 1, wherein the data packets follow a Transmission Control Protocol over an Internet Protocol for being communicated via the computer network to the game cloud system, wherein the computer network is the Internet.

6. The video game controller of claim 1, wherein the router is configured to receive the data packets from the communications circuit, buffer the data packets, and send the data packets via the computer network to the game cloud system.

7. The video game controller of claim 1, wherein the game console is configured to receive the input data from the video game controller, execute a game program based on the input data to output media data, and provide the media data to the display device for display of the video output.

8. The video game controller of claim 1, wherein the game cloud system is configured to process the input data to determine a state of the video game, wherein the state includes a position, a color, and a texture of a virtual game object within the video game, wherein the interaction control includes the state.

9. The video game controller of claim 1, wherein the video output includes a plurality of video frames, wherein the plurality of video frames are encoded according to an H.264 standard, wherein each of the plurality of compressed frames have a frame size and is a P frame, an I frame, or a B frame, wherein the stream of the compressed frames is configured to have a frame rate.

10. The video game controller of claim 1, wherein the display device includes a television screen or a computer screen or a head mounted display, wherein the head-mounted display is configured to be worn on a head of the user.

11. The video game controller of claim 1, wherein the router is coupled to the communications circuit via a wireless connection.

12. A system comprising:

a video game controller including;

an input control configured to receive a selection from a user for play of a video game, the input control configured to generate input data for the selection; and

a communications circuit interfaced with the input control, wherein the communications circuit is configured to generate data packets having the input data, wherein the communications circuit is configured to communicate the data packets via a router and a computer network to a game cloud system without using a game console between the video game controller and the computer network, wherein the game cloud system is configured to process the input data to generate interaction control for a video output that is encoded to produce a plurality of compressed frames; and

a display device coupled to the game cloud system via the computer network, wherein the display device is configured to receive the plurality of compressed frames and decode the plurality of compressed frames to display the video output.

13. The system of claim 12, wherein the input control includes one or more buttons for selection by the user to generate the input data, wherein the input control includes a joystick to generate the input data, wherein the input data indicates that the selection of the input control is made.

14. The system of claim 12, wherein the input control includes one or more inertial sensors, wherein the inertial sensors include a magnetometer, an accelerometer, and a gyroscope to detect a movement of the video game controller, wherein the input data indicates that the video game controller performed the movement.

15. The system of claim 12, wherein the communications circuit is coupled to the input control to receive the input data from the input control.

16. The system of claim 12, wherein the data packets follow a Transmission Control Protocol over an Internet Protocol for being communicated via the computer network to the game cloud system, wherein the computer network is the Internet.

17. The system of claim 12, wherein the router is configured to receive the data packets from the communications circuit, buffer the data packets, and send the data packets via the computer network to the game cloud system.

18. The system of claim 12, wherein the game console is configured to receive the input data from the video game controller, execute a game program based on the input data to output media data, and provide the media data to the display device for display of the video output.

19. The system of claim 12, wherein the game cloud system is configured to process the input data to determine a state of the video game, wherein the state includes a position, a color, and a texture of a virtual game object within the video game, wherein the interaction control includes the state.

20. The system of claim 12, wherein the video output includes a plurality of video frames, wherein the plurality of video frames are encoded according to an H.264 standard, wherein each of the plurality of compressed frames have a frame size and is a P frame, an I frame, or a B frame.

21. The system of claim 12, wherein the display device is a head mounted display, wherein the head mounted display includes a decoder configured to decode the plurality of compressed frames, wherein the head mounted display includes a camera, wherein the camera is configured to capture one or more images of a real-world environment surrounding the head mounted display for sending to the game cloud system via the router and the computer network without using the game console.