Differentiation for achieving buffered decoding and bufferless decoding

• US 9,723,319 B1
• Filed: 05/31/2010
• Issued: 08/01/2017
• Est. Priority Date: 06/01/2009
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment America LLC (Sony Group Corp.)
• Inventors: Pereira, Rui Filipe Andrade, Gault, Andrew Buchanan
• Primary Examiner(s): Senfi, Behrooz
• Assistant Examiner(s): Colon, Maria Vazquez
• Application Number:
  US12/790,948
• Time in Patent Office: 2,619 Days
• CPC Class Codes: A63F 13/00 Video games, i.e. games using an electronically generated display having two or more dimensions gaming systems which provide a financial reward G07F17/32
• CPC Class Codes: A63F 13/10 Control of the course of the game, e.g. start, progess, end
• CPC Class Codes: A63F 13/12 involving interaction between a plurality of game devices, e.g. transmisison or distribution systems protocols for networked virtual reality, networked games H04L29/06034
• CPC Class Codes: A63F 13/30 Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
• CPC Class Codes: A63F 13/335 using Internet
• CPC Class Codes: A63F 13/35 Details of game servers
• CPC Class Codes: A63F 13/355 Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/45 Controlling the progress of the video game
• CPC Class Codes: A63F 13/537 using indicators, e.g. showing the condition of a game character on screen
• CPC Class Codes: A63F 13/71 using secure communication between game devices and game servers, e.g. by encrypting game data or authenticating players
• CPC Class Codes: A63F 13/77 involving data related to game devices or game servers, e.g. configuration data, software version or amount of memory
• CPC Class Codes: A63F 13/795 for finding other players; for building a team; for providing a buddy list
• CPC Class Codes: A63F 13/86 Watching games played by other players
• CPC Class Codes: A63F 13/87 Communicating with other players during game play, e.g. by e-mail or chat
• CPC Class Codes: A63F 2300/407 Data transfer via internet
• CPC Class Codes: A63F 2300/409 Data transfer via television network
• CPC Class Codes: A63F 2300/51 Server architecture
• CPC Class Codes: A63F 2300/538 for performing operations on behalf of the game client, e.g. rendering
• CPC Class Codes: A63F 2300/552 for downloading to client devices, e.g. using OS version, hardware or software profile of the client device
• CPC Class Codes: G06F 2009/45579 : I/O management, e.g. providing access to device drivers or storage
• CPC Class Codes: G06F 2009/45595 : Network integration; Enabling network access in virtual machine instances
• CPC Class Codes: G06F 9/45537 : Provision of facilities of other operating environments, e.g. WINE I/O emulation G06F13/105
• CPC Class Codes: G06F 9/45558 : Hypervisor-specific management and integration aspects
• CPC Class Codes: H04L 65/1101 : Session protocols
• CPC Class Codes: H04L 65/61 : for supporting one-way streaming services, e.g. Internet radio
• CPC Class Codes: H04L 65/70 : Media network packetisation
• CPC Class Codes: H04L 65/80 : Responding to QoS
• CPC Class Codes: H04L 67/10 : in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/131 : Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04L 67/52 : specially adapted for the location of the user terminal
• CPC Class Codes: H04N 19/136 : Incoming video signal characteristics or properties
• CPC Class Codes: H04N 19/42 : characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation H04N19/635 takes precedence
• CPC Class Codes: H04N 19/44 : Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 19/46 : Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/70 : characterised by syntax aspects related to video coding, e.g. related to compression standards

Abstract

Systems and methods of managing H.264 compliant video that lacks B Frames include decoding without the use of a significant video frame buffer. This variant of the H.264 standard may include a flag indicating that the video does not include B Frames. The video may be used in applications, such as computer games, in which processing of B Frames introduces undesirable lag.

Claims

1. A video system comprising:

a video decoding microprocessor configured to generate a video output to be displayed on a display device, the video decoding microprocessor comprising;

a standard H.264 image generation logic circuit including a buffer,

a variant H.264 image generation logic circuit lacking a buffer; and

a type identification logic circuit in communication with the standard H.264 image generation logic circuit and the variant H.264 image generation logic circuit, wherein the type identification logic is configured to receive a video and configured to differentiate between a standard H.264 video and a variant H.264 video based on an inclusion and an exclusion of B frames in the received video, the type identification logic circuit further configured to direct the standard H.264 video to the standard H.264 image generation logic circuit and the variant H.264 video to the variant H.264 image generation logic circuit based on the differentiation,

wherein the standard H.264 image generation logic circuit is configured to decode the standard H.264 video to generate a portion of the video output,

wherein the variant H.264 image generation logic circuit is configured to decode the variant H.264 video without buffering any frames of the variant H.264 video during the decode of the variant H.264 video, wherein the variant H.264 image generation logic is configured to decode the variant H.264 video to generate a portion of the video output; and

a memory device coupled to the video decoding microprocessor and configured to store video decoding logic for generating the video output.

2. The video system of claim 1, further comprising user input controls and the display device.

3. The video system of claim 1, wherein the standard H.264 image generation logic circuit and the variant H.264 image generation logic circuit share computing instructions.

4. The video system of claim 1, wherein the video decoding logic is configured to execute within a browser.

5. The video system of claim 1, wherein the type identification logic circuit is configured to differentiate between the standard H.264 video and the variant H.264 video by examining a codec index of the received video.

6. The video system of claim 1, wherein the type identification logic circuit is configured to differentiate between the standard H.264 video and the variant H.264 video by detecting a flag within the received video.

10. The video system of claim 1, wherein the variant H.264 image generation logic circuit is configured to generate the portion of the video output using the variant H.264 video at an Extended Profile, or a Main Profile, or a High Profile, or a High 10 Profile, or a High 4:

2;

2 Profile, or a High 4;

4;

4 Predictive Profile.

11. The video system of claim 1, wherein the display device is configured to display the portion of the video output generated by and received from the variant H.264 image generation logic circuit.

12. The video system of claim 1, wherein the variant H.264 image generation logic circuit is configured to decode the variant H.264 video in a time period between 25 milliseconds and 175 milliseconds after being received by a client.

7. A non-transitory computer readable medium having stored thereon computing instructions, the computing instructions comprising:

logic configured to receive encoded video;

standard H.264 image generation logic for using a buffer during execution of the standard H.264 image generation logic;

variant H.264 image generation logic; and

type identification logic configured to differentiate between a standard H.264 video and a variant H.264 video based on an inclusion and exclusion of B frames in the received encoded video and to direct the standard H.264 video to the standard H.264 image generation logic and the variant H.264 video to the variant H.264 image generation logic based on the differentiation;

wherein the standard H.264 image generation logic is configured to decode the standard H.264 video to generate a decoded video output, the decoded video output generated by the standard H.264 image generation logic being configured for display on a computing device;

wherein the variant H.264 image generation logic is configured to decode the variant H.264 video without using a buffer to buffer any frames of the variant H.264 video during the decode of the variant H.264 video, wherein the variant H.264 image generation logic is configured to decode the H.264 video to generate a decoded video output, the decoded video output generated by the variant H.264 image generation logic being configured for display on a computing device.

8. The non-transitory computer readable medium of claim 7, wherein the variant H.264 image generation logic is configured to generate the decoded video output at an Extended Profile, or a Main Profile, or a High Profile, or a High 10 Profile, or a High 4:

2;

2 Profile, or a High 4;

4;

4 Predictive Profile.

9. The non-transitory computer readable medium of claim 7, wherein the variant H.264 generation logic is configured to generate the decoded video output less than 150 milliseconds after the variant H.264 video is received at a client.

13. A method comprising:

receiving a video, the video including a plurality of standard H.264 video frames and a plurality of variant H.264 video frames;

differentiating between the standard H.264 video frames and the variant H.264 video frames based on an inclusion and an exclusion of B frames in the received video;

directing the standard H.264 video frames to a standard H.264 image generation logic circuit;

directing the variant H.264 video frames to a variant H.264 image generation logic circuit based on said differentiating;

decoding the standard H.264 video frames to generate a portion of a video output; and

decoding the variant H.264 video frames without buffering any of the variant H.264 video frames during the decoding of the variant H.264 video frames, wherein said decoding the variant H.264 video frames is performed to generate a portion of the video output.

14. The method of claim 13, wherein said differentiating between the standard H.264 video frames and the variant H.264 video frames is performed by examining a codec index of the received video.

15. The method of claim 13, wherein said differentiating between the standard H.264 video frames and the variant H.264 video frames is performed detecting a flag within the received video.

16. The method of claim 13, wherein said decoding the variant H.264 frames is performed using an Extended Profile, or a Main Profile, or a High Profile, or a High 10 Profile, or a High 4:

2;

2 Profile, or a High 4;

4;

4 Predictive Profile.

17. The method of claim 13, wherein said decoding the variant H.264 video frames is performed in a time period ranging from 25 milliseconds to 175 milliseconds.