Virtual frame buffer system and method

• US 8,824,560 B2
• Filed: 07/06/2011
• Issued: 09/02/2014
• Est. Priority Date: 07/07/2010
• Status: Active Grant

• Current Assignee: Netzyn, Inc.
• Original Assignee: Netzyn, Inc.
• Inventors: Bakke, Steve
• Primary Examiner(s): Kelley, Christopher S
• Assistant Examiner(s): Zhou, Zhihan
• Application Number:
  US13/176,843
• Publication Number:
  US 20120033738A1
• Time in Patent Office: 1,154 Days
• Field of Search: 375/240.24, 375/240.01, 375/240.02, 375/240.16, 375/240.2, 375/240.23, 375/240.25, 375/240.26, 348/545, 348/546, 348/547
• US Class Current: 375/240.24
• CPC Class Codes: G06F 3/1415 with means for detecting differences between the image stored in the host and the images displayed on the displays
• CPC Class Codes: G09G 2310/04 Partial updating of the display screen
• CPC Class Codes: G09G 2320/103 Detection of image changes, e.g. determination of an index representative of the image change
• CPC Class Codes: G09G 2350/00 Solving problems of bandwidth in display systems
• CPC Class Codes: G09G 2360/18 Use of a frame buffer in a display terminal, inclusive of the display panel
• CPC Class Codes: G09G 5/395 Arrangements specially adapted for transferring the contents of the bit-mapped memory to the screen G09G5/399 takes precedence
• CPC Class Codes: G09G 5/397 Arrangements specially adapted for transferring the contents of two or more bit-mapped memories to the screen simultaneously, e.g. for mixing or overlay G09G5/02 takes precedence
• CPC Class Codes: G09G 5/399 using two or more bit-mapped memories, the operations of which are switched in time, e.g. ping-pong buffers
• CPC Class Codes: H04N 19/423 characterised by memory arrangements H04N19/433 takes precedence
• CPC Class Codes: H04N 19/433 characterised by techniques for memory access
• CPC Class Codes: H04N 19/50 using predictive coding H04N19/61 takes precedence
• CPC Class Codes: H04N 19/507 using conditional replenishment
• CPC Class Codes: H04N 19/513 Processing of motion vectors

Abstract

A method encodes or decodes a frame (also file), such as a video, graphic, media, or other frame or data, representing a real-time graphic output from a frame buffer, output by a video camera, or another file or data. The file includes frames each comprising macroblocks. Reference frame buffers (PFTs), virtual frame buffer tables (VFTBs) of equal number to the PFTs, each VFTB corresponds to a respective PFT, and respective sectors of each PFT for respective macroblocks are created. Frames of the file are encoded/decoded by successive encode/decode of macroblocks. A pointer is created in the VFBT associated with the PFT rather than encoding/decoding any matching macroblock. The pointer and its reference are relied on for each already encoded/decoded macroblock retained in the PFT. Processing, memory, bandwidth and power requirements for encoding or decoding are reduced.

Claims

1. A device for encoding a file, the file comprises frames and each frame comprises macroblocks for encoding, comprising:

memory;

an encoder communicatively connected to the memory, for processing macroblocks, in succession, of each frame, in succession, of the file;

a plurality of reference frame buffers created in the memory by the encoder, wherein each reference frame buffer is distinct from any other of the plurality of reference frame buffers in the memory, each reference frame buffer, respectively, for containing the macroblocks, encoded by the encoder, of one of the frame in respective sectors of the reference frame buffer;

a plurality of virtual frame buffer tables, equal in number to number of the reference frame buffers in the memory, created in the memory by the encoder, each virtual frame buffer table, respectively, corresponds to distinct one of the reference frame buffers, respectively; and

a respective pointer created by the encoder during processing of the macroblocks of one of the frame, and saved by the encoder in the virtual frame buffer table corresponding to the reference frame buffer then-being used by the encoder for processing the one of the frame, if the macroblock then-being processed by the encoder would, when encoded, be the same as any macroblock previously encoded by the encoder and then-being retained in any of the reference frame buffers;

wherein the encoder either encodes each next one of the macroblock of the frame or, if the next one of the macroblock, when encoded, would be the same as another macroblock, previously encoded, and then-retained in any of the plurality of reference frame buffers, instead creates the respective pointer and stores the respective pointer in the virtual frame buffer table corresponding to the reference frame buffer frame then-being used by the encoder for processing the frame;

wherein each of the virtual frame buffer tables is a respective single bit array in the memory and each single bit of the array is positioned in the array to indirectly reference a respective location of the macroblock, previously encoded, then-retained in the one of the reference frame buffers referenced by the respective pointer.

3. The system of claim 1, wherein the encoder obtains an encoded file representing the file, the encoded file comprises encoded frames and each frame comprises encoded macroblocks, further comprising:

a communication link communicatively connected to the encoder, selected from the group of;

wire, wireless link, cellular link, optic link, satellite, public switched telephone network, Internet, local area network, wireless local area network, or other communication network, or any combination of any or some of these;

decode memory;

a decoder communicatively connected to the communication link and to the decode memory for processing encoded macroblocks, in succession, of each encoded frame, in succession, of the encoded file;

a plurality of decode reference frame buffers created in the decode memory by the decoder, wherein each decode reference frame buffer is distinct from any other of the plurality of decode reference frame buffers in the decode memory, each decode reference frame buffer, respectively, for containing the encoded macroblocks of the encoded frame in respective sectors of the decode reference frame buffer;

a plurality of decode virtual frame buffer tables, equal in number to number of the decode reference frame buffers in the decode memory, created in the decode memory by the decoder, each decode virtual frame buffer table, respectively, corresponds distinctly to one of the decode reference frame buffers, respectively; and

a respective decode pointer created by the decoder during decoding of the encoded macroblocks of one of the encoded frame, and saved by the decoder in the decode virtual frame buffer table corresponding to the decode reference frame buffer then-being used by the encoder for processing the one of the encoded frame, if the encoded macroblock then-being processed by the decoder would, when decoded, be the same as any encoded macroblock previously decoded by the decoder and then being retained in any of the decode reference frame buffers;

wherein the decoder either decodes each next one of the encoded macroblock of the encoded frame or, if the next one of the encoded macroblock, when decoded, would be the same as another encoded macroblock, previously decoded, and then-retained in any of the plurality of decode reference frame buffers, instead creates the respective decode pointer and stores the respective decode pointer in the decode virtual frame buffer table corresponding to the decode reference frame buffer then being used by the decoder for decoding the encoded frame.

4. The system of claim 3, wherein at least certain of the (i) memory and the decode memory and (ii) the encoder and the decoder, are respectively unitized.

7. The device of claim 1,

wherein the file is a graphical type screen (such as a word processor or spreadsheet) of size about 5,000 to about 10,000 macroblocks;

and

wherein only about one to about ten macroblocks need be updated in a changed frame, wherein the processing, memory requirements, bandwidth and power are thereby reduced in encoding or decoding, respectively.

12. The device of claim 1, wherein the device is selected from the group consisting of:

circuit, computer program stored in a non-transitory computer readable storage medium, processor, and any combination of these.

2. A device for decoding an encoded file, the encoded file comprises encoded frames and each frame comprises encoded macroblocks for decoding, comprising:

memory;

a decoder communicatively connected to the memory for processing encoded macroblocks, in succession, of each encoded frame, in succession, of the encoded file;

a plurality of reference frame buffers created in the memory by the decoder, wherein each reference frame buffer is distinct from any other of the plurality of reference frame buffers in the memory, each reference frame buffer, respectively, for containing the encoded macroblocks of the encoded frame in respective sectors of the reference frame buffer;

a plurality of virtual frame buffer tables, equal in number to number of the reference frame buffers in the memory, created in the memory by the decoder, each virtual frame buffer table, respectively, corresponds distinctly to one of the reference frame buffers, respectively; and

a respective pointer created by the decoder during decoding of the encoded macroblocks of one of the encoded frame, and saved by the decoder in the virtual frame buffer table corresponding to the reference frame buffer then-being used by the encoder for processing the one of the encoded frame, if the encoded macroblock then-being processed by the decoder would, when decoded, be the same as any encoded macroblock previously decoded by the decoder and then being retained in any of the reference frame buffers;

wherein the decoder either decodes each next one of the encoded macroblock of the encoded frame or, if the next one of the encoded macroblock, when decoded, would be the same as another encoded macroblock, previously decoded, and then-retained in any of the plurality of reference frame buffers, instead creates the respective pointer and stores the respective pointer in the virtual frame buffer table corresponding to the reference frame buffer then being used by the decoder for decoding the encoded frame;

wherein each of the virtual frame buffer tables is a respective single bit array in the memory and each single bit of the array is positioned in the array to indirectly reference a respective location of the encoded macroblock, previously decoded, then-retained in the one of the reference frame buffers referenced by the respective pointer.

13. The device of claim 2, wherein the device is selected from the group consisting of:

circuit, computer program stored in a non-transitory computer readable storage medium, processor, and any combination of these.

14. The device of claim 2,

wherein the file is a graphical type screen (such as a word processor or spreadsheet) of size about 5,000 to about 10,000 macroblocks;

and

wherein only about one to about ten macroblocks need be updated in a changed frame, wherein the processing, memory requirements, bandwidth and power are thereby reduced in encoding or decoding, respectively.

5. A method of encoding a file, the file comprises frames and each frame comprises macroblocks for encoding, comprising the steps of:

creating in a memory a plurality of reference frame buffers, wherein each reference frame buffer is distinct from any other of the plurality of reference frame buffers in the memory, each reference frame buffer, respectively, for containing the macroblocks, when encoded, of one of the frame in respective sectors of the reference frame buffer;

creating in the memory a plurality of virtual frame buffer tables, equal in number to number of the reference frame buffers in the memory, each virtual frame buffer table, respectively, corresponds distinctly to one of the reference frame buffers, respectively;

encoding the frames of the file, in succession, by successively encoding each respective one of the macroblocks of the frame, unless any already encoded macroblock (if any) then-retained in any of the respective sectors of any of the reference frame buffers would be the same as the encoded macroblock, once encoded in the step of encoding;

storing respective ones of the encoded macroblocks, each respectively corresponding to respective ones of the macroblocks of the frame of the file from the step of encoding, in respective ones of the plurality of respective sectors, except not storing one of the encoded macroblocks if any already encoded macroblock (if any) then-retained in any of the respective sectors of any of the reference frame buffers would be the same as the encoded macroblock, once encoded in the step of encoding;

creating a respective pointer in the virtual frame buffer table associated with the reference frame buffer then available for the step of encoding, for each macroblock of the frame then-currently for the step of encoding, wherein (i) if any macroblock of one of the frame previously encoded matches the macroblock of the step of creating a respective pointer and if the encoded macroblock of the frame previously encoded is then-retained in any of the reference frame buffers, the respective pointer references the reference frame buffer containing the encoded macroblock of the frame previously encoded, and (ii) otherwise, the respective pointer references the reference frame buffer corresponding to the frame then-currently for the step of encoding;

storing the respective pointer from the step of creating the respective pointer (if performed), in the virtual frame buffer table associated with the reference frame buffer for the step of encoding, positioned in the virtual frame buffer table as follows;

(i) if any macroblock of one of the frame previously encoded matches the macroblock of the step of creating a respective pointer and is then-retained in any of the reference frame buffers, positioned to indirectly reference the respective sector holding the encoded macroblock of the frame previously encoded, and (ii) otherwise, positioned to indirectly reference the respective sector of the reference frame buffer for the step of encoding in which the macroblock, encoded in the step of encoding, is retained; and

returning to the step of encoding;

wherein the encoder during encoding either encodes each next one of the macroblock of the frame or, if any macroblock of another one of the frames previously encoded is then-retained in any of the plurality of reference frame buffers, creates the respective pointer and stores the respective pointer in the virtual frame buffer table corresponding to the frame then being encoded.

8. The method of claim 5,

wherein the file is a graphical type screen (such as a word processor or spreadsheet) of size about 5,000 to about 10,000 macroblocks;

and

wherein only about one to about ten macroblocks need be encoded in the step of encoding for any frame of macroblocks not previously encoded, wherein the processing, memory requirements, bandwidth and power are thereby reduced in encoding.

10. An encode device operating in accordance with claim 5.

6. A method of decoding an encoded file, the encoded file comprises encoded frames and each encoded frame comprises encoded macroblocks for decoding and the file, when decoded, comprises frames and each frame comprises macroblocks, comprising the steps of:

creating in a memory a plurality of reference frame buffers, wherein each reference frame buffer is distinct from any other of the plurality of reference frame buffers in the memory, each reference frame buffer, respectively, for containing the encoded macroblocks, when decoded, of one of the encoded frame in respective sectors of the reference frame buffer;

creating in the memory a plurality of virtual frame buffer tables, equal in number to number of the reference frame buffers in the memory, each virtual frame buffer table, respectively, corresponds distinctly to one of the reference frame buffer, respectively;

decoding the encoded frames of the encoded file, in succession, by successively decoding each respective one of the encoded macroblocks of the encoded frame, unless the encoded macroblock, once decoded, would match any macroblock previously decoded and then-retained in any of the respective sectors of any of the reference frame buffers;

storing respective ones of the macroblocks, each respectively corresponding to respective ones of the macroblocks of the frame of the file from the step of decoding, in respective ones of the plurality of respective sectors, except not storing one of the macroblocks if the macroblock, once decoded, would match any macroblock previously decoded and then-retained in any of the respective sectors of any of the reference frame buffers;

creating a respective pointer in the virtual frame buffer table associated with the reference frame buffer then available for the step of decoding, for each encoded macroblock of the encoded frame then-currently for the step of decoding, wherein (i) if the encoded macroblock, once decoded, would match any macroblock previously decoded and retained in any of the reference frame buffers, the respective pointer references the reference frame buffer containing the encoded macroblock, once decoded, of the frame previously decoded, and (ii) otherwise, the respective pointer references the reference frame buffer corresponding to the frame then-currently for the step of decoding;

storing the respective pointer from the step of creating the respective pointer (if performed), in the virtual frame buffer table associated with the reference frame buffer for the step of decoding, positioned in the virtual frame buffer table as follows;

(i) if any encoded macroblock, once decoded, of one of the frame previously decoded matches the encoded macroblock to then be decoded in the step of decoding and is then-retained in any of the reference frame buffers, positioned to indirectly reference the respective sector holding the encoded macroblock, once decoded, of the frame previously decoded, and (ii) otherwise, positioned to indirectly reference the respective sector of the reference frame buffer for the step of decoding in which the encoded macroblock, decoded in the step of decoding, is retained; and

returning to the step of decoding;

wherein the decoder during decoding either decodes each next one of the encoded macroblock of the encoded frame or, if any encoded macroblock of another one of the encoded frames previously decoded is then-retained in any of the plurality of reference frame buffers, creates the respective pointer and stores the respective pointer in the virtual frame buffer table corresponding to the encoded frame then being decoded.

9. The method of claim 6,

wherein the encoded file represents a graphical type screen (such as a word processor or spreadsheet) of size about 5,000 to about 10,000 macroblocks;

and

wherein only about one to about ten encoded macroblocks need be decoded in the step of decoding for any encoded frame of encoded macroblocks not previously decoded of another encoded frame, wherein the processing, memory requirements, bandwidth and power are thereby reduced in decoding.

11. A decode device operating in accordance with claim 6.