Progressive block encoding using region analysis

• US 8,107,527 B1
• Filed: 09/29/2006
• Issued: 01/31/2012
• Est. Priority Date: 07/28/2005
• Status: Active Grant

• Current Assignee: Teradici Corporation (HP Inc.)
• Original Assignee: Teradici Corporation (HP Inc.)
• Inventors: Hobbs, David Victor, Ratto, Patrick
• Primary Examiner(s): Patel, Jayanti K
• Assistant Examiner(s): Pe, Geepy
• Application Number:
  US11/537,545
• Time in Patent Office: 1,950 Days
• Field of Search: 375/240.01, 375/240.02, 375/240.2, 375/240.21
• US Class Current: 375/240.01
• CPC Class Codes: G06T 9/00 Image coding bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/136 Incoming video signal characteristics or properties
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/164 Feedback from the receiver or from the transmission channel
• CPC Class Codes: H04N 19/174 the region being a slice, e.g. a line of blocks or a group of blocks
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/18 the unit being a set of transform coefficients
• CPC Class Codes: H04N 19/184 the unit being bits, e.g. of the compressed video stream
• CPC Class Codes: H04N 19/1883 the unit relating to sub-band structure, e.g. hierarchical level, directional tree, e.g. low-high [LH], high-low [HL], high-high [HH]
• CPC Class Codes: H04N 19/192 the adaptation method, adaptation tool or adaptation type being iterative or recursive
• CPC Class Codes: H04N 19/34 Scalability techniques involving progressive bit-plane based encoding of the enhancement layer, e.g. fine granular scalability [FGS]
• CPC Class Codes: H04N 19/507 using conditional replenishment

Abstract

The present invention provides a method of progressively encoding and transmitting visual data. First image information updates are acquired for a region of visual data. The region of visual data is defined by a plurality of blocks of pixel data, and individual blocks of the plurality are associated with individual change histories for their respective pixel data. Stored region information including a recent encoded state for each block in the plurality is retrieved. Available network bandwidth is determined, and then encoding parameters for each block in the region of visual data are determined from the retrieved region information and the available network bandwidth. The plurality of blocks is then encoded, the retrieved region information is updated in accordance with the determined encoding parameters; and the encoded blocks are transmitted across a computer network.

Claims

1. A method of progressively encoding and transmitting visual data, comprising:

acquiring image information updates for a region of visual data, the region of visual data defined by a plurality of blocks of pixel data, wherein individual blocks of the plurality are associated with individual change histories for their respective pixel data;

retrieving stored region information for the region of visual data, the retrieved information including a recent encoded state for each block in the plurality;

determining available network bandwidth;

determining encoding parameters for each block in the region of visual data from the retrieved region information and the available network bandwidth;

encoding the plurality of blocks;

updating the retrieved region information in accordance with the determined encoding parameters; and

transmitting the encoded blocks across a computer network, wherein determining encoding parameters comprises determining an initial encoding quality level for pixel values that have changed, and wherein determining encoding parameters comprises determining an incremental encoding quality level for blocks that have been previously encoded, have not reached a lossless encoded state and have unchanged pixel values.

2. The method of claim 1, wherein the stored region information comprises an image history for the region.

3. The method of claim 2, wherein the image history comprises an initialized encoding state for at least one block in the plurality.

4. The method of claim 2, wherein the image history comprises a history of encoding at least one block in the plurality since pixel data in the at least one block was changed.

5. The method of claim 2, wherein the image history comprises an indication of a number of frames that have been acquired since the most recent encoding.

6. The method of claim 2, wherein the image history comprises a record of the time since the most recent encoding.

7. The method of claim 2, wherein the image history comprises information describing image types contained within blocks in the plurality.

8. The method of claim 7, wherein the image types comprise selections from the group consisting of text, picture, background, video and object.

9. The method of claim 1, wherein the recent encoded state comprises the most recent progressive encoded state for the block.

10. The method of claim 1, wherein acquiring image information update comprises acquiring pixel updates for a block.

11. The method of claim 1, wherein acquiring image information update comprises acquiring image type information for a block.

12. The method of claim 11, wherein the image types comprise selections from the group consisting of text, picture, background, video and object.

13. The method of claim 1, wherein determining encoding parameters comprises determining block encoding requirements.

14. The method of claim 13, wherein the block encoding requirements comprises an incremental progressive encoding step.

15. The method of claim 13, wherein the block encoding requirements comprises an initial encoding quality level.

16. The method of claim 13, wherein the block encoding requirements comprises determining how a block is to be encoded.

17. The method of claim 16, wherein the block is encoded with at least one encoding method.

18. The method of claim 1, further comprising storing the updated region information.

19. The method of claim 18, wherein storing the updated region information comprises storing updated image content.

20. The method of claim 18, wherein storing the updated region information comprises storing a pixel mask for a block, the pixel mask indicating pixel data in the block that has been encoded.

21. The method of claim 1, wherein determining encoding parameters comprises generating a summary table.

22. The method of claim 21, wherein the summary table includes a sum of the number of blocks at each possible encoding state.

23. The method of claim 1, wherein acquiring image information comprises acquiring attributes, the attributes describing properties of the region, by applying a decomposition filter to the acquired image information.

24. The method of claim 1, wherein acquiring image information comprises acquiring a present encoding quality level for blocks in the plurality.

25. The method of claim 1, wherein acquiring image information comprises acquiring indicia that a block is being progressively encoded but pixel data within the progressively encoded block have not reached a lossless state.

26. The method of claim 1, wherein acquiring image information comprises identifying pixel data as being related to a video sequence, wherein encoding the plurality of blocks comprises performing video encoding on blocks in the plurality.

27. The method of claim 1, further comprising encoding all changed blocks within the region at a substantially similar encoding quality level.

28. The method of claim 1, wherein determining the incremental encoding quality level for each previously encoded block is determined such that all blocks complete their encoding at the substantially similar encoding quality level.

29. The method of claim 1, wherein determining the incremental encoding quality level for each previously encoded block is determined such that all blocks increment their encoding quality level in a perceptually similar encoding quality step size.

30. The method of claim 1, wherein determining the incremental encoding quality level for each previously encoded block is determined such that all blocks in the plurality are incremented to an encoding quality level that is inversely proportional to their respective current encoding quality level.

31. The method of claim 1, wherein determining the encoding quality level comprises accounting for a duration of time for which a block has remained associated with an encoding quality level.

32. The method of claim 1, wherein determining the encoding quality level comprises accounting for the time since the pixel data values have changed.

33. The method of claim 1, wherein determining the encoding quality level comprises determining a maximum encoding quality increment that can be achieved within the available network bandwidth.

34. Apparatus for progressively encoding and transmitting visual data, comprising:

means for acquiring image information updates for a region of visual data, the region of visual data defined by a plurality of blocks of pixel data, wherein individual blocks of the plurality are associated with individual change histories for their respective pixel data;

means for retrieving stored region information for the region of visual data, the retrieved information including a recent encoded state for each block in the plurality;

means for determining available network bandwidth;

means for determining encoding parameters for each block in the region of visual data from the retrieved region information and the available network bandwidth;

means for encoding the plurality of blocks;

means for updating the retrieved region information in accordance with the determined encoding parameters; and

means for transmitting the encoded blocks across a computer network, wherein determining encoding parameters comprises determining an initial encoding quality level for pixel values that have changed, and wherein determining encoding parameters comprises determining an incremental encoding quality level for blocks that have been previously encoded, have not reached a lossless encoded state and have unchanged pixel values.

35. A non-transitory computer readable medium encoded with a computer program which, when executed by a computer, progressively encodes and transmits visual data, by:

acquiring image information updates for a region of visual data, the region of visual data defined by a plurality of blocks of pixel data, wherein individual blocks of the plurality are associated with individual change histories for their respective pixel data;

retrieving stored region information for the region of visual data, the retrieved information including a recent encoded state for each block in the plurality;

determining available network bandwidth;

determining encoding parameters for each block in the region of visual data from the retrieved region information and the available network bandwidth;

encoding the plurality of blocks;

updating the retrieved region information in accordance with the determined encoding parameters; and

transmitting the encoded blocks across a computer network, wherein determining encoding parameters comprises determining an initial encoding quality level for pixel values that have changed, and wherein determining encoding parameters comprises determining an incremental encoding quality level for blocks that have been previously encoded, have not reached a lossless encoded state and have unchanged pixel values.