Progressive block encoding using region analysis

• US 8,345,768 B1
• Filed: 10/13/2006
• Issued: 01/01/2013
• Est. Priority Date: 07/28/2005
• Status: Active Grant

• Current Assignee: Teradici Corporation (HP Inc.)
• Original Assignee: Teradici Corporation (HP Inc.)
• Inventors: Hobbs, David Victor, Ratto, Patrick
• Primary Examiner(s): Ustaris, Joseph
• Assistant Examiner(s): MCINNISH, KEVIN K
• Application Number:
  US11/549,577
• Time in Patent Office: 2,272 Days
• Field of Search: None
• US Class Current: 375/240.24
• CPC Class Codes: G06T 9/00 Image coding bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00
• CPC Class Codes: H04N 19/102 characterised by the element, parameter or selection affected or controlled by the adaptive coding
• CPC Class Codes: H04N 19/103 Selection of coding mode or of prediction mode
• CPC Class Codes: H04N 19/136 Incoming video signal characteristics or properties
• CPC Class Codes: H04N 19/154 Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion use of rate-distortion criteria H04N19/147
• CPC Class Codes: H04N 19/164 Feedback from the receiver or from the transmission channel
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/196 being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters processing of motion vectors H04N19/513
• CPC Class Codes: H04N 19/34 Scalability techniques involving progressive bit-plane based encoding of the enhancement layer, e.g. fine granular scalability [FGS]
• CPC Class Codes: H04N 19/507 using conditional replenishment

Abstract

The present invention provides a method of encoding and transmitting a block of visual data within a visual data stream. First, image information updates are acquired for the block of visual data that includes pixel data and is associated with a change history of pixel data. Next, stored block information is retrieved for the block of visual data, including a recent change history for the block of visual data. Next, a determination is made from the retrieved block information and the image information updates that the block of visual data is a video image type. Available network bandwidth is then determined. Encoding parameters for the block of visual data from the retrieved block information, the image information updates and the available network bandwidth are determined. The block of visual data is encoded and transmitted across a computer network.

Claims

1. A method of processing a sequence of images, the method comprising:

determining, by a computer comprising one or more processors, a plurality of pixel image blocks of a frame in the sequence of images are of a picture type, the frame in the sequence of images comprising a plurality of static image types determined using intra-frame decomposition analysis of the frame, the plurality of static image types including the picture type, and a text type;

determining change information for the plurality of pixel image block blocks, the change information comprising a pixel change detection of associated block locations of the plurality of pixel image block with respect to a plurality of frames of the sequence of images;

processing a first pixel image block of the plurality of pixel image blocks as a video type image based on the picture type and the change information indicating a periodic change rate slower than a frame rate of the sequence of images for the first pixel image block;

and processing a second pixel image block of the plurality of pixel image blocks as non-video type image based on a non-periodic change record of the second pixel image block, the processing of the second pixel image block comprising encoding the second pixel image block at an initial quality followed by a second encoding at an increased quality if the second pixel image block remains unchanged, wherein encoding the first pixel image block as the video type image comprises encoding the pixel image block once at a fixed quality while the first pixel image block remains unchanged.

2. The method of claim 1 wherein the change information comprises a record of changes for the block locations of the plurality of pixel image blocks in the sequence of images.

3. A method of encoding an image stream, the method comprising:

obtaining, by a computer comprising one or more processors, a plurality of pixels from a frame of the image stream, generated by the computer;

obtaining, by the computer, change information and an image type of each block of the plurality of pixels, the change information comprising a record of changes for each block of the plurality of pixels, the image type selected, based on intra-frame decomposition analysis of the frame, for each block of the plurality of pixels from a set comprising a picture type and a text type, wherein the change information comprises a periodic change history at a video frame rate for a first block position in the image stream and non-video frame rate change history for a second block position in the image stream; and

encoding, by the computer, based on the change information and the image type, the plurality of pixels, the encoding comprising selecting an encoding quality for each block of the picture type of the plurality of pixels based on a block associated multi change record from the record of changes, encoding pixels of the first block position at a fixed quality and encoding pixels of the second block position using progressive encoding.

4. The method of claim 3, wherein the video frame rate is at a lower frequency than an update rate of the image stream.

5. The method of claim 4 wherein encoding the pixels of the first block position at the fixed quality comprises encoding the pixels of the first block position only when at least one pixel of the pixels of the first block position changes.

6. The method of claim 3 wherein using progressive encoding comprises encoding successive quality improvements of a constant image at the second block position until a lossless encoding is encoded.

7. The method of claim 3 further comprising determining the pixels of the second block position comprise the non-video frame rate change history if the pixels of the second block position are unchanged for a duration exceeding a window of defined limits, determining the pixels of the first block position comprise the periodic change history at the video frame rate if the pixels of the first block position have changed and if a first duration since the pixels of the first block position previously changed is within the window of defined limits, and determining the pixels has of the second block position comprises the non-video frame rate change history if the pixels of the second block position have changed and if a second duration since the pixels of the second block position previously changed is outside the window of defined limits.

8. The method of claim 3 further comprising determining encoding parameters as optimized for reduced latency encoding if the change information comprises a change rate outside a video frame rate window.

9. The method of claim 3, wherein the image type comprises static image type information for the plurality of pixels determined by intra-frame filtering the plurality of pixels.

10. The method of claim 3, wherein the obtaining the image type comprises applying a decomposition filter to the plurality of pixels.

11. The method of claim 3, wherein the change information comprises an indication of duration since at least one pixel of each block of the plurality of pixels changed.

12. The method of claim 11, wherein the indication of duration indicates a frame count associated with the image stream.

13. The method of claim 3, further comprising determining encoding parameters by identifying a plurality of changes to the plurality of pixels, each duration between successive changes in the plurality of changes occurring within a window of defined limits.

14. A method of encoding and transmitting an image of a media stream, comprising:

acquiring image information for a frame of a pixel image block, the image information (i) indicating a picture image type from a selection comprising a non-picture image type based on an intra-frame decomposition analysis of the frame, and (ii) indicating that the pixel image block comprises a non-video image type change record, the non-video image type change record comprising one of successive frames of changed pixel values or a duration of constant pixel values for at least a defined period for the pixel image block;

determining, from a selection comprising an initial encoding, a progressive build encoding and a held quality encoding, based on the image information, an encoding operation to perform on the frame of the pixel image block, the encoding operation further selected based on a frame count of the media stream since the pixel image block changed; and

encoding the pixel image block using the determined encoding operation to generate an encoded image block, wherein the determining the encoding operation comprises selecting;

the initial encoding that encodes an initial quality set of bit planes of the pixel image block if the pixel image block has changed since a previous encoding of the pixel image block, or

the progressive build encoding that encodes an incremental quality set of the bit planes of the pixel image block for a defined duration if the pixel image block is unchanged.

15. The method of claim 14, wherein the determining the encoding operation comprises selecting the encoding operation that communicates an indication of a gap for the encoded image block if the pixel image block is unchanged and if the pixel image block has been encoded to a defined quality level.

16. The method of claim 15, wherein the defined quality level is determined by an encoded frame count of unchanged pixel values of the pixel image block in successive frames of the media stream.

17. The method of claim 14, further comprising transmitting the encoded image block, wherein transmitting the encoded image block comprises transmitting a delay indication, the delay indication signaling a delay in display of an image resulting from the encoded image block.

18. A method of encoding an image block of a plurality of image blocks of an image, the method comprising:

obtaining information for the image block, the information comprising (i) pixel values, (ii) change information of the pixel values as related to previous encoded frames of the image and (iii) a level of previous encoded bit planes of the pixel values, wherein the change information indicating the pixel values are unchanged for at least the defined period indicates that the image block is changed from a video image type to a non-video type and wherein the encoding requirement is further determined according to the non-video type;

determining an encoding requirement responsive to the change information indicating the pixel values and encoded data of the pixel values are unchanged for at least a defined period, the defined period comprising a plurality of the previous encoded frames where other image blocks of the plurality of image blocks have been encoded; and

encoding the pixel values according to the encoding requirement to generate an additional encoded bit plane to the previous encoded data bit planes of the pixel values, wherein the encoding the pixel values comprises performing repeated encodings of at least one of a progressive encoding operation or a residual encoding operation until a lossless encoding of the pixel values has been performed.

19. The method of claim 18 wherein the change information comprises a temporal change pattern or interval.

20. The method of claim 18, wherein the information for the image block further comprises a present encoded quality level and wherein the encoding is further encoded based on the present encoded quality level.

21. The method of claim 20, wherein the present encoded quality level comprises an indication of a communication of an initial encoding at a defined quality level.

22. The method of claim 18, wherein the progressive encoding operation is suitable for encoding an unchanged picture type image block by encoding one additional bit plane of the image block on each sequential encoding.

23. A method of encoding an image stream, comprising:

detecting a non-video type pixel block and a video type pixel block in a frame of the image stream, the video type pixel block comprising a periodic history of frames with at least one pixel change separated by one or more frames of constant pixel values, the non-video type pixel block comprising a non-periodic history of pixel changes;

encoding the non-video type pixel block using progressive build encoding, the progressive build encoding comprising an initial first quality encoding on a frame with a pixel change to the non-video type pixel block and an accumulating sequence of increasing quality refinement encodings for each frame while the non-video type pixel block remains unchanged, wherein the accumulating sequence of increasing quality terminates on achieving a lossless encoding of the non-video type pixel block;

and encoding the video type pixel block using a constant quality level encoding comprising an initial second quality encoding on a frame with a pixel change to the video type pixel block and skipping encoding the video type pixel block on each frame while the video type pixel block remains unchanged.

24. The method of claim 23 wherein a first quality level and a second quality level are determined by an available network bandwidth.

25. The method of claim 24 wherein the initial first quality encoding and the encoded non video type pixel block accumulating sequence of increasing quality refinement encodings are allocated different portions of the available network bandwidth.

26. The method of claim 23 further comprising:

detecting re-classified type block comprising the video type pixel block remaining unchanged for a defined number of frames and encoding the re-classified type block using the accumulating sequence of increasing quality refinement encodings.

27. The method of claim 23 further comprising encoding block type information associated with the encoded non-video type pixel block and the encoded video type pixel block.