Compressing image segmentation data using video coding

• US 10,015,504 B2
• Filed: 07/27/2016
• Issued: 07/03/2018
• Est. Priority Date: 07/27/2016
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Wei, Jian, Chen, Ying, Wang, Lei
• Primary Examiner(s): Osinski, Michael
• Application Number:
  US15/220,856
• Publication Number:
  US 20180035120A1
• Time in Patent Office: 706 Days
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: H04N 19/10 using adaptive coding
• CPC Class Codes: H04N 19/12 Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
• CPC Class Codes: H04N 19/44 Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder

Abstract

In an example of this disclosure, a method may include receiving or generating image segmentation data for a plurality of pixels. The image segmentation data may include one or more image segmentation values for each pixel of the plurality of pixels. The method may include converting the one or more image segmentation values into one or more image component values. The method may include sending the one or more image component values corresponding to the one or more image segmentation values to a video encoder.

Claims

1. A method comprising:

receiving or generating, by a processing unit implemented in circuitry, image segmentation data for a plurality of pixels, wherein the image segmentation data includes a plurality of Gaussian Mixture Model (GMM) parameter values for one or more GMM models for each pixel of the plurality of pixels, and wherein the plurality of GMM parameter values includes a mean value, a variance value, and a weight value for each pixel of the plurality of pixels;

converting, by the processing unit, the plurality of GMM parameter values into a plurality of image component values, wherein converting the plurality of GMM parameter values comprises converting the mean value of the plurality of GMM parameter values into a first image component value of the plurality of image component values using one or more look up tables (LUTs), converting the variance value of the plurality of GMM parameter values into a second image component value of the plurality of image component values using the one or more LUTs, and converting the weight value of the plurality of GMM parameter values into a third image component value of the plurality of image component values using the one or more LUTs; and

encoding, by a video encoder, the plurality of image component values.

2. The method of claim 1, further comprising:

encoding, by the video encoder, the plurality of image component values according to a video coding standard or a video coding specification.

3. The method of claim 2, wherein the video coding standard includes at least one of H.263, H.264, H.265, or a video coding standard different from H.263, H.264, and H.265.

4. The method of claim 1, further comprising:

receiving the image segmentation data from a memory accessible by the processing unit, wherein the memory is external to the processing unit or is not external to the processing unit.

5. The method of claim 1, further comprising:

storing the encoded plurality of image component values in a memory accessible by the processing unit.

6. The method of claim 5, wherein the memory is external to the processing unit.

7. The method of claim 1, wherein the plurality of image component values correspond to one or more images, and wherein the plurality of pixels corresponds to an image that is different from the one or more images.

8. The method of claim 7, further comprising:

encoding, by the video encoder, the plurality of image component values by encoding the one or more images, wherein the video encoder is a component of the processing unit or the video encoder is not a component of the processing unit.

9. The method of claim 1, wherein each pixel of the plurality of pixels is represented by one or more image component values, wherein the one or more image component values corresponding to each pixel include at least one of a red color component value, a blue color component value, a green color component value, a luma component value, or a chroma component value, and wherein the plurality of image component values corresponding to the plurality of GMM parameter values include at least one of a red color component value, a blue color component value, a green color component value, a luma component value, or a chroma component value.

10. The method of claim 1, wherein the plurality of pixels includes all pixels of an image or less than all pixels of the image.

11. A device comprising:

a first memory;

a processing unit implemented in circuitry, the processing unit being configured to;

receive or generate image segmentation data for a plurality of pixels, wherein the image segmentation data includes a plurality of Gaussian Mixture Model (GMM) parameter values for one or more GMM models for each pixel of the plurality of pixels, and wherein the plurality of GMM parameter values includes a mean value, a variance value, and a weight value for each pixel of the plurality of pixels;

convert the plurality of GMM parameter values into a plurality of image component values, wherein, to convert the plurality of GMM parameter values, the processing unit is configured to convert the mean value of the plurality of GMM parameter values into a first image component value of the plurality of image component values using one or more look up tables (LUTs), convert the variance value of the plurality of GMM parameter values into a second image component value of the plurality of image component values using the one or more LUTs, and convert the weight value of the plurality of GMM parameter values into a third image component value of the plurality of image component values using the one or more LUTs; and

store the plurality of image component values corresponding to the plurality of GMM parameter values in the first memory; and

a video encoder configured to encode the plurality of image component values.

12. The device of claim 11, further comprising a second memory, wherein the processing unit is configured to receive the image segmentation data for the plurality of pixels from the first memory or the second memory, wherein the first memory is external to the processing unit or is not external to the processing unit.

13. The device of claim 12, further comprising a second memory, wherein at least one of the video encoder or the processing unit is configured to store the encoded plurality of image component values in the first memory or the second memory.

14. The device of claim 11, wherein the plurality of image component values correspond to one or more images, and wherein the plurality of pixels corresponds to an image that is different from the one or more images.

15. The device of claim 14, wherein the video encoder is configured to encode the plurality of image component values by being configured to encode the one or more images, wherein the video encoder is a component of the processing unit or the video encoder is not a component of the processing unit.

16. The device of claim 11, wherein each pixel of the plurality of pixels is represented by one or more image component values, wherein the one or more image component values corresponding to each pixel include at least one of a red color component value, a blue color component value, a green color component value, a luma component value, or a chroma component value, and wherein the plurality of image component values corresponding to the plurality of GMM parameter values include at least one of a red color component value, a blue color component value, a green color component value, a luma component value, or a chroma component value.

17. The device of claim 11, wherein the plurality of pixels includes all pixels of an image or less than all pixels of the image.

18. An apparatus comprising:

means for receiving or generating image segmentation data for a plurality of pixels, wherein the image segmentation data includes a plurality of Gaussian Mixture Model (GMM) parameter values for one or more GMM models for each pixel of the plurality of pixels, and wherein the plurality of GMM parameter values includes a mean value, a variance value, and a weight value for each pixel of the plurality of pixels;

means for converting the plurality of GMM parameter values into a plurality of image component values, wherein the means for converting the plurality of GMM parameter values comprises means for converting the mean value of the plurality of GMM parameter values into a first image component value of the plurality of image component values using one or more look up tables (LUTs), means for converting the variance value of the plurality of GMM parameter values into a second image component value of the plurality of image component values using the one or more LUTs, and means for converting the weight value of the plurality of GMM parameter values into a third image component value of the plurality of image component values using the one or more LUTs; and

means for encoding the plurality of image component values.

19. The apparatus of claim 18, further comprising:

means for storing the encoded plurality of image component values corresponding to the plurality of GMM parameter values.

20. The apparatus of claim 18, wherein the plurality of image component values correspond to one or more images, and wherein the plurality of pixels corresponds to an image that is different from the one or more images.

21. The apparatus of claim 20, wherein the means for encoding the plurality of image component values includes means for encoding the one or more images.

22. The apparatus of claim 18, wherein each pixel of the plurality of pixels is represented by one or more image component values, wherein the one or more image component values corresponding to each pixel include at least one of a red color component value, a blue color component value, a green color component value, a luma component value, or a chroma component value, and wherein the plurality of image component values corresponding to the plurality of GMM parameter values include at least one of a red color component value, a blue color component value, a green color component value, a luma component value, or a chroma component value.

23. The apparatus of claim 18, wherein the plurality of pixels includes all pixels of an image or less than all pixels of the image.

24. A method comprising:

receiving, by a video decoder, a plurality of encoded image component values;

decoding, by the video decoder, the plurality of encoded image component values to generate a plurality of decoded image component values; and

converting, by a processing unit implemented in circuitry, the plurality of decoded image component values into a plurality of Gaussian Mixture Model (GMM) parameter values for one or more GMM models for each pixel of the plurality of pixels, wherein converting the plurality of decoded image component values comprises converting a first image component value of the plurality of image comonent values into a mean value of the plurality of GMM parameter values using one or more look up tables (LUTs), converting a second image component value of the plurality of image component values into a variance value of the plurality of GMM parameter values using the one or more LUTs, and converting a third image component value of the plurality of image component values into a weight value of the plurality of GMM parameter values using the one or more LUTs, and wherein the video decoder is a component of the processing unit or the video decoder is not a component of the processing unit.