Classification and encoder selection based on content

• US 8,600,155 B2
• Filed: 01/30/2013
• Issued: 12/03/2013
• Est. Priority Date: 03/31/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Fagg, Graham Edward, Abdo, Nadim Y.
• Primary Examiner(s): COUSO, JOSE L
• Application Number:
  US13/753,876
• Publication Number:
  US 20130142427A1
• Time in Patent Office: 307 Days
• Field of Search: 382232-233, 382/236, 382239-240, 382/244, 345/619, 704/500, 704/504, 709203-204, 709/233, 709/246
• US Class Current: 382/165
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: H04N 19/12 Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
• CPC Class Codes: H04N 19/136 Incoming video signal characteristics or properties
• CPC Class Codes: H04N 19/164 Feedback from the receiver or from the transmission channel
• CPC Class Codes: H04N 19/42 characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation H04N19/635 takes precedence

Abstract

In various embodiments, methods and systems are disclosed for dynamic runtime implementation and end-to-end biased tuning of a two stage image classification system based on a decision function that uses network packet sizes and multiple image characteristics to determine the selection of an encoding codec to reduce overall network bandwidth consumption.

Claims

1. A computer-implemented method for classifying image data for transmission over a computer network, the method comprising:

receiving a plurality of images representative of a target system'"'"'s graphics usage;

testing the plurality of images using a plurality of encoding codecs;

encoding the plurality of images using the plurality of encoding codecs;

compressing the encoded plurality of images to generate encoded data for transmission over a network stream;

testing the compressed encoded plurality of images to determine a set of characteristic responses; and

selecting a subset of the characteristic responses for input to an image classifier.

2. The computer-implemented method of claim 1, wherein said image classifier is configured to classify image data based on the input.

3. The computer-implemented method of claim 2, wherein said image classifier is further configured to select one of a plurality of graphics encoders based on the classifying.

4. The computer-implemented method of claim 3, wherein said image classifier is further configured to select one of a plurality of graphics encoders based on link data indicative of network conditions.

5. The computer-implemented method of claim 4, wherein the image data is encoded using the selected encoder.

6. The computer-implemented method of claim 5, wherein the selected encoder substantially optimizes a compression ratio resulting from bulk compressing the encoded image data.

7. The computer-implemented method of claim 2, wherein the classifying comprises determining a rate of change of the image data.

8. The computer-implemented method of claim 7, wherein said determining comprises determining the rate of change of RGB values.

9. The computer-implemented method of claim 2, wherein said classifying comprises maintaining a color histogram.

10. The computer-implemented method of claim 7, wherein said determining comprises calculating differences by pixel rows and incrementing a counter for each R, G, or B value that is different than a previous pixel value.

11. The computer-implemented method of claim 10, further comprising summing individual RGB change counters.

12. The computer-implemented method of claim 9, wherein said maintaining comprises maintaining occurrences of color values in an array of colors.

13. The computer-implemented method of claim 12, further comprising determining a count of unique colors and counts for each of R, G, and B color.

14. The computer-implemented method of claim 13, further comprising using the counts for frequency analysis and color content comparison.

15. A system comprising:

at least one processor; and

at least one memory communicatively coupled to said at least one processor when the system is operational, the memory having stored therein computer-executable instructions that, upon execution by the processor, at least cause the system to;

receive image data representative of a target system'"'"'s graphics usage;

analyze the image data using a plurality of encoding codecs;

encode the image data using the plurality of encoding codecs;

generate encoded image data for transmission over a data network;

determine a set of characteristic responses for the encoded image data; and

send the characteristic responses to a computing device that at least implements an image classification subsystem.

16. The system of claim 15, wherein said image classification subsystem is operable to classify remote presentation data based on said characteristic responses and select one of a plurality of graphics encoders based on the classifying.

17. The system of claim 15, wherein said image classification subsystem is operable to select one of a plurality of graphics encoders based on link data indicative of network conditions.

18. A computer readable storage device storing thereon computer executable instructions for transmitting graphics data comprising:

instructions for testing a plurality of images using a plurality of encoding codecs;

instructions for encoding and compressing the plurality of images;

instructions for analyzing the compressed encoded plurality of images to determine a set of characteristic responses; and

instructions for selecting one or more of the characteristic responses for input to an image classifier.

19. The computer readable storage device of claim 18, wherein the characteristic responses are usable to identify one of the plurality of encoding codecs to substantially optimize a compression ratio resulting from bulk compressing the images encoded with the selected encoding codec.

20. The computer readable storage device of claim 18, wherein each of the plurality of encoders is substantially optimized for a type of image data.