Strategies for processing image information using a color information data structure

• US 20050024384A1
• Filed: 07/29/2004
• Published: 02/03/2005
• Est. Priority Date: 08/01/2003
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Estrop, Stephen J., Evans, Glenn F.
• Granted Patent: US 7,643,675 B2
• Time in Patent Office: Days
• US Class Current: 345/604
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G09G 2310/0229 De-interlacing
• CPC Class Codes: G09G 2340/0435 Change or adaptation of the frame rate of the video stream
• CPC Class Codes: G09G 2360/125 using unified memory architecture [UMA]
• CPC Class Codes: G09G 5/363 Graphics controllers
• CPC Class Codes: H04N 23/83 specially adapted for colour signals
• CPC Class Codes: H04N 23/85 for matrixing
• CPC Class Codes: H04N 7/012 Conversion between an interlaced and a progressive signal for graphics images G09G2310/0229
• CPC Class Codes: H04N 7/0122 the input and the output signals having different aspect ratios
• CPC Class Codes: H04N 9/642 Multi-standard receivers
• CPC Class Codes: H04N 9/67 for matrixing camera processing pipelines for matrixing of colour signals H04N23/85
• CPC Class Codes: H04N 9/69 for modifying the colour signals by gamma correction controlling camera response for colour signals H04N23/83

Abstract

Strategies are described for processing image information using a data structure which conveys color information. The color information describes color-related formatting applied to the image information. The data structure can be passed down a video processing pipeline, where each functional component in the pipeline can draw color information from the data structure to improve the accuracy of its processing. Further, each component can supply previously unknown color information to the data structure for use by other (downstream) components in the pipeline. Exemplary fields of the data structure can include one or more of: a video transfer function field; a color primaries field; an image light field; a transfer matrix field; a nominal range field; and a video chroma sampling field. The transfer matrix field can be used to convert image information from one color space to another, such as a luma-related color space to an RGB-related color space. Processing operations can be performed on progressive and linear RGB image information.

Claims

1. A method for processing image information having color content expressed in a prescribed format, comprising:

providing a color information data structure that contains color information that defines at least one aspect of the prescribed format;

passing the color information data structure to at least one functional component used to process the image information, along with the image information; and

processing the image information at said at least one functional component based on the color information in the color information data structure.

2. The method according to claim 1, wherein said at least one functional component belongs to a group of functional components defining a video processing pipeline, wherein the color information data structure is passed down the video processing pipeline for use by downstream functional components.

3. The method according to claim 2, wherein the video processing pipeline performs the operations of:

converting input image information in a luma-related color space into linear image information within an RGB-related color space; and

performing processing on the linear image information in the RGB-related color space to produce processed information.

4. The method according to claim 3, wherein the converting also involves transforming the input image information from an interlaced format into a progressive format prior to the processing operation.

5. The method according to claim 3, wherein the video processing pipeline further performs the operations of:

converting the processed information into nonlinear image information within a luma-related color space.

6. The method according to claim 5, wherein the converting of the processed information also involves transforming the processed information from a progressive format into an interlaced format.

7. The method according to claim 2, wherein the video processing pipeline performs color conversion, and wherein the color information in the color information data structure governs the color conversion.

8. The apparatus according to claim 1, further comprising initializing unknown values within the color information data structure to a prescribed default value.

9. The method according to claim 1, wherein said at least one functional component further determines at least one other aspect of the prescribed format and adds said at least one other aspect to at least one unknown value in the color information data structure.

10. The method according to claim 1, wherein the passing of the color information data structure comprises passing the color information data structure in an unused field of an existing data structure.

11. The method according to claim 1, further comprising setting a flag which indicates that said at least one functional component has the capacity to process the color information data structure.

12. The method according to claim 1, wherein the color information data structure comprises a video transfer function field which conveys information regarding the transfer function used to convert linear image information into nonlinear image information, and/or vice versa.

13. The method according to claim 1, wherein the color information data structure comprises a color primaries field which conveys color primaries associated with the image information.

14. The method according to claim 1, wherein the color information data structure comprises an image light field which conveys an intended viewing condition associated with the image information.

15. The method according to claim 1, wherein the color information data structure comprises a transfer matrix field which conveys information regarding a transfer matrix that can be used to modify the color space of the image information.

16. The method according to claim 1, wherein the color information data structure comprises a nominal range field which conveys information regarding potential toe room and head room associated with the image information.

17. The method according to claim 1, wherein the color information data structure comprises a video chroma sampling field which indicates the manner in which chroma samples in the image information are sampled and aligned relative to associated luma samples.

18. The method according to claim 1, wherein the color information data structure comprises:

a video transfer function field which conveys information regarding the transfer function used to convert linear image information into nonlinear image information, and/or vice versa;

a color primaries field which conveys color primaries associated with the image information;

an image light field which conveys an intended viewing condition associated with the image information;

a transfer matrix field that can be used to modify the color space of the image information;

a nominal range field which conveys information regarding potential toe room and head room associated with the image information; and

a video chroma sampling field which indicates the manner in which chroma samples in the image information are sampled and aligned relative to associated luma samples.

19. One or more processor-readable media including, stored thereon, processor-executable instructions for implementing the method of claim 1.

20. An apparatus including logic configured to perform each of the providing, passing and processing of claim 1.

21. An apparatus including logic configured to perform each of the providing, passing and processing using the color information data structure of claim 18.

22. A data structure stored on one or more processor-readable media, the data structure including color information that describes the format of associated image information, the data structure including at least one of:

a video transfer function field which conveys information regarding the transfer function used to convert linear image information into nonlinear image information, and/or vice versa;

a color primaries field which conveys color primaries associated with the image information;

an image light field which conveys an intended viewing condition associated with the image information;

a transfer matrix field that can be used to modify the color space of the image information;

a nominal range field which conveys information regarding potential toe room and head room associated with the image information;

or

a video chroma sampling field which indicates the manner in which chroma samples in the image information are sampled and aligned relative to associated luma samples.

23. The data structure of claim 22, including each of the identified fields.

24. A system for processing image information having color content expressed in a prescribed format, comprising:

plural functional components defining a video processing pipeline,

wherein each of the plural functional components includes logic configured to processing a color information data structure that contains color information that defines at least one aspect of the prescribed format.

25. The system according to claim 24, wherein the color information data structure comprises at least one of:

a video transfer function field which conveys information regarding the transfer function used to convert linear image information into nonlinear image information, and/or vice versa;

a color primaries field which conveys color primaries associated with the image information;

an image light field which conveys an intended viewing condition associated with the image information;

a transfer matrix field that can be used to modify the color space of the image information;

a nominal range field which conveys information regarding potential toe room and head room associated with the image information;

or

a video chroma sampling field which indicates the manner in which chroma samples in the image information are sampled and aligned relative to associated luma samples.

26. A method for processing image information within a data processing module that includes plural units for storing respective image information, comprising:

assigning at least one of the plural units to a principal video stream of image information;

assigning at least one other of the plural units to a video sub-stream of image information; and

performing at least two tasks involving the principal video stream and the video sub-stream in a single operation to produce composite video information.

27. The method according to claim 26, wherein the plural units comprise plural graphics texture units.

28. One or more processor-readable media including, stored thereon, processor-executable instructions for implementing the method of claim 26.

29. An apparatus including logic configured to perform the assigning and performing of claim 26.

30. A method for processing image information, comprising:

providing a principal video stream of image information;

providing a video sub-stream of image information; and

performing, based on color information specified in a color information data structure, at least two tasks involving the principal video stream and the video sub-stream to produce output video information expressed in a specified color space.

31. The method according to claim 30, wherein the specified color space is an RGB-related color space.

32. The method according to claim 31, wherein the RGB-related color space defines nonlinear RGB data.

33. The method according to claim 31, wherein the RGB-related color space defines linear RGB data.

34. The method according to claim 30, wherein the specified color space is a luma-related color space.

35. The method according to claim 30, wherein the specified color space defines a common color space for processing plural streams of image information.

36. The method according to claim 30, wherein the color information identifies the specified color space.

37. The method according to claim 30, wherein the performing operates on plural streams of image information, each having its own respective color information data structure associated therewith.

38. The method according to claim 30, wherein said at least two tasks include deinterlacing the principal video stream, and wherein the color information data structure includes information which governs the deinterlacing.

39. The method according to claim 30, wherein said at least two tasks include converting image information from a first color space to a second color space.

40. The method according to claim 39, wherein the first color space is a luma-related color space and the second color space is an RGB-related color space.

41. The method according to claim 39, wherein the color information in the color information data structure governs the converting.

42. The method according to claim 30, wherein said at least two tasks include at least two of:

(a) deinterlacing the principal video stream;

(b) combining the principal video stream with the video sub-stream;

or

(c) converting image information from a first color space to a second color space.

43. The method according to claim 30, wherein said at least two tasks are performed in a single operation.

44. One or more processor-readable media including, stored thereon, processor-executable instructions for implementing the method of claim 30.

45. An apparatus including logic configured to perform the providing of the principal video stream, providing of the sub-stream, and performing of claim 30.

46. A method for processing image information, comprising:

converting input image information in a luma-related color space into linear image information within an RGB-related color space;

performing processing on the linear image information in the RGB-related color space to produce processed information; and

converting the processed information into nonlinear image information within a luma-related color space.

47. The method according to claim 46, wherein the converting also involves transforming the input image information from an interlaced format into a progressive format prior to the processing operation, and, after processing, transforming the processed information from a progressive format into an interlaced format.

48. The method according to claim 46, further comprising supplying a color information data structure, and using color information specified in the color information data structure to govern at least one of the converting of the input information, the processing, or the converting of the processed information.

49. One or more processor-readable media including, stored thereon, processor-executable instructions for implementing the method of claim 46.

50. An apparatus including logic configured to perform the converting of input information, the performing, and the converting of processed information of claim 46.