Dynamically adaptive multimedia application program interface and related methods

• US 7,668,242 B2
• Filed: 10/04/2004
• Issued: 02/23/2010
• Est. Priority Date: 04/21/2000
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Fogg, Chad E., Sullivan, Gary J.
• Primary Examiner(s): CZEKAJ, DAVID J
• Application Number:
  US10/958,120
• Publication Number:
  US 20050041743A1
• Time in Patent Office: 1,968 Days
• Field of Search: 37524001- 7, 37524012- 16, 37524025- 26, 375/240.29, 348/384.1, 348/387.1, 348/390.1
• US Class Current: 375/240.25
• CPC Class Codes: H04N 19/126 Details of normalisation or weighting functions, e.g. normalisation matrices or variable uniform quantisers
• CPC Class Codes: H04N 19/156 Availability of hardware or computational resources, e.g. encoding based on power-saving criteria
• CPC Class Codes: H04N 19/184 the unit being bits, e.g. of the compressed video stream
• CPC Class Codes: H04N 19/37 with arrangements for assigning different transmission priorities to video input data or to video coded data
• CPC Class Codes: H04N 19/42 characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation H04N19/635 takes precedence
• CPC Class Codes: H04N 19/577 Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/86 involving reduction of coding artifacts, e.g. of blockiness
• CPC Class Codes: H04N 19/90 using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
• CPC Class Codes: H04N 21/25808 Management of client data terminal profiles in network data switching protocols H04L67/303
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/4143 embedded in a Personal Computer [PC]
• CPC Class Codes: H04N 21/42653 for processing graphics
• CPC Class Codes: H04N 21/4431 characterized by the use of Application Program Interface [API] libraries
• CPC Class Codes: H04N 21/4516 involving client characteristics, e.g. Set-Top-Box type, software version or amount of memory available
• CPC Class Codes: H04N 21/454 Content or additional data filtering, e.g. blocking advertisements

Abstract

A method of interfacing non-integrated media processing system elements is presented, the method comprising identifying one or more characteristics of one or more media processing system elements, and dynamically negotiating which system elements will perform certain media processing tasks based, at least in part, on the identified one or more characteristics of the system elements.

Claims

1. A storage medium comprising a plurality of executable instructions which, when executed on a computing system, implement a method comprising:

identifying one or more characteristics of one or more elements of a media processing system, wherein identifying the one or more characteristics of the one or more elements comprises;

generating a first negotiation data structure in accordance with a proposed set of media processing capabilities;

issuing the first negotiation data structure to one or more elements of a media processing system;

determining whether each of the one or more media processing system elements accepted the first negotiation data structure;

executing media processing tasks within the proposed media processing capabilities if the first negotiation data structure is accepted by the elements of the media processing system;

generating one or more auto-negotiation data structure(s) denoting a proposed set of media processing system capabilities and/or a proposed split in media processing among media system elements; and

iteratively issuing the auto-negotiation data structure(s) to media processing system elements until a proposed set of media processing system capabilities and/or split in media processing is agreed upon by each element of the media processing system;

dynamically negotiating which element of a media processing system will perform certain media processing tasks based, at least in part, on the identified one or more characteristics of the system element(s);

if the media processing system elements did not previously accept the first negotiation data structure, generating a second negotiation data structure that is different from the first negotiation data structure in accordance with another proposed set of media processing capabilities; and

iteratively performing the issuing, determining and generating until the media processing system elements accept a negotiation data structure.

2. A method according to claim 1, wherein the method is implemented by an application program interface (API) of an operating system executing on a general purpose computing system hosting the media processing system.

3. A method according to claim 1, further comprising:

generating operational data structure(s) to pass between the one or more media processing system elements to facilitate shared decoding of multimedia content between media processing system elements.

4. A method according to claim 3, wherein the data structures include at least, one or more residual difference data structures and one or more macroblock control command data structures which have fixed sizes, determined on a per-frame basis.

5. A method according to claim 3, wherein the operational data structures comprise a raw bitstream data structure, dynamically generated to pass media content from a decoder application to a hardware accelerator to facilitate decoding by the accelerator.

6. A method according to claim 1, wherein the data structure(s) include one or more auto-negotiation data structure(s).

7. A method according to claim 6, wherein the auto-negotiation data structure(s) include a connection mode data structure denoting a proposed set of required processing system capabilities that indicate capabilities needed for decoding data in a format specified by a particular media processing standard.

8. A method according to claim 6, wherein the auto-negotiation data structure(s) include a connection configuration data structure denoting a proposed split in media processing between media processing system elements.

9. A computing system comprising:

a storage medium having stored therein a plurality of executable instructions; and

an execution unit, coupled to the storage medium, to execute at least a subset of the plurality of executable instructions to implement a method according to claim 1.

10. A computing system comprising:

a media processing application;

a media processing accelerator; and

an operating system, executing on the computing system, including an application program interface (API) to facilitate communication between the media processing application and the media processing accelerator, wherein the API includes a first set of auto-negotiation data structure(s) and operational data structure(s) to dynamically negotiate at least a first set of media processing system capabilities and/or a first split in media processing among system elements suitable to each of the media processing application and the media processing accelerator and to processing of received media content, respectively, wherein if the at least a first set of media processing system capabilities and/or the first split in media processing among system elements are not accepted by each of the system elements, the API generates a second set of auto-negotiation data structure(s) and operational data structure(s) that are different from the first set of auto-negotiation data structure(s) to negotiate at least a second set of media processing system capabilities and/or a second split in media processing among system elements suitable to each of the media processing application and the media processing accelerator and to processing of received media content, respectively, said API iteratively performing the generating until each of the system elements accepts a set of auto-negotiation data structure(s) and operational data structure(s), wherein the operational data structure(s) comprise;

one or more residual difference data structures, generated by the API to pass residual difference information between the media processing application and the media processing accelerator for media processing; and

one or more control command data structures, generated by the API to pass control commands between the media processing application and the media processing accelerator.

11. A computing system according to claim 10, the auto-negotiation data structures comprising:

a processing mode data structure, generated by the API to propose a media set of media processing capabilities and/or a split in media processing among system elements to each of the media processing application and the media processing accelerator.

12. A computing system according to claim 11, wherein the processing mode data structure is a ConnectMode data structure.

13. A computing system according to claim 10, the auto-negotiation data structures comprising:

a connection mode data structure, specifying a set of media processing system capabilities, and/or

a processing configuration data structure, generated by the API to propose a split in media processing between the media processing application and the media processing accelerator.

14. A computing system according to claim 13, wherein the processing configuration data structure comprises a ConnectConfig data structure.

15. A computing system according to claim 10, the operational data structure(s) comprising:

a raw bitstream data structure, generated by the API to pass media content in raw bitstream form from the media processing application to the media processing accelerator.

16. A computing system according to claim 10, further comprising:

a storage medium including a plurality of executable instructions; and

an execution unit, coupled to the storage medium, to execute at least a subset of the plurality of executable instructions to implement the operating system and associated API.

17. A computing system according to claim 16, wherein the execution unit executes at least a subset of the plurality of executable instructions to implement the media processing application.

18. A storage medium comprising a plurality of executable instructions including instructions which, when executed on a computing system, implement an application program interface (API) to facilitate media processing between media processing system elements, the API including one or more auto-negotiation data structure(s) dynamically generated and issued to the media processing system elements to negotiate a set of media processing capabilities and/or a split in media processing tasks between the elements of the media processing system suitable to each of the media processing system elements, wherein the auto-negotiation data structures include a processing mode data structure, iteratively issued to the media processing system elements denoting an iterative changing of the proposed set of media processing capabilities until accepted by all media processing system elements where each iterative change is different than the previous iterative change, and wherein the API further comprises:

one or more operational data structures, to facilitate media processing between media processing system elements based, at least in part, on a negotiated set of media processing capabilities and/or split in media processing responsibility between the media processing system elements wherein the operational data structure(s) comprise;

one or more residual difference data structures, generated by the API to pass residual difference information between a media processing application and a media processing accelerator for media processing; and

one or more control command data structures, generated by the API to pass control commands between the media processing application and the media processing accelerator.

19. A storage medium according to claim 18, wherein the auto-negotiation data structures include a processing configuration data structure, iteratively issued to the media processing system elements denoting an iteratively changing proposed split in media processing between media processing system elements.

20. A computing system comprising:

means for identifying one or more characteristics of one or more elements of a media processing system, wherein said means for identifying the one or more characteristics of the system one or more elements comprises;

means for generating a first negotiation data structure in accordance with a proposed set of media processing capabilities;

means for issuing the first negotiation data structure to one or more elements of a media processing system;

means for determining whether each of the one or more media processing system elements accepted the first negotiation data structure;

means for executing media processing tasks within the proposed media processing capabilities if the first negotiation data structure is accepted by the elements of the media processing system;

means for generating one or more auto-negotiation data structure(s) denoting a proposed set of media processing system capabilities and/or a proposed split in media processing among media system elements; and

means for iteratively issuing the auto-negotiation data structure(s) to media processing system elements until a proposed set of media processing system capabilities and/or split in media processing is agreed upon by each element of the media processing system;

means for dynamically negotiating which element of a media processing system will perform certain media processing tasks based, at least in part, on the identified one or more characteristics of the system element(s);

means for generating a second negotiation data structure that is different from the first negotiation data structure in accordance with another proposed set of media processing capabilities if the media processing system elements did not previously accept the negotiation data structure; and

means for iteratively performing the issuing, determining and generating until the media processing system elements accept a negotiation data structure.

21. A computing system according to claim 20, wherein said means for identifying and means for dynamically negotiating comprise an application program interface (API) of an operating system executing on a general purpose computing system hosting the media processing system.

22. A computing system according to claim 20, further comprising:

means for generating operational data structure(s) to pass between the one or more media processing system elements to facilitate shared decoding of multimedia content between media processing system elements.

23. A computing system according to claim 22, wherein the data structures include at least, one or more residual difference data structures and one or more macroblock control command data structures which have fixed sizes, determined on a per-frame basis.

24. A computing system according to claim 22, wherein the operational data structures comprise a raw bitstream data structure, dynamically generated to pass media content from a decoder application to a hardware accelerator to facilitate decoding by the accelerator.

25. A computing system according to claim 20, wherein the data structure(s) include one or more auto-negotiation data structure(s).

26. A computing system according to claim 25, wherein the auto-negotiation data structure(s) include a connection mode data structure denoting a proposed set of required processing system capabilities that indicate capabilities needed for decoding data in a format specified by a particular media processing standard.

27. A computing system according to claim 25, wherein the auto-negotiation data structure(s) include a connection configuration data structure denoting a proposed split in media processing between media processing system elements.

28. One or more computer-readable media comprising executable instructions which, when executed by a computing system, cause the one or more computer-readable media to implement a method comprising:

identifying one or more characteristics of one or more elements of a media processing system, wherein identifying the one or more characteristics of the system elements comprises;

generating a first negotiation data structure in accordance with a first proposed set of media processing capabilities;

issuing the first negotiation data structure to one or more elements of a media processing system;

determining whether each of the one or more media processing system elements accepted the first negotiation data structure; and

executing media processing tasks within the first set of proposed media processing capabilities if the first negotiation data structure is accepted by the elements of the media processing system; and

generating one or more auto-negotiation data structure(s) denoting a proposed set of media processing system capabilities and/or a proposed split in media processing among media system elements; and

iteratively issuing the auto-negotiation data structure(s) to media processing system elements until a proposed set of media processing system capabilities and/or split in media processing is agreed upon by each element of the media processing system;

dynamically negotiating which element of a media processing system will perform certain media processing tasks based, at least in part, on the identified one or more characteristics of the system element(s);

if the media processing system elements did not previously accept the first negotiation data structure, generating a second negotiation data structure that is different from the first negotiation data structure in accordance with a second proposed set of media processing capabilities; and

iteratively performing the issuing, determining and generating until the media processing system elements accept a negotiation data structure.

29. The one or more computer-readable media according to claim 28, wherein the instructions further cause the one or more processors to:

generate operational data structure(s) to pass between the one or more media processing system elements to facilitate shared decoding of multimedia content between media processing system elements.

30. The one or more computer-readable media according to claim 29, wherein the data structures include at least, one or more residual difference data structures and one or more macroblock control command data structures which have fixed sizes, determined on a per-frame basis.

31. The one or more computer-readable media according to claim 29, wherein the operational data structures comprise a raw bitstream data structure, dynamically generated to pass media content from a decoder application to a hardware accelerator to facilitate decoding by the accelerator.

32. The one or more computer-readable media according to claim 28, wherein the data structure(s) include one or more auto-negotiation data structure(s).

33. The one or more computer-readable media according to claim 32, wherein the auto-negotiation data structure(s) include a connection mode data structure denoting a proposed set of required processing system capabilities that indicate capabilities needed for decoding data in a format specified by a particular media processing standard.

34. The one or more computer-readable media according to claim 32, wherein the auto-negotiation data structure(s) include a connection configuration data structure denoting a proposed split in media processing between media processing system elements.