Frame coding for spatial audio data

• US 10,535,355 B2
• Filed: 05/31/2017
• Issued: 01/14/2020
• Est. Priority Date: 11/18/2016
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: McDowell, Brian C., Edry, Philip Andrew, Ibrahim, Ziyad, Heitkamp, Robert Norman, Wilssens, Steven
• Primary Examiner(s): Mooney, James K
• Application Number:
  US15/609,418
• Publication Number:
  US 20180144752A1
• Time in Patent Office: 958 Days
• Field of Search: None
• CPC Class Codes: G10L 19/008 Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
• CPC Class Codes: G10L 19/167 Audio streaming, i.e. formatting and decoding of an encoded audio signal representation into a data stream for transmission or storage purposes
• CPC Class Codes: G10L 19/173 Transcoding, i.e. converting between two coded representations avoiding cascaded coding-decoding
• CPC Class Codes: H04S 2400/11 Positioning of individual sound objects, e.g. moving airplane, within a sound field H04S2420/13 takes precedence
• CPC Class Codes: H04S 3/008 in which the audio signals are in digital form, i.e. employing more than two discrete digital channels data reduction aspects thereof based on psychoacoustics G10L19/02
• CPC Class Codes: H04S 7/30 Control circuits for electronic adaptation of the sound field
• CPC Class Codes: H04S 7/301 Automatic calibration of stereophonic sound system, e.g. with test microphone

Abstract

The techniques disclosed herein provide apparatuses and related methods for the communication of spatial audio and related metadata. In some implementations, a source provides prerecorded spatial audio that has embedded metadata. A computing device processes the prerecorded spatial audio to generate an audio codec that is segmented to include a first section of audio data and a second section that includes metadata extracted from the prerecorded spatial audio. The generated audio codec may be received by a device that includes an encoder. The encoder may process the generated audio codec to generate audio data that includes the metadata.

Claims

1. A computing device, comprising:

a processor;

a computer-readable storage medium in communication with the processor, the computer-readable storage medium having computer-executable instructions stored thereupon which, when executed by the processor, cause the processor to;

receive a codec frame having a predetermined length and comprising first and second separated sections, the first section including at least a portion of audio data from a prerecorded spatial audio stream and a second section including at least one metadata component extracted from the audio data;

extract the at least one metadata component from the second section;

associate the at least one metadata component at an offset position between a beginning of the at least a portion of audio data comprised in the first section and an end of the at least the portion of the audio data comprised in the first section to provide an audio data frame having the at least one metadata component embedded therein at the offset position;

generate an audio stream comprising at least the audio data frame; and

communicate the audio stream to one or more audio rendering elements to playback the at least a portion of the audio data.

2. The computing device according to claim 1, wherein the second section includes a plurality of metadata components extracted from the audio data, each of the plurality of metadata components disposed in a segmented section of the second section.

3. The computing device according to claim 2, wherein the plurality of associated metadata components comprises positional metadata including one or more coordinates to render the at least a portion of the audio data in a three-dimensional space, a gain of the at least a portion of audio data, and calibration information for the one or more audio rendering elements to playback the at least a portion of the audio data.

4. The computing device according to claim 1, wherein the audio data is pulse code modulation (PCM) audio data and the predetermined length is 32 ms and comprises 1536 PCM samples.

5. The computing device according to claim 1, wherein the computer-executable instructions, when executed by the processor, cause the computing device to advertise a metadata format identification indicating that the computing device supports the codec frame having the predetermined length and comprising the first and second separated sections.

6. The computing device according to claim 5, wherein the computer-executable instructions, when executed by the processor, cause the computing device to communicate an acknowledgment that the computing device supports the codec frame having the predetermined length and comprising the first and second separated sections.

7. The computing device according to claim 6, wherein the acknowledgment is communicated in response to the metadata format identification advertised by the processor.

8. The computing device according to claim 1, wherein the spatial audio stream is associated with prerecorded media provided by a streaming service provider that provides streaming media content to endpoint devices and users of the endpoint devices.

9. A computing device, comprising:

a processor;

a computer-readable storage medium in communication with the processor, the computer-readable storage medium having computer-executable instructions stored thereupon which, when executed by the processor, cause the processor to;

receive a codec frame having a predetermined length and comprising first and second separated sections, the first section including at least a portion of audio data from a spatial audio stream and a second section including at least one metadata component extracted from the audio data;

extract the at least one metadata component from the second section;

associate the at least one metadata component at a time based offset position between a beginning of the at least a portion of audio data comprised in the first section and an end of the at least the portion of the audio data comprised in the first section to provide an audio data frame having the at least one metadata component embedded therein at the time based offset position;

generate an audio stream comprising at least the audio data frame; and

communicate the audio stream to one or more audio rendering elements to playback the at least a portion of the audio data.

10. The computing device according to claim 9, wherein the second section includes a plurality of metadata components extracted from the audio data, each of the plurality of metadata components disposed in a segmented section of the second section.

11. The computing device according to claim 10, wherein the plurality of associated metadata components comprises positional metadata including one or more coordinates to render the at least a portion of the audio data in a three-dimensional space, a gain of the at least a portion of audio data, and calibration information for the one or more audio rendering elements to playback the at least a portion of the audio data.

12. The computing device according to claim 9, wherein the audio data is pulse code modulation (PCM) audio data and the predetermined length is 32 ms and comprises 1536 PCM samples.

13. The computing device according to claim 9, wherein the computer-executable instructions, when executed by the processor, cause the computing device to advertise a metadata format identification indicating that the computing device supports the codec frame having the predetermined length and comprising the first and second separated sections.

14. The computing device according to claim 13, wherein the computer-executable instructions, when executed by the processor, cause the computing device to communicate an acknowledgment that the computing device supports the codec frame having the predetermined length and comprising the first and second separated sections.

15. The computing device according to claim 14, wherein the acknowledgment is communicated in response to the metadata format identification advertised by the processor.

16. The computing device according to claim 9, wherein the spatial audio stream is associated with prerecorded media provided by a streaming service provider that provides streaming media content to endpoint devices and users of the endpoint devices.

17. A computer implemented method, the method comprising:

receiving a codec frame having a predetermined length and comprising first and second sections, the first section including at least a portion of audio data from a spatial audio stream and a second section including at least one metadata component extracted from the audio data;

extracting the at least one metadata component from the second section;

associating the at least one metadata component at an offset position between a beginning of the at least a portion of audio data comprised in the first section and an end of the at least the portion of the audio data comprised in the first section to provide an audio data frame having the at least one metadata component embedded therein at the offset position;

generating an audio stream comprising at least the audio data frame; and

communicating the audio stream to one or more audio rendering elements to playback the at least a portion of the audio data.

18. The computer implemented method according to claim 17, further comprising advertising a metadata format identification indicating that the codec frame having the predetermined length and comprising the first and second separated sections is supported by a computing device.

19. The computer implemented method according to claim 18, further comprising communicating an acknowledgment indicating support of the codec frame having the predetermined length and comprising the first and second separated sections.