Synchronization and switch over methods and systems for an adaptive audio system

• US 8,838,262 B2
• Filed: 06/27/2012
• Issued: 09/16/2014
• Est. Priority Date: 07/01/2011
• Status: Active Grant

• Current Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Inventors: Mehta, Sripal S., Martinez, Sergio, Grossman, Ethan A., Thayer, Brad, Bullock, Dean, Neary, John
• Primary Examiner(s): KOSTAK, VICTOR R
• Application Number:
  US14/127,135
• Publication Number:
  US 20140139738A1
• Time in Patent Office: 811 Days
• Field of Search: 348/738, 348/515, 370/503, 370/516, 370/517, 370/519, 370509-512, 375/240.28, 352/12, 352/23, 700/94, 381/123, 704/270, 704/500
• US Class Current: 700/94
• CPC Class Codes: H04N 21/233 Processing of audio elementary streams monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/43072 of multiple content streams on the same device
• CPC Class Codes: H04N 21/44016 involving splicing one content stream with another content stream, e.g. for substituting a video clip
• CPC Class Codes: H04N 21/8547 involving timestamps for synchronizing content
• CPC Class Codes: H04N 5/04 Synchronising for television systems using pulse code modulation H04N7/56
• CPC Class Codes: H04N 5/607 for more than one sound signal, e.g. stereo, multilanguages H04N5/602 takes precedence
• CPC Class Codes: H04R 2227/003 Digital PA systems using, e.g. LAN or internet
• CPC Class Codes: H04R 27/00 Public address systems circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; monitoring or testing arrangements for public address systems H04R29/007; amplifiers H03F
• CPC Class Codes: H04S 2420/03 Application of parametric coding in stereophonic audio systems
• CPC Class Codes: H04S 3/008 in which the audio signals are in digital form, i.e. employing more than two discrete digital channels data reduction aspects thereof based on psychoacoustics G10L19/02

Abstract

Embodiments are described for a synchronization and switchover mechanism for an adaptive audio system in which multi-channel (e.g., surround sound) audio is provided along with object-based adaptive audio content. A synchronization signal is embedded in the multi-channel audio stream and contains a track identifier and frame count for the adaptive audio stream to play out. The track identifier and frame count of a received adaptive audio frame is compared to the track identifier and frame count contained in the synchronization signal. If either the track identifier or frame count does not match the synchronization signal, a switchover process fades out the adaptive audio track and fades in the multi-channel audio track. The system plays the multi-channel audio track until the synchronization signal track identifier and frame count and adaptive audio track identifier and frame count match, at which point the adaptive audio content will be faded back in.

Claims

1. A method for processing audio signals, comprising:

generating multi-channel audio content comprising channel-based audio signals and playback information specifying speakers of a speaker array through which respective channel-based signals are to be played;

generating adaptive audio content comprising object-based audio signals and three-dimensional location rendering information for the object-based audio signals;

wherein the multi-channel audio content or the adaptive audio content are to be played out with video content;

wherein the multi-channel audio content is synchronized with the video content; and

providing a synchronization signal along with a current frame of the multi-channel audio content for synchronizing playback of the object-based audio signals relative to the video content, wherein the synchronization signal includes track identifier and frame count information for a current track and a current frame of the adaptive audio content;

wherein the current frame of the adaptive audio content is to be played out if a track identifier and frame count of the received adaptive audio content matches the track identifier and frame count in the synchronization signal, and otherwise the multi-channel audio content is to be played back.

2. The method of claim 1 further comprising:

comparing the track identifier and frame count information in the synchronization signal for the adaptive audio content with the track identifier and frame count information for a received object-based audio signal of the adaptive audio content; and

implementing a switchover process if the track identifier and frame count information in the synchronization signal does not match the track identifier and frame count information for the received object-based audio signal, or if the synchronization signal is not present.

3. The method of claim 2 wherein the switchover process comprises:

fading out the object-based audio signals until a silence threshold is reached; and

fading in the channel-based audio signals.

4. The method of claim 3 wherein, upon a match of the track identifier and frame count information in the synchronization signal with track identifier and frame count information for a subsequent received object-based audio signal, the switchover process further comprises:

fading out the channel-based audio signals; and

fading in the subsequent object-based audio signals.

5. The method of claim 1 wherein the synchronization signal is included as part of an audio visual input signal that contains the video content, the multi-channel audio content, and the adaptive audio content.

6. The method of claim 1 wherein the synchronization signal is generated automatically by a media block component that formats the multi-channel audio content received in accordance with a digital audio signal transport standard to produce real-time audio content.

7. A system for synchronizing audio and video signals comprising:

a server receiving an audio visual signal and outputting video content and multi-channel audio content over a first Ethernet connection and outputting adaptive object-based audio content over a second Ethernet connection;

wherein the multi-channel audio content is synchronized with the video content;

a media block coupled to the server over the first Ethernet connection and receiving the multi-channel audio content and outputting the multi-channel audio content, wherein a current frame of the multi-channel audio content is output along with a synchronization signal containing track identifier and frame count information for a current frame of the adaptive object-based audio content for comparison with track identifier and frame count information of the adaptive object-based audio content;

an adaptive audio processor coupled to the server over the second Ethernet connection and to the media block and receiving the multi-channel audio content, the synchronization signal and the adaptive object-based audio content;

a comparator circuit of the adaptive audio processor comparing a track identifier and frame count of the received adaptive object-based audio content with the track identifier and frame count in the synchronization signal; and

a playback system configured to render and playback the current frame of the adaptive object-based audio content if the track identifier and frame count of the received adaptive object-based audio content matches the track identifier and frame count in the synchronization signal, and otherwise to playback the multi-channel audio content.

8. The system of claim 7 wherein the multi-channel audio content comprises digital cinema content including channel-based audio for playback through a surround sound audio system.

9. The system of claim 8 further comprising a speaker array including surround sound speakers positioned in a defined surround configuration and a plurality of additional speakers positioned in a room defining an audio playback environment.

10. The system of claim 9 wherein the adaptive object-based audio content comprises object-based audio signals and metadata comprising three-dimensional location information specifying a location in three-dimensional space that respective signals of the object-based audio signals are intended to emanate from in a playback environment containing the speaker array.

11. The system of claim 10 further comprising a switchover component configured to fade out audio playback through the speaker array if the track identifier and frame count of the received adaptive object-based audio content does not match the track identifier and frame count encoded in the synchronization signal, fade in and playback the multi-channel audio content until the frame number of the received adaptive object-based audio content matches the frame number encoded with the synchronization signal, and then fade in a subsequent adaptive audio frame when its track identifier and frame count matches the track identifier and frame count encoded with the synchronization signal after fading out the multi-channel audio content.

12. The system of claim 11 wherein the multi-channel audio content is formatted as real-time audio, and wherein the synchronization signal comprises an AES synchronization signal that is embedded in a DCI audio track file of the multi-channel audio content.

13. The system of claim 12 wherein the synchronization signal is formatted as a SMPTE 337M non-audio stream signal, and comprises a plurality of data fields specifying at least a track identifier and frame count.

14. The system of claim 13 further comprising a synchronization manager component receiving both the multi-channel audio content and the adaptive object-based audio content, and one or more input and output buffers coupled to the synchronization manager component for storing audio samples of the multi-channel audio content.

15. A method for synchronizing audio and video signals comprising:

receiving an input audio visual signal and outputting video content and multi-channel audio content over a first Ethernet connection and adaptive object-based audio content over a second Ethernet connection;

wherein the multi-channel audio content is synchronized with the video content;

receiving the multi-channel audio content in a media block and outputting the multi-channel audio content, wherein the multi-channel audio content is provided along with frame information including a track identifier and frame count for the adaptive object-based audio content for synchronizing frames of the adaptive object-based audio content to corresponding frames of the multi-channel audio content;

comparing a track identifier and frame count of a current frame of the received adaptive object-based audio content with the track identifier and frame count provided with a current frame of the multi-channel audio content; and

rendering and playing back the adaptive object-based audio content if the track identifier and frame number of the received adaptive object-based audio content matches the track identifier and frame number associated with the multi-channel audio content, and otherwise playing back the multi-channel audio content.

16. The method of claim 15 wherein the multi-channel audio content comprises digital cinema content including channel-based audio for playback through a surround sound audio system that includes surround sound speakers positioned in a defined surround configuration and a plurality of additional speakers positioned in a room defining an audio playback environment.

17. The method of claim 16 wherein the adaptive object-based audio content comprises object-based audio signals and metadata comprising three-dimensional location information specifying a location in three-dimensional space that respective signals of the object-based audio signals are intended to emanate from in a playback environment containing the surround sound audio system.

18. The method of claim 17 further comprising:

fading out audio playback through the surround sound audio system if the track identifier and frame count of the received adaptive object-based audio content does not match the track identifier and frame count encoded in the multi-channel audio content; and

fading in the multi-channel audio content until the track identifier and frame count of the received adaptive object-based audio content matches the frame number associated with the multi-channel audio content.

19. A method for processing audio signals, comprising:

generating multi-channel audio content comprising channel-based audio signals and playback information specifying speakers of a speaker array through which respective channels of the channel-based audio signals are to be played;

generating adaptive audio content comprising object-based audio signals and three-dimensional location rendering information for the object-based audio signals;

providing a synchronization signal along with the multi-channel audio content for synchronizing playback of the object-based audio signals relative to video content to be played back with the multi-channel audio content or the adaptive audio content;

wherein the multi-channel audio content is synchronized with the video content; and

implementing a switchover process to play back a current frame of the channel-based audio signals with the video content if a comparison of track identifier and frame count information of a current frame of the adaptive audio content included in the synchronization signal with track identifier and frame count information of the current frame of the adaptive audio content results in a mismatch.

20. The method of claim 19 wherein the synchronization signal includes track identifier and frame count information for a current track of the adaptive audio content.