Audio and video playback system and method for playing audio data applied thereto

• US 10,992,451 B2
• Filed: 11/05/2018
• Issued: 04/27/2021
• Est. Priority Date: 12/01/2017
• Status: Active Grant

• Current Assignee: MediaTek, Inc.
• Original Assignee: MediaTek, Inc.
• Inventors: Zhu, Fu Jun
• Primary Examiner(s): Hussain, Tauqir
• Application Number:
  US16/180,073
• Publication Number:
  US 20190173663A1
• Time in Patent Office: 904 Days
• Field of Search: 709248
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: H04L 2007/045 Fill bit or bits, idle words
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/764 at the destination reformatting of additional data in video clients H04N21/4355
• CPC Class Codes: H04L 7/041 using special codes as synchronising signal
• CPC Class Codes: H04N 21/24 Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests monitoring of server performance or load G06F11/34; arrangements for observation, testing or troubleshooting for broadcast or for distribution combined with broadcast H04H20/12
• CPC Class Codes: H04N 21/43076 of the same content streams on multiple devices, e.g. when family members are watching the same movie on different devices
• CPC Class Codes: H04N 21/43637 involving a wireless protocol, e.g. Bluetooth, RF or wireless LAN [IEEE 802.11] arrangements for wireless networking or broadcasting of information in indoor or near-field type systems H04B10/114
• CPC Class Codes: H04N 21/4392 involving audio buffer management

Abstract

An audio and video playback system includes an audio and video playback device having a local audio device, and a secondary audio device. A method for playing audio data includes: allocating a local audio buffer space and a secondary audio buffer space to the local audio device and the secondary audio device, respectively; processing obtained multimedia data to generate local audio data and secondary audio data; writing the local audio data and the secondary audio data to the local audio buffer space and the secondary audio buffer space, respectively; reading the local audio data and the secondary audio data buffered in the local audio buffer space and the secondary audio buffer space to the local audio device and the secondary audio device, to have the local audio device and the secondary audio device play the local audio data and the secondary audio data, respectively.

Claims

1. A method for playing audio data, the method applied to an audio and video playback system, the audio and video playback system comprising a secondary audio device and an audio and video playback device having a local audio device, wherein the audio and video playback device plays both audio and video, the method comprising:

allocating a local audio buffer space and a secondary audio buffer space to the local audio device and the secondary audio device, respectively;

processing obtained multimedia data to generate local audio data and secondary audio data;

writing the local audio data and the secondary audio data to the local audio buffer space and the secondary audio buffer space, respectively, wherein the local audio data and the secondary audio data are not the same;

reading the local audio data and the secondary audio data buffered in the local audio buffer space and the secondary audio buffer space to the local audio device and the secondary audio device, respectively, to have the local audio device and the secondary audio device simultaneously play the local audio data and the secondary audio data, respectively,

wherein the local audio buffer space and the secondary audio buffer space are each a ring buffer space, such that while the local audio data and the secondary audio data buffered in the local audio buffer space and the secondary audio buffer space are read to the local audio device and the secondary audio device, the local audio data and the secondary audio data are continuously further written to the local audio buffer space and the secondary audio buffer space, respectively; and

determining whether the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device are synchronous;

when it is determined that the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device are asynchronous, executing a synchronization mechanism to synchronize the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device are synchronous;

wherein the step of determining whether the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device are synchronous further comprises;

acquiring delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device, respectively; and

determining, according to the respective delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device, whether the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device are synchronous;

wherein the step of executing the synchronization mechanism to synchronize the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device further comprises;

determining a difference between the delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device;

generating a null packet having a corresponding size according to the difference between the delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device; and

writing the null packet to the local audio buffer space or the secondary audio buffer space corresponding to the local audio device or the secondary audio device having the smaller delay period, to cause the local audio device or the secondary audio device to output the null packet when outputting the local audio data or the secondary audio data, thereby synchronizing the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device.

2. The method according to claim 1, wherein the step of generating the null packet having the corresponding size according to the difference between the delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device comprises:

determining whether the difference between the delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device is greater than a predetermined value; and

when the difference between the delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device is greater than the predetermined value, generating the null packet having the corresponding size according to the difference between the delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device.

5. The method according to claim 1, wherein the audio and video playback device is a television or a television system consisting of a set-top box (STB) and a television, the local audio device is a local speaker built in the television, and the secondary audio device is an external acoustic enclosure connected to the television.

6. The method according to claim 5, wherein the secondary audio device and the television are connected by means of USB wireless connection.

3. A method for playing audio data, the method applied to an audio and video playback system, the audio and video playback system comprising a secondary audio device and an audio and video playback device having a local audio device, wherein the audio and video playback device plays both audio and video, the method comprising:

allocating a local audio buffer space and a secondary audio buffer space to the local audio device and the secondary audio device, respectively;

processing obtained multimedia data to generate local audio data and secondary audio data;

writing the local audio data and the secondary audio data to the local audio buffer space and the secondary audio buffer space, respectively, wherein the local audio data and the secondary audio data are not the same;

reading the local audio data and the secondary audio data buffered in the local audio buffer space and the secondary audio buffer space to the local audio device and the secondary audio device, respectively, to have the local audio device and the secondary audio device simultaneously play the local audio data and the secondary audio data, respectively;

wherein the local audio buffer space and the secondary audio buffer space are each a ring buffer space, such that while the local audio data and the secondary audio data buffered in the local audio buffer space and the secondary audio buffer space are read to the local audio device and the secondary audio device, the local audio data and the secondary audio data are continuously further written to the local audio buffer space and the secondary audio buffer space, respectively; and

determining whether the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device are synchronous;

when it is determined that the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device are asynchronous, executing a synchronization mechanism to synchronize the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device are synchronous;

wherein the step of determining whether the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device are synchronous further comprises;

acquiring delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device, respectively; and

determining, according to the respective delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device, whether the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device are synchronous,

wherein the step of executing the synchronization mechanism to synchronize the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device further comprises;

determining a difference between the delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device;

obtaining a corresponding data size according to the difference between the delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device; and

discarding data having the corresponding data size from the local audio buffer space or the secondary audio buffer space corresponding to the local audio device or the secondary audio device having the greater delay period, thereby synchronizing the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device.

4. The method according to claim 3, wherein the step of obtaining the corresponding data size according to the difference between the delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device further comprises:

determining whether the difference between the delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device is greater than a predetermined value; and

when the difference between the delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device is greater than the predetermined value, obtaining the corresponding data size according to the difference between the delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device.

7. An audio and video playback system, comprising:

an audio and video playback device, for playing multimedia data, the audio and video playback device comprising a local audio device, a processor, a memory and a buffer unit, wherein the audio and video playback device plays both audio and video; and

a secondary audio device, connected to the audio and video playback device;

wherein the processor in the audio and video playback device performs steps of;

allocating a local audio buffer space and a secondary audio buffer space to the local audio device and the secondary audio device, respectively;

processing the obtained multimedia data to generate local audio data and secondary audio data;

writing the local audio data and the secondary audio data to the local audio buffer space and the secondary audio buffer space, respectively; and

reading the local audio data and the secondary audio data buffered in the local audio buffer space and the secondary audio buffer space to the local audio device and the secondary audio device, respectively, to have the local audio device and the secondary audio device simultaneously play the local audio data and the secondary audio data, respectively;

wherein, an audio device management program is stored in the memory of the audio and video playback device;

when the audio device management program is executed, the audio and video playback device controls the local audio device and the secondary audio device according to the audio device management program to perform a collaborated playback task to play audio data in the multimedia data;

wherein when the audio and video playback device controls the local audio device and the secondary audio device according to the audio device management program to perform the collaborated playback task to play audio data in the multimedia data;

wherein the local audio buffer space and the secondary audio buffer space are each a ring buffer space, such that while the local audio data and the secondary audio data buffered in the local audio buffer space and the secondary audio buffer space are read to the local audio device and the secondary audio device, the local audio data and the secondary audio data are continuously further written to the local audio buffer space and the secondary audio buffer space, respectively;

wherein when the audio and video playback controls the local audio device and the secondary audio device to collaboratively play the audio data in the multimedia data, the processor further performs steps of;

determining whether the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device are synchronous;

when it is determined that the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device are asynchronous, executing a synchronization mechanism to synchronize the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device are synchronous;

wherein the step of determining whether the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device are synchronous further comprises;

acquiring delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device, respectively; and

determining, according to the respective delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device, whether the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device are synchronous;

wherein the step of the step of executing the synchronization mechanism to synchronize the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device further comprises;

determining a difference between the delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device;

generating a null packet having a corresponding size according to the difference between the delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device; and

writing the null packet to the local audio buffer space or the secondary audio buffer space corresponding to the local audio device or the secondary audio device having the smaller delay period, to cause the local audio device or the secondary audio device to output the null packet when outputting the local audio data or the secondary audio data, thereby synchronizing the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device.

9. The audio and video playback system according to claim 7, wherein the audio device management program further comprises an audio device playback mode selection function;

when the audio device playback mode selection function is executed, the audio and video playback device drives the local audio device and/or the secondary audio device according to the selected audio device playback mode to play the audio data in the multimedia data.

10. The audio and video playback system according to claim 7, wherein the audio and video playback device is a television or a television system consisting of a set-top box (STB) and a television, the local audio device is a local speaker built in the television, and the secondary audio device is an external acoustic enclosure connected to the television.

11. The audio and video playback system according to claim 7, wherein the secondary audio device and the television are connected by means of USB wireless connection.

8. An audio and video playback system, comprising:

an audio and video playback device, for playing multimedia data, the audio and video playback device comprising a local audio device, a processor, a memory and a buffer unit, wherein the audio and video playback device plays both audio and video; and

a secondary audio device, connected to the audio and video playback device;

wherein the processor in the audio and video playback device performs steps of;

allocating a local audio buffer space and a secondary audio buffer space to the local audio device and the secondary audio device, respectively;

processing the obtained multimedia data to generate local audio data and secondary audio data;

writing the local audio data and the secondary audio data to the local audio buffer space and the secondary audio buffer space, respectively; and

reading the local audio data and the secondary audio data buffered in the local audio buffer space and the secondary audio buffer space to the local audio device and the secondary audio device, respectively, to have the local audio device and the secondary audio device simultaneously play the local audio data and the secondary audio data, respectively;

wherein, an audio device management program is stored in the memory of the audio and video playback device;

when the audio device management program is executed, the audio and video playback device controls the local audio device and the secondary audio device according to the audio device management program to perform a collaborated playback task to play audio data in the multimedia data;

wherein when the audio and video playback device controls the local audio device and the secondary audio device according to the audio device management program to perform the collaborated playback task to play audio data in the multimedia data;

wherein the local audio buffer space and the secondary audio buffer space are each a ring buffer space, such that while the local audio data and the secondary audio data buffered in the local audio buffer space and the secondary audio buffer space are read to the local audio device and the secondary audio device, the local audio data and the secondary audio data are continuously further written to the local audio buffer space and the secondary audio buffer space, respectively;

wherein when the audio and video playback controls the local audio device and the secondary audio device to collaboratively play the audio data in the multimedia data, the processor further performs steps of;

determining whether the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device are synchronous;

when it is determined that the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device are asynchronous, executing a synchronization mechanism to synchronize the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device are synchronous;

wherein the step of determining whether the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device are synchronous further comprises;

acquiring delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device, respectively; and

determining, according to the respective delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device, whether the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device are synchronous;

wherein the step of executing the synchronization mechanism to synchronize the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device further comprises;

determining a difference between the delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device;

obtaining the corresponding data size according to the difference between the delay periods of the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device; and

discarding data having the corresponding data size from the local audio buffer space or the secondary audio buffer space corresponding to the local audio device or the secondary audio device having the greater delay period, thereby synchronizing the local audio data and the secondary audio data outputted from the local audio device and the secondary audio device.