Systems, methods, and apparatus for synchronization of audio and video signals

• US 8,363,161 B2
• Filed: 05/25/2007
• Issued: 01/29/2013
• Est. Priority Date: 05/26/2006
• Status: Active Grant

• Current Assignee: Avago Technologies International Sales Pte Limited (Broadcom, Inc.)
• Original Assignee: Broadcom Corporation (Broadcom, Inc.)
• Inventors: Pearlstein, Larry
• Primary Examiner(s): Natnael, Paulos M
• Application Number:
  US11/754,096
• Publication Number:
  US 20070276670A1
• Time in Patent Office: 2,076 Days
• Field of Search: 348/515, 348/473, 348/460, 348/461, 348/464, 348/512, 725/136, 725/112, 725/113
• US Class Current: 348/515
• CPC Class Codes: G10L 19/167 Audio streaming, i.e. formatting and decoding of an encoded audio signal representation into a data stream for transmission or storage purposes
• CPC Class Codes: H04N 21/2368 Multiplexing of audio and video streams
• CPC Class Codes: H04N 21/43072 of multiple content streams on the same device
• CPC Class Codes: H04N 21/4341 Demultiplexing of audio and video streams
• CPC Class Codes: H04N 21/44004 involving video buffer management, e.g. video decoder buffer or video display buffer

Abstract

Methods of processing an audiovisual signal that has a video portion and an audio portion are described. One example includes detecting a video synchronization event in the video portion and, in response to the detecting, embedding a marker relating to the video synchronization event into a serial data stream carrying the audio portion. The serial data stream includes a series of packets, each packet having (A) a preamble that includes a synchronization sequence, (B) an auxiliary data field, and (C) a main data field.

Claims

1. A method of processing an audiovisual signal that has a video portion and an audio portion, said method comprising:

detecting a video synchronization event in the video portion; and

in response to said detecting, embedding a marker relating to the video synchronization event into a serial data stream carrying the audio portion, wherein the serial data stream comprises at least one packet, each packet having a preamble that includes a synchronization sequence, an auxiliary data field, and a main data field, wherein said embedding comprises inserting the marker into the auxiliary data field of at least one of the packets; and

embedding, into the at least one packet, a tag that indicates a presence of the marker in the packet and is separate from the marker.

2. The method of processing an audiovisual signal according to claim 1, wherein said marker indicates a time of said video synchronization event relative to a reference time.

3. The method of processing an audiovisual signal according to claim 1, wherein said detecting a video synchronization event in the video portion comprises performing, on at least one scan line of each of a plurality of frames of the video portion, at least one among a Fourier transform and a correlation with a code sequence.

4. A method of processing an audiovisual signal that has a video portion and an audio portion, said method comprising:

detecting a video synchronization event in the video portion; and

in response to said detecting, embedding a marker relating to the video synchronization event into a serial data stream carrying the audio portion,

wherein the serial data stream comprises at least one packet, each packet having a preamble that includes a synchronization sequence, an auxiliary data field, and a main data field, wherein said embedding comprises inserting the marker into the auxiliary data field of at least one of the packets, and wherein the synchronization sequence of each of the packets occurs in the first four time slots of the packet, and wherein the auxiliary data field of each of the packets occurs in the second four time slots of the packet.

5. A method of processing an audiovisual signal that has a video portion and an audio portion, said method comprising:

detecting a video synchronization event in the video portion; and

in response to said detecting, embedding a marker relating to the video synchronization event into a serial data stream carrying the audio portion,

wherein the serial data stream comprises at least one packet, each packet having a preamble that includes a synchronization sequence, an auxiliary data field, and a main data field, wherein the serial data stream comprises a series of blocks, each of the series of blocks having one hundred ninety-two frames, and wherein each of the one hundred ninety-two frames of each of the series of blocks has two of the series of packets.

6. A method of processing an audiovisual signal that has a video portion and an audio portion, said method comprising:

performing a video processing operation on the video portion; and

embedding, into a serial data stream carrying the audio portion, a marker relating to the video processing operation,

wherein the serial data stream comprises at least one packet, each packet having a preamble that includes a synchronization sequence, an auxiliary data field, and a main data field, wherein embedding the marker comprises inserting the marker into the auxiliary data field of at least one of the packets, wherein the synchronization sequence of each of the packets occurs in a first group of time slots of the packet, and wherein the auxiliary data field of each of the packets occurs in a second group of time slots of the packet following the first group of time slots.

7. The method of processing an audiovisual signal according to claim 6, wherein said performing a video processing operation comprises performing at least one among a scaling operation, a de-interlacing operation, and a noise reduction operation on the video portion.

8. The method of processing an audiovisual signal according to claim 6, wherein the marker relates to a delay of the video processing operation and indicates one among a number of video frames and a number of audio samples.

9. The method of processing an audiovisual signal according to claim 6, wherein the marker relates to a delay of the video processing operation and wherein said performing a video processing operation on the video portion includes delaying the video portion by a period of at least one video frame.

10. The method of processing an audiovisual signal according to claim 6, wherein the synchronization sequence of each of the packets occurs in the first four time slots of the packet, and wherein the auxiliary data field of each of the packets occurs in the second four time slots of the packet.

11. The method of processing an audiovisual signal according to claim 6, wherein said method comprises embedding, into the at least one packet, a tag that indicates the presence of the marker in the packet and is separate from the marker.

12. The method of processing an audiovisual signal according to claim 6, wherein the serial data stream comprises a series of blocks, each of the series of blocks having one hundred ninety-two frames, and

wherein each of the one hundred ninety-two frames of each of the series of blocks has two of the series of packets.

13. An apparatus for processing an audiovisual signal that has a video portion and an audio portion, said apparatus comprising:

a video processor configured to perform a video processing operation on the video portion; and

a timing information embedder configured to embed, into a serial data stream carrying the audio portion, a marker relating to the video processing operation, wherein the timing information embedder is configured to embed the marker in response to detecting a video synchronization event in the video portion,

wherein the serial data stream comprises at least one packet, each packet having a preamble that includes a synchronization sequence, an auxiliary data field, and a main data field, wherein the timing information embedder is configured to embed the marker by inserting the marker into the auxiliary data field of at least one of the packets, and wherein the timing information embedder is further configured to embed, into the at least one packet, a tag that indicates a presence of the marker in the packet and is separate from the marker.

14. The apparatus for processing an audiovisual signal according to claim 13, wherein said video processing operation comprises at least one among a scaling operation, a de-interlacing operation, and a noise reduction operation.

15. The apparatus for processing an audiovisual signal according to claim 13, wherein said apparatus comprises an integrated circuit including said video processor and said timing information embedder.

16. The apparatus for processing an audiovisual signal according to claim 13, wherein said apparatus comprises a display panel configured to display a video signal based on the processed video portion.

17. A non-transitory computer program product for processing an audiovisual signal that has a video portion and an audio portion, said product comprising a computer-readable medium, said medium comprising:

code for causing at least one computer to detect a video synchronization event in the video portion; and

code for causing at least one computer to embed, in response to said detecting, a marker relating to said video synchronization event into a serial data stream carrying the audio portion,

wherein the serial data stream comprises at least one packet, each packet having a preamble that includes a synchronization sequence, an auxiliary data field, and a main data field, wherein the code that causes at least one computer to embed the marker inserts the marker into the auxiliary data field of at least one of the packets, and

code that causes at least one computer to embed, into the at least one packet, a tag that indicates a presence of the marker in the packet and is separate from the marker.

18. A non-transitory computer program product for processing an audiovisual signal that has a video portion and an audio portion, said product comprising a computer-readable medium, said medium comprising:

code for causing at least one computer to perform a video processing operation on the video portion; and

code for causing at least one computer to embed, into a serial data stream carrying the audio portion, a marker relating to the video processing operation,

wherein the serial data stream comprises at least one packet, each packet having a preamble that includes a synchronization sequence, an auxiliary data field, and a main data field, wherein embedding the marker comprises inserting the marker into the auxiliary data field of at least one of the packets, wherein the synchronization sequence of each of the packets occurs in a first group of time slots of the packet, and wherein the auxiliary data field of each of the packets occurs in a second group of time slots of the packet following the first group of time slots.