System and method for effectively performing an audio/video synchronization procedure

• US 7,697,571 B2
• Filed: 09/28/2006
• Issued: 04/13/2010
• Est. Priority Date: 11/20/2001
• Status: Expired due to Fees

• Current Assignee: Saturn Licensing LLC (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Inventors: Zimmermann, Klaus
• Primary Examiner(s): Sheikh; Ayaz R
• Assistant Examiner(s): LEE, ANDREW CHUNG CHEUNG
• Application Number:
  US11/540,865
• Publication Number:
  US 20070019684A1
• Time in Patent Office: 1,293 Days
• Field of Search: 370/486, 370/498, 370/473, 370/412, 370/503, 370/464, 370/100.1, 370/487, 370/529, 370/535, 370/537, 375/240, 375/240.12, 375/240.27
• US Class Current: 370/503
• CPC Class Codes: H04N 21/4305 Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets arrangements for synchronising receiver with transmitter by comparing receiver clock with transmitter clock H04L7/0012; arrangements for synchronising receiver with transmitter wherein the receiver takes measures against momentary loss of synchronisation H04L7/0083
• CPC Class Codes: H04N 21/43072 of multiple content streams on the same device
• CPC Class Codes: H04N 21/4341 Demultiplexing of audio and video streams

Abstract

A system and method are disclosed for effectively performing an audio/video synchronization procedure in a receiver device that is embodied in a computer that receives input data from a source device and provides output data to a destination device. The receiver device may preferably include a demultiplexer configured to recover elementary bitstreams from a multiplexed bitstream. The demultiplexer may also preferably extract decode timestamps and output timestamps corresponding to the elementary bitstreams. One or more decoders may then decode the elementary bitstreams to produce decoded frames in accordance with the foregoing decode timestamps. One or more output modules may then process the decoded frames to produce processed frames in accordance with the output timestamps. In accordance with the present invention, an output controller may preferably perform an output timing resynchronization procedure following a program change event to align output timings of the processed frames in accordance with new output timestamps from the selected program.

Claims

1. A system for performing a data synchronization procedure, comprising:

a demultiplexer that recovers elementary bitstreams, and separately extracts decode timestamps and output timestamps from said elementary bitstreams, said elementary bitstreams being stored into an input buffer;

one or more decoders that decode said elementary bitstreams to produce decoded frames that are stored into an output buffer;

an input controller that synchronizes said one or more decoders to said decode timestamps to read said elementary bitstreams from said input buffer without varying from said decode timestamps;

one or more output modules that process said decoded frames to produce processed frames; and

an output controller that controls said one or more output modules to read said decoded frames from said output buffer in an output timing resynchronization procedure that resynchronizes output frame timings of said processed frames to a new series of said output timestamps that are generated after a data change in said elementary bitstreams.

2. The system of claim 1 wherein said data synchronization procedure is performed by a computer that receives a multiplexed bitstream from a data source and responsively generates said processed frames to one or more destination devices.

3. The system of claim 2 wherein said one or more elementary bitstreams include a video bitstream and an audio bitstream, said one or more decoders including a video decoder and an audio decoder, said one or more output modules including a video output module and an audio output module.

4. The system of claim 3 wherein said input controller and said output controller are decoupled to operate independently, and wherein said computer handles said video bitstream and said audio bitstream independently by utilizing a plurality of different timebases.

5. The system of claim 3 wherein said output timing resynchronization procedure is performed after receiving new output timestamps as a result of a program change event.

6. The system of claim 3 wherein said video decoder accesses said video bitstream from a video decoder buffer and stores decoded video frames into a video output buffer, said video output module accessing said decoded video frames from said video output buffer, said audio decoder accessing said audio bitstream from an audio decoder buffer and storing decoded audio frames into an audio output buffer, said audio output module accessing said decoded audio frames from said audio output buffer.

7. The system of claim 3 wherein said demultiplexer separates a composite bitstream into said elementary bitstreams, said decode timestamps including video decode timestamps and audio decode timestamps, said output timestamps including video output timestamps and audio output timestamps.

8. The system of claim 7 wherein said input controller instructs said video decoder to generate a decoded video frame when a corresponding one of said video decode timestamps equals a system time clock, said input controller also instructing said audio decoder to generate a decoded audio frame when a corresponding one of said audio decode timestamps equals said system time clock.

9. The system of claim 8 wherein said output controller instructs said video output module to output a processed video frame when a corresponding one of said video output timestamps equals said system time clock, said output controller also instructing said audio output module to output a processed audio frame when a corresponding one of said audio output timestamps equals said system time clock.

10. The system of claim 9 wherein said computer generates a series of decoded video frames, said computer also generating a series of decoded audio frames, said computer subsequently outputting a series of processed video frames corresponding to said decoded video frames, said computer also subsequently outputting a series of processed audio frames corresponding to said decoded audio frames.

11. The system of claim 3 wherein a system user instructs said computer to select a new program, said computer responsively performing a program search procedure to locate said new program.

12. The system of claim 11 wherein said demultiplexer separates a composite bitstream into said elementary bitstreams, said decode timestamps including new decode timestamps, said output timestamps including new output timestamps.

13. The system of claim 12 wherein said input controller instructs said one or more decoders to generate one of said decoded frames when a corresponding one of said new decode timestamps equals a system time clock.

14. The system of claim 13 wherein said one or more decoders store said one of said decoded frames into a buffer memory for said one or more output modules to access.

15. The system of claim 14 wherein said computer generates a series of decoded frames, said computer subsequently outputting a series of processed frames corresponding to said decoded frames.

16. The system of claim 14 wherein said output controller determines whether said output frame timings of said processed frames are aligned with said new output timestamps.

17. The system of claim 16 wherein said output controller performs said output timing resynchronization procedure to align said output frame timings of said processed frames with said new output timestamps.

18. The system of claim 17 wherein said output controller instructs said one or more output modules to sequentially output one of said processed frames when a corresponding one of said new output timestamps equals said system time clock.

19. The system of claim 18 wherein said computer outputs a series of processed frames corresponding to said decoded frames.

20. The system of claim 10 wherein said computer outputs said series of processed video frames to a video display device, said computer also outputting said series of processed audio frames to an audio reproduction system.

21. The system of claim 1 wherein said output timing resynchronization procedure is performed after receiving new output timestamps as a result of a bitstream discontinuity.

22. The system of claim 1 wherein said output timing resynchronization procedure is performed after receiving new output timestamps as a result of a powerup initialization event.

23. A method for performing a data synchronization procedure, comprising the steps of:

extracting decode timestamps and output timestamps corresponding to elementary bitstreams, said elementary bitstreams being stored into an input buffer;

controlling one or more decoders to read said elementary bitstreams from said input buffer according to said decode time stamps to produce decoded frames without varying from said decode timestamps, said decoded frames being stored into an output buffer; and

controlling one or more output modules to read said decoded frames from said output buffer according to said output timestamps to perform an output timing resynchronization procedure that resynchronizes output frame timings of processed output frames according to said output timestamps.

24. The method of claim 23 wherein said data synchronization procedure is performed by a computer device in a distributed electronic network.

25. The method of claim 24 wherein said computer is a client of an encoder device that provides said elementary bitstreams over said distributed electronic network.

26. The method of claim 24 wherein said computer is a client of a broadcasting device that provides said elementary bitstreams over said distributed electronic network.

27. The method of claim 24 wherein said elementary bitstreams include one or more audio/video programs.

28. The method of claim 23 wherein said output timing resynchronization procedure resynchronizes output frame timings of said processed frames to a new series of said output timestamps that are generated after a data change in said elementary bitstreams.

29. The method of claim 28 wherein said data change is a bitstream discontinuity in said elementary bitstreams.

30. The method of claim 28 wherein said data change is a powerup event in said computer.

31. The method of claim 28 wherein said data change is a program change event in said computer.

32. The method of claim 23 wherein said data synchronization procedure is performed by a wireless communications device in a distributed electronic network.

33. The method of claim 24 wherein said computer is a server device that provides said elementary bitstreams to one or more client destination devices over said distributed electronic network.

34. The method of claim 24 wherein said input controller extracts said decode timestamps, and an output controller extracts said output timestamps, said input controller and said output controller being decoupled to operate independently, and wherein said computer handles said video bitstream and said audio bitstream independently by utilizing a plurality of different timebases.

35. The method of claim 24 wherein said computer is a client of a data source device that transmits said elementary bitstreams over said distributed electronic network.

36. An apparatus for performing a data synchronization procedure in an electronic network, comprising:

a computer that extracts decode timestamps and output timestamps corresponding to an elementary bitstream that is stored into an input buffer;

one or more decoders that are synchronized to read said elementary bitstreams from said input buffer according to said decode timestamps to produce decoded frames without varying from said decode timestamps, said decoded frames being stored into an output buffer; and

one or more output modules that are controlled to read said decoded frames from said output buffer according to said output timestamps to resynchronize output frame timings of processed output frames,

said computer providing said processed output frames to one or more destination devices in said electronic network.

37. The apparatus of claim 36 wherein said computer and said one or more destination devices bi-directionally communicate in said electronic network by utilizing wireless communication techniques.

38. An apparatus for performing a data synchronization procedure in an electronic network, comprising:

a computer that extracts first timestamps and second timestamps corresponding to an elementary bitstream that is stored into an input buffer;

one or more decoders that are synchronized to read said elementary bitstreams from said input buffer according to said first timestamps to produce decoded frames without varying from said first timestamps, said decoded frames being stored into an output buffer; and

one or more input modules that are controlled to read said decoded frames from said output buffer according to said second timestamps to resynchronize frame timings of processed frames, said computer receiving said elementary bitstream from a data source device that transmits said elementary bitstream over said electronic network.

39. The apparatus of claim 38 wherein said computer and a data source device bi-directionally communicate in said electronic network by utilizing wireless communication techniques.

40. A computer device for performing a data synchronization procedure in an electronic network, comprising:

an input module that extracts decode timestamps and output timestamps corresponding to an elementary bitstream that is stored into an input buffer;

one or more decoders that are synchronized according to said decode timestamps to produce decoded frames without varying from said decode timestamps, said decoded frames being stored into an output buffer; and

one or more output modules that are controlled to read said decoded frames from said output buffer according to said output timestamps to resynchronize output frame timings of processed output frames, said computer device providing said processed output frames to one or more destination devices in said electronic network.

41. The computer device of claim 40 wherein said computer device and said one or more destination devices bi-directionally communicate in said electronic network by utilizing wireless communication techniques.

42. A computer device for performing a data synchronization procedure in an electronic network, comprising:

an input module that extracts first timestamps and second timestamps corresponding to an elementary bitstream that is stored into an input buffer;

one or more decoders that are synchronized to read said elementary bitstreams from said input buffer according to said first timestamps to produce decoded frames without varying from said decode timestamps, said decoded frames being stored into an output buffer; and

one or more input modules that are controlled to read said decoded frames from said output buffer according to said second timestamps to resynchronize frame timings of processed frames, said computer device receiving said elementary bitstream from a data source device that transmits said elementary bitstream over said electronic network.

43. The computer device of claim 42 wherein said computer device and a data source device bi-directionally communicate in said electronic network by utilizing wireless communication techniques.

44. A device for performing a data synchronization procedure in an electronic network, comprising:

means for receiving an elementary bitstream from a data source device that transmits said elementary bitstream over said electronic network, said elementary bitstream being stored into an input buffer

means for extracting first timestamps and second timestamps corresponding to said elementary bitstream without varying from said first timestamps and said second timestamps;

means for controlling one or more decoders to read said elementary bitstreams from said input buffer according to said first timestamps to produce decoded frames, said decoded frames being stored into an output buffer; and

means for controlling one or more input modules to read said decoded frames from said output buffer according to said second timestamps to perform a timing resynchronization procedure that resynchronizes frame timings of processed frames according to said second timestamps.

45. The device of claim 44 wherein said device is implemented as a client of an encoder that provides said elementary bitstream over said electronic network.

46. The device of claim 44 wherein said device is implemented as a client of a broadcasting entity that provides said elementary bitstream over said electronic network.

47. The device of claim 44 wherein said timing resynchronization procedure is triggered by a bitstream discontinuity in said elementary bitstream.

48. The device of claim 44 wherein said device is implemented to include a receiver module.

49. A device for performing a data synchronization procedure in an electronic network, comprising:

means for extracting first timestamps and second timestamps corresponding to an elementary bitstream that is stored into an input buffer;

means for controlling one or more decoders to read said elementary bitstreams from said input buffer according to said first timestamps to produce decoded frames without varying from said first timestamps, said decoded frames being stored into an output buffer;

means for controlling one or more interface modules to read said decoded frames from said output buffer according to said second timestamps to perform a timing resynchronization procedure that resynchronizes frame timings of processed frames according to said second timestamps; and

means for providing said processed output frames to one or more destination devices in said electronic network.

50. The device of claim 49 wherein said timing resynchronization procedure is triggered by a powerup event in said device.

51. The device of claim 49 wherein said timing resynchronization procedure is triggered by a program change event in said device.

52. The device of claim 49 wherein said elementary bitstream includes one or more audio and/or video programs.