Synchronized transmission of audio and video data from a computer to a client via an interface

• US 7,970,926 B2
• Filed: 03/28/2008
• Issued: 06/28/2011
• Est. Priority Date: 06/13/2003
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Agnoli, Giovanni M., Dillon, Jeffrey C., Delwiche, James A., Abt, John O., Yanowitz, Andrew, Bowman, Samuel R.
• Primary Examiner(s): Nguyen; Quang N.
• Application Number:
  US12/079,832
• Publication Number:
  US 20080250469A1
• Time in Patent Office: 1,187 Days
• Field of Search: 709/224, 709/231, 709/232, 709/249, 370/229, 370/233, 370/235, 370/389, 370/465
• US Class Current: 709/233
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G09G 2350/00 Solving problems of bandwidth in display systems
• CPC Class Codes: G09G 2370/025 LAN communication management
• CPC Class Codes: G09G 2370/10 Use of a protocol of communication by packets in interfaces along the display data pipeline
• CPC Class Codes: H04L 47/30 in combination with information about buffer occupancy at either end or at transit nodes
• CPC Class Codes: H04N 19/152 by measuring the fullness of the transmission buffer
• CPC Class Codes: H04N 21/4113 PC
• CPC Class Codes: H04N 21/43632 involving a wired protocol, e.g. IEEE 1394 high-speed IEEE 1394 serial bus H04L12/40052
• CPC Class Codes: H04N 21/64707 for transferring content from a first network to a second network, e.g. between IP and wireless

Abstract

A method for controlling data transmission between a computer and a video client via an interface, the method comprising: the computer polling the interface a first time to determine the size of the buffer on the interface; receiving a first buffer size value from the interface; sending a plurality of frames of video and audio data to the buffer on the interface such that a delay period exists between the sending of each frame; the computer polling the interface a second time to determine buffer size after the frames are sent to the interface; receiving a second buffer size value from the interface; and modifying the amount of time between the transmission of frames.

Claims

1. In a system having a computer, a video client, and an interface coupled between the computer and the video client, the interface having a buffer adapted to store data frames received from the computer to be sent to the video client, a method of performing data transmission flow control, the method comprising:

determining a first buffer fill amount of the buffer;

sending a plurality of frames of data in an isochronous manner to the buffer via the interface such that a number of cycles exists between the sending of at least a portion of said plurality of frames;

determining a second buffer fill amount after the plurality of frames are sent to the buffer on the interface; and

changing the number of cycles between transmission of frames from the computer to the buffer on the interface based in part on the relative size of the second buffer fill amount as compared with the first buffer fill amount.

2. The method of claim 1, wherein the size of at least one frame of said plurality of frames varies to accommodate various video formats.

3. The method of claim 1, wherein each frame of said plurality of frames comprises a NTSC format frame.

4. The method of claim 1, wherein each frame of said plurality of frames comprises a PAL format frame.

5. The method of claim 1, wherein a video portion and an audio portion of each frame of said plurality of frames are of a predetermined size.

6. The method of claim 5, wherein any empty portions of the video portion and the audio portion of each frame in said plurality of frames are filled with zeros.

7. The method of claim 1, wherein said number of cycles existing between the sending of at least a portion of said plurality of frames is based in part on the first buffer fill amount.

8. The method of claim 1, wherein said number of cycles existing between the sending of at least a portion of said plurality of frames is based in part on the difference between the first buffer fill amount and the second buffer fill amount.

9. A computer readable apparatus that performs data transmission flow control, said computer readable apparatus having a storage medium containing instructions which, when executed by a computer:

polls an interface buffer in communication with the computer to determine a first buffer fill amount, the buffer storing a number of data frames received from the computer, the buffer having a fill amount that varies with the number of data frames contained in the buffer;

sends the number of data frames in an isochronous manner to the buffer of the interface such that a number of cycles exists between the sending of at least some of the frames;

polls the interface to determine a second buffer fill amount, said polling to determine said second buffer fill amount occurring after the number of data frames are sent to the buffer; and

modulates the number of cycles between frames in a subsequent transmission from the computer to the buffer based at least in part on the relative size of the second buffer fill amount as compared with the first buffer fill amount.

10. The computer readable apparatus of claim 9, wherein a video portion and an audio portion of each frame of said plurality of frames are of a predetermined size.

11. The computer readable apparatus of claim 10, wherein said modulation comprises insertion of one or more additional cycles between transmission of at least two sets of sequential frames.

12. The computer readable apparatus of claim 9, said modulation is based at least in part on the relative size of the second buffer fill amount as compared with the first buffer fill amount, and maintains said buffer level at approximately an optimal level.

13. The computer readable apparatus of claim 9, wherein said modulation is based at least in part on the relative size of the second buffer fill amount as compared with the first buffer fill amount, and prevents an overflow condition and an underflow condition within said buffer.

14. The computer readable apparatus of claim 9, wherein said interface is in data communication with said computer via a standardized bus utilizing a serialized protocol.

15. The computer readable apparatus of claim 9, wherein the size of each frame of said plurality of frames varies to accommodate various video formats.

16. In a system having a computer, a video client, and an interface coupled between the computer and video client that facilitates data transmission between the computer and the video client, the interface connected to the computer via a standardized bus, the interface having a buffer for storing data frames received from the computer to be sent to the video client, an apparatus for performing data transmission flow control, the apparatus comprising:

a first apparatus adapted to determine a first buffer fill amount of the buffer on the interface at a first time;

a second apparatus adapted to send a plurality of frames of data in an isochronous manner to the buffer on the interface such that a delay period exists between the sending of at least a portion of the plurality of frames;

a third apparatus adapted to determine a second buffer fill amount after the plurality of frames are sent to the buffer on the interface; and

where if the second buffer fill amount value is smaller than an optimal buffer fill amount value, and smaller than the first buffer fill amount value, said second apparatus decreases the delay period, and if the second buffer fill amount value is larger than the optimal buffer fill amount value, and larger than the first buffer fill amount value, said second apparatus increases the delay period.

17. The apparatus for performing data transmission flow control of claim 16, wherein each frame of said plurality of frames comprises a NTSC format frame.

18. The apparatus for performing data transmission flow control of claim 16, wherein the size of at least several frames of said plurality of frames varies to accommodate various video formats.

19. The apparatus for performing data transmission flow control of claim 18, wherein said interface receives signals from said video client and converts said signals into a second plurality of frames, and sends one frame from said second plurality of frames to said computer at each vertical blanking interval.

20. A device adapted for performing data transmission flow control, comprising:

a first apparatus adapted to determine a first buffer fill amount of a buffer on an interface between a computer and a video client;

a second apparatus adapted to send a plurality of frames of data in an isochronous manner to the buffer on the interface such that a delay period exists between the sending of each frame;

a third apparatus adapted to determine a second buffer fill amount of the buffer on the interface after the frames are sent to the buffer on the interface; and

a fourth apparatus adapted to increase or decrease the delay period by a multiple of a fixed interval between the sending of frames to the buffer on the interface based at least in part on the relative size of the second buffer fill amount as compared with the first buffer fill amount.

21. The device of claim 20, wherein the size of at least one frame of said plurality of frames varies to accommodate various video formats.

22. The device of claim 20, wherein each frame of said plurality of frames comprises an NTSC format frame.

23. The device of claim 20, wherein each frame of said plurality of frames comprises a PAL format frame.

24. The device of claim 20, wherein a video portion and an audio portion of each frame of said plurality of frames are of a predetermined size.

25. The device of claim 24, wherein any empty portions of the video portion and the audio portion of each frame in said plurality of frames are filled with zeros.

26. The device of claim 20, wherein the interface is in data communication with the computer via a bus utilizing a serialized protocol.

27. A device adapted for performing data transmission flow control, comprising:

a computer readable storage medium containing instructions which, when executed by a computer;

determines a first buffer fill amount of a buffer on an interface between a computer and a video client, wherein the buffer on the interface facilitates data transmission between the computer and the video client;

sends a plurality of frames of data in an isochronous manner to the buffer on the interface such that a delay period exists between the sending of at least some of the plurality of frames;

determines a second buffer fill amount of the buffer on the interface after the plurality of frames are sent to the buffer on the interface; and

modulates the delay period by a multiple of a fixed interval between transmission of individual ones of said frames from the computer to the buffer on the interface based in part on the relative size of the second buffer fill amount as compared with the first buffer fill amount.

28. The device of claim 27, wherein the size of at least one frame of said plurality of frames varies to accommodate various video formats.

29. The device of claim 27, wherein each frame of said plurality of frames comprises an NTSC format frame.

30. The device of claim 27, wherein each frame of said plurality of frames comprises a PAL format frame.

31. The device of claim 27, wherein a video portion and an audio portion of each frame of said plurality of frames are of a predetermined size.

32. The device of claim 27, wherein any empty portions of a video portion and an audio portion of each frame in said plurality of frames are filled with zeros.

33. The device of claim 27, wherein the interface is in data communication with the computer via a bus utilizing a serialized bus protocol.

34. A method of performing data transmission flow control, comprising:

determining a first buffer fill amount value of a buffer on an interface between a computer and a video client, wherein the buffer on the interface facilitates data transmission between the computer and the video client;

sending data in an isochronous manner to the buffer on the interface, said interface converting said data into a plurality of frames having a delay period between at least some of the frames;

determining a second buffer fill amount value of the buffer on the interface; and

if the second buffer fill amount value is smaller than an optimal buffer fill amount value, and smaller than the first buffer fill amount value, decreasing the delay period, and if the second buffer fill amount value is larger than the optimal buffer fill amount value, and larger than the first buffer fill amount value, increasing the delay period.

35. The method of claim 34, wherein the size of at least one frame of said plurality of frames varies to accommodate various video formats.

36. The method of claim 34, wherein each frame of said plurality of frames comprises an NTSC format frame.

37. The method of claim 34, wherein each frame of said plurality of frames comprises a PAL format frame.

38. The method of claim 34, wherein a video portion and an audio portion of each frame of said plurality of frames are of a predetermined size.

39. The method of claim 34, wherein any empty portions of a video portion or an audio portion of each frame in said plurality of frames are filled with zeros.

40. The method of claim 34, wherein the interface is in data communication with the computer via a bus utilizing a serialized bus protocol.