Late frame recovery method

• US 7,177,278 B2
• Filed: 02/25/2002
• Issued: 02/13/2007
• Est. Priority Date: 12/09/1999
• Status: Expired due to Fees

• Current Assignee: Avago Technologies General IP PTE Limited (Broadcom, Inc.)
• Original Assignee: Broadcom Corporation (Broadcom, Inc.)
• Inventors: LeBlanc, Wilfrid
• Primary Examiner(s): Pham; Chi
• Assistant Examiner(s): TRAN, PHUC H
• Application Number:
  US10/082,749
• Publication Number:
  US 20020080779A1
• Time in Patent Office: 1,814 Days
• Field of Search: 370235-236, 370/320, 370/335, 370/342, 370/441, 370412-429
• US Class Current: 370/235
• CPC Class Codes: H04B 3/23 using a replica of transmitted signal in the time domain, e.g. echo cancellers
• CPC Class Codes: H04J 3/0632 Synchronisation of packets and cells, e.g. transmission of voice via a packet network, circuit emulation service [CES] queuing arrangements in packet switching elements H04L49/90; synchronising systems for the synchronous transmission of a pulse code modulated video signal with one or more other pulse code modulated signals H04N7/56
• CPC Class Codes: H04L 12/2801 Broadband local area networks
• CPC Class Codes: H04L 12/66 Arrangements for connecting between networks having differing types of switching systems, e.g. gateways
• CPC Class Codes: H04L 2001/0093 Point-to-multipoint
• CPC Class Codes: H04L 2012/5671 Support of voice
• CPC Class Codes: H04L 47/10 Flow control; Congestion control
• CPC Class Codes: H04L 47/11 Identifying congestion
• CPC Class Codes: H04L 47/263 Rate modification at the source after receiving feedback
• CPC Class Codes: H04L 47/28 in relation to timing considerations
• CPC Class Codes: H04L 65/1026 at the edge
• CPC Class Codes: H04L 65/1036 at the edge
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 65/752 adapting media to network capabilities
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04L 67/61 taking into account QoS or priority requirements
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]
• CPC Class Codes: H04L 69/40 for recovering from a failure of a protocol instance or entity, e.g. service redundancy protocols, protocol state redundancy or protocol service redirection management of faults, events, alarms or notifications in data switching networks H04L41/06
• CPC Class Codes: H04L 9/40 Network security protocols
• CPC Class Codes: H04M 2201/40 : using speech recognition speech recognition per se G10L15/00
• CPC Class Codes: H04M 2203/2027 : Live party detection
• CPC Class Codes: H04M 2203/2066 : Call type detection of indication, e.g. voice or fax, mobile of fixed, PSTN or IP
• CPC Class Codes: H04M 3/002 : Applications of echo suppressors or cancellers in telephonic connections in two-way loud-speaking telephone systems H04M9/08, echo suppressors or cancellers per se H04B3/20
• CPC Class Codes: H04M 3/22 : Arrangements for supervision, monitoring or testing
• CPC Class Codes: H04M 3/2209 : for lines also used for data transmission
• CPC Class Codes: H04M 3/2272 : Subscriber line supervision circuits, e.g. call detection circuits
• CPC Class Codes: H04M 7/006 : Networks other than PSTN/ISDN providing telephone service, e.g. Voice over Internet Protocol (VoIP) , including next generation networks with a packet-switched transport layer H04L65/00 takes precedence; aspects not specific to the type of network H04M3/00; special services in those networks H04M3/42
• CPC Class Codes: H04M 7/0069 : comprising a residential gateway, e.g. those which provide an adapter for POTS or ISDN terminals
• CPC Class Codes: H04M 7/0072 : Speech codec negotiation H04L65/1069 takes precedence
• CPC Class Codes: H04Q 1/45 : using multi-frequency signalling H04Q1/46 takes precedence

Abstract

Method of processing a transmitted encoded media data stream is received. If a data element arrives prior to, or at, a predetermined playout deadline, the data element is decoded, the media represented by the decoded data element is played, and the data element is provided to a decoder state machine to update a decoder state. If a data element arrives after the predetermined playout deadline, the data element is provided to the decoder state machine to update the decoder state. In one embodiment, if the specified data element fails to arrive by the playout deadline, a subsequently received data element is saved in memory. Then, if the specified data element arrives after the predetermined playout deadline, the specified data element and the saved, subsequently received, data element are provided to the decoder state machine to update the decoder state.

Claims

1. A method of processing communication signals, comprising steps of:

(a) receiving a transmitted encoded media data stream comprising a plurality of data elements, each data element representing a portion of a transmitted media session;

(b) if a data element arrives prior to, or at, a predetermined playout deadline, performing steps of;

(b)(i) decoding the data element;

(b)(ii) playing the media represented by the decoded data element; and

(b)(iii) providing the data element to a decoder state machine to update a decoder state, wherein the decoder state machine is operable to estimate characteristics of unreceived data elements based on characteristics of already-received data elements; and

(c) if a data element arrives after the predetermined playout deadline, providing the data element to the decoder state machine to update the decoder state.

2. The method of claim 1 wherein if a data element arrives after the predetermined playout deadline, pursuant to step (c), the data element is not decoded and the media represented by the decoded data element is not played.

3. The method of claim 1 wherein the media data stream is an encoded audio data stream comprising a plurality of audio data elements, each representing a portion of a transmitted audio session.

4. The method of claim 3 wherein the media data stream is an encoded voice data stream comprising a plurality of voice data elements, each representing a portion of a transmitted voice session.

5. The method of claim 1 wherein the media data stream is an encoded video data stream comprising a plurality of video data elements, each representing a portion of a transmitted video session.

6. The method of claim 1 wherein the playout deadline comprises an end of a predetermined interval that starts at an expected arrival time of the data element.

7. The method of claim 1 wherein the playout deadline comprises an end of a predetermined interval that starts when a previous data element in the data stream is received.

8. The method of claim 1 wherein decoding step (b)(i) comprises releasing the data element to a decoder that decodes the data element and wherein the playout deadline comprises an end of a predetermined interval that starts when a previous data element in the data stream is released to the decoder.

9. The method of claim 1 wherein the playout deadline comprises an end of a predetermined interval that starts when a first data element in the data stream is received.

10. The method of claim 1 wherein the data elements are packets of encoded data.

11. The method of claim 1 wherein the data elements are frames of encoded data.

12. The method of claim 1 wherein the data elements are cells of encoded data.

13. A method of processing communication signals, comprising:

(a) receiving a transmitted encoded media data stream comprising a plurality of data elements, each data element representing a portion of a transmitted media session;

(b) if a data element arrives prior to, or at, a predetermined playout deadline, performing steps of;

(b)(i) decoding the data element;

(b)(ii) playing the media represented by the decoded data element; and

(b)(iii) providing the data element to a decoder state machine to update a decoder state, wherein the decoder state machine is used to assist in determining characteristics of received data elements based on characteristics of previously-received data elements; and

(c) if a data element arrives after the predetermined playout deadline, providing the data element to the decoder state machine to update the decoder state.

14. A method of processing communication signals, comprising:

(a) receiving a transmitted encoded media data stream comprising a plurality of data elements, each data element representing a portion of a transmitted media session;

(b) if a data element arrives prior to, or at, a predetermined playout deadline, performing steps of;

(b)(i) decoding the data element;

(b)(ii) playing the media represented by the decoded data element; and

(b)(iii) providing the data element to a decoder state machine to update a decoder state;

(c) if a data element arrives after the predetermined playout deadline, providing the data element to the decoder state machine to update the decoder state; and

(d) if a data element has not arrived by the playout deadline, performing steps of;

(d)(i) estimating the characteristics of the non-arriving data element;

(d)(ii) playing the media represented by the estimated non-arriving data element; and

(d)(iii) providing the estimated non-arriving data element to a decoder state machine to update a decoder state.

15. The method of claim 14 wherein estimating step (d)(i) comprises estimating the characteristics of the non-arriving data element based upon characteristics of already-received data elements.

16. The method of claim 14 wherein step (c) comprises:

if a data element whose characteristics have been estimated pursuant to step (d) arrives after the predetermined playout deadline, providing the data element to the decoder state machine to update the decoder state.

17. A method of processing communication signals, comprising steps of:

(a) receiving a transmitted encoded media data stream comprising a plurality of data elements, each data element representing a portion of a transmitted media session;

(b) if a specified data element arrives prior to, or at, a predetermined playout deadline, performing steps of;

(b)(i) decoding the data element;

(b)(ii) playing the media represented by the decoded data element; and

(b)(iii) providing the data element to a decoder state machine to update a decoder state, wherein the decoder state machine is operable to estimate characteristics of lost data elements based on characteristics of already-received data elements; and

(c) if the specified data element fails to arrive prior to, or at, the playout deadline, performing steps of;

(c)(i) saving a subsequently received data element in memory;

(c)(ii) if the specified data element arrives after the predetermined playout deadline, providing the specified data element and the saved, subsequently received, data element to the decoder state machine to update the decoder state.

18. The method of claim 17 wherein saving step (c)(i) comprises saving a plurality of subsequently received data elements in memory and wherein step (c)(ii) comprises:

if the specified data element arrives after the predetermined playout deadline, providing the specified data element and the saved, subsequently received, data elements to the decoder state machine to update the decoder state.

19. The method of claim 18 wherein saving step (c)(i) comprises saving all data elements that are received subsequent to the playout deadline but before the arrival of the specified data element and wherein step (c)(ii) comprises:

if the specified data element arrives after the predetermined playout deadline, providing the specified data element and the saved, subsequently received, data elements to the decoder state machine to update the decoder state.

20. The method of claim 17 wherein if the specified data element arrives after the predetermined playout deadline, pursuant to step (c)(ii), the specified data element is not decoded and the media represented by the decoded data element is not played.

21. The method of claim 17 wherein the media data stream is an encoded audio data stream comprising a plurality of audio data elements, each representing a portion of a transmitted audio session.

22. The method of claim 21 wherein the media data stream is an encoded voice data stream comprising a plurality of voice data elements, each representing a portion of a transmitted voice session.

23. The method of claim 17 wherein the media data stream is an encoded video data stream comprising a plurality of video data elements, each representing a portion of a transmitted video session.

24. The method of claim 17 wherein the playout deadline comprises an end of a predetermined interval that starts at an expected arrival time of the specified data element.

25. The method of claim 17 wherein the playout deadline comprises an end of a predetermined interval that starts when a previous data element in the data stream is received.

26. The method of claim 17 wherein decoding step (b)(i) comprises releasing the data element to a decoder that decodes the data element and wherein the playout deadline comprises an end of a predetermined interval that starts when a previous data element in the data stream is released to the decoder.

27. The method of claim 17 wherein the playout deadline comprises an end of a predetermined interval that starts when a first data element in the data stream is received.

28. The method of claim 17 wherein the data elements are packets of encoded data.

29. The method of claim 17 wherein the data elements are frames of encoded data.

30. A method of processing communication signals, comprising:

(a) receiving a transmitted encoded media data stream comprising a plurality of data elements, each data element representing a portion of a transmitted media session;

(b) if a specified data element arrives prior to, or at, a predetermined playout deadline, performing steps of;

(b)(i) decoding the data element;

(b)(ii) playing the media represented by the decoded data element; and

(b)(iii) providing the data element to a decoder state machine to update a decoder state, wherein the decoder state machine is used to assist in determining characteristics of received data elements based on characteristics of previously-received data elements; and

(c) if the specified data element fails to arrive prior to, or at, the playout deadline, performing steps of;

(c)(i) saving a subsequently received data element in memory;

(c)(ii) if the specified data element arrives after the predetermined playout deadline, providing the specified data element and the saved, subsequently received, data element to the decoder state machine to update the decoder state.

31. A method of processing communication signals, comprising:

(a) receiving a transmitted encoded media data stream comprising a plurality of data elements, each data element representing a portion of a transmitted media session;

(b) if a specified data element arrives prior to, or at, a predetermined playout deadline, performing steps of;

(b)(i) decoding the data element;

(b)(ii) playing the media represented by the decoded data element; and

(b)(iii) providing the data element to a decoder state machine to update a decoder state; and

(c) if the specified data element fails to arrive prior to, or at, the playout deadline, performing steps of;

(c)(i) estimating the characteristics of the specified data element;

(c)(ii) playing the media represented by the estimate of the specified data element;

(c)(iii) providing the estimate of the specified data element to a decoder state machine to update a decoder state;

(c)(iv) saving a subsequently received data element in memory; and

(c)(v) if the specified data element arrives after the predetermined playout deadline, providing the specified data element and the saved, subsequently received, data element to the decoder state machine to update the decoder state.

32. The method of claim 31 wherein estimating step (c)(i) comprises estimating the characteristics of the specified data element based upon characteristics of already-received data elements.

33. A communication system for processing a transmitted encoded media data stream comprising a plurality of data elements, each data element representing a portion of a transmitted media session, the communication system comprising:

a jitter buffer operable to receive the data stream and to store each data element until a playout deadline for the data element is reached, at which time the data element is provided to a decoder and to a decoder state machine, wherein if the jitter buffer receives a late data element after the playout deadline, the late data element is provided to the decoder state machine;

a decoder operable to decode media data elements provided by the jitter buffer;

a decoder state machine operable to maintain and update a decoder state based on data elements provided by the jitter buffer, the decoder state machine being operable to estimate characteristics of unreceived data elements based on characteristics of already-received data elements provided by the jitter buffer; and

an output element operable to receive decoded data elements from the decoder and to play the media represented by the decoded data elements.

34. The communication system of claim 33 wherein if the jitter buffer receives a data element after the playout deadline, the data element is not decoded and the media represented by the decoded data element is not played.

35. The communication system of claim 33 wherein the media data stream is an encoded audio data stream comprising a plurality of audio data elements, each representing a portion of a transmitted audio session.

36. The communication system of claim 33 wherein the media data stream is an encoded voice data stream comprising a plurality of voice data elements, each representing a portion of a transmitted voice session.

37. The communication system of claim 33 wherein the media data stream is an encoded video data stream comprising a plurality of video data elements, each representing a portion of a transmitted video session.

38. The communication system of claim 33 wherein the playout deadline for each data element comprises an end of a predetermined interval that starts at an expected arrival time of the data element.

39. The communication system of claim 33 wherein the playout deadline for each data element comprises an end of a predetermined interval that starts when a previous data element in the data stream is received by the jitter buffer.

40. The communication system of claim 33 wherein the playout deadline for each data element comprises an end of a predetermined interval that starts when the jitter buffer provides the previous data element in the data stream to the decoder.

41. The communication system of claim 33 wherein the playout deadline for each data element comprises an end of a predetermined interval that starts when the jitter buffer receives a first data element in the data stream.

42. The communication system of claim 33 wherein the data elements are packets of encoded data.

43. The communication system of claim 33 wherein the data elements are frames of encoded data.

44. A communication system for processing a transmitted encoded media data stream comprising a plurality of data elements, each data element representing a portion of a transmitted media session, the communication system comprising:

a jitter buffer operable to receive the data stream and to store each data element until a playout deadline for the data element is reached, at which time the data element is provided to a decoder and to a decoder state machine, wherein if the jitter buffer receives a late data element after the playout deadline, the late data element is provided to the decoder state machine;

a decoder operable to decode media data elements provided by the jitter buffer;

a decoder state machine operable to maintain and update a decoder state based on data elements provided by the jitter buffer, wherein the decoder state machine is operable to assist in determining characteristics of data elements received by the decoder based on characteristics of previously-received data elements provided by the jitter buffer; and

an output element operable to receive decoded data elements from the decoder and to play the media represented by the decoded data elements.

45. A communication system for processing a transmitted encoded media data stream comprising a plurality of data elements, each data element representing a portion of a transmitted media session, the communication system comprising:

a jitter buffer operable to receive the data stream and to store each data element until a playout deadline for the data element is reached, at which time the data element is provided to a decoder and to a decoder state machine, wherein if the jitter buffer receives a late data element after the playout deadline, the late data element is provided to the decoder state machine;

a decoder operable to decode media data elements provided by the jitter buffer;

a decoder state machine operable to maintain and update a decoder state based on data elements provided by the jitter buffer;

an output element operable to receive decoded data elements from the decoder and to play the media represented by the decoded data elements; and

a lost-data-element recovery engine, wherein if a given data element is not received by the jitter buffer by the playout deadline, the lost-data-element recovery engine estimates the characteristics of the unreceived data element and provides the estimated data element to the output element and to the decoder state machine to update the decoder state.

46. The communication system of claim 45 wherein the lost-data-element recovery engine estimates the characteristics of the unreceived data element based upon characteristics of data elements previously received by the jitter buffer.

47. The communication system of claim 45 wherein if a late data element is received by the jitter buffer after the playout deadline and after the lost-data-element recovery engine has estimated the data element, the jitter buffer provides the late data element to the decoder state machine to update the decoder state.