Method and apparatus for real-time synchronization of voice communications

• US 8,699,383 B2
• Filed: 10/17/2008
• Issued: 04/15/2014
• Est. Priority Date: 10/19/2007
• Status: Expired due to Fees

• Current Assignee: Voxer IP LLC (Voxer, Inc.)
• Original Assignee: Voxer IP LLC (Voxer, Inc.)
• Inventors: Katis, Thomas E., Panttaja, James T., Panttaja, Mary G., Ranney, Matthew J.
• Primary Examiner(s): SEFCHECK, GREGORY B
• Application Number:
  US12/253,816
• Publication Number:
  US 20090103689A1
• Time in Patent Office: 2,006 Days
• Field of Search: None
• US Class Current: 370/260
• CPC Class Codes: H04L 12/1831 Tracking arrangements for later retrieval, e.g. recording contents, participants activities or behavior, network status
• CPC Class Codes: H04L 51/04 Real-time or near real-time messaging, e.g. instant messaging [IM]
• CPC Class Codes: H04L 65/1083 In-session procedures
• CPC Class Codes: H04L 65/1094 Inter-user-equipment sessions transfer or sharing
• CPC Class Codes: H04L 65/4015 where at least one of the additional parallel sessions is real time or time sensitive, e.g. white board sharing, collaboration or spawning of a subconference
• CPC Class Codes: H04L 65/4053 without floor control
• CPC Class Codes: H04L 65/752 adapting media to network capabilities
• CPC Class Codes: H04L 65/764 at the destination reformatting of additional data in video clients H04N21/4355

Abstract

A method and system for synchronizing in real-time the voice media of a conversation conducted over a network between a first communication device and a second communication. The method includes at each of the first and second communication devices progressively storing in first and second storage elements and transmitting the voice media created using the first and second communication devices to the other communication device respectively. Both the first and second communication devices store in the first and second storage elements the progressively received media from the other device respectively. A mechanism to continually review, ascertain and request the media stored in the first storage element, but not the second storage element, and vice-versa is provided to ensure that the two storage elements contain the same voice media. As a result, the first and second storage elements each maintain real-time synchronized copies of the voice media of the conversation respectively.

Claims

1. An application embedded in a non-transitory computer readable medium and intended to run on a first client communication device, the application comprising:

a storage module configured to progressively store in a first persistent storage element on the first client communication device voice media pertaining to a conversation and created using the first communication device as the voice media is created while using the first client communication device;

a transmission module configured to progressively transmit the voice media to a second client communication device over a network as the voice media is created and stored in the first storage element;

a network receive module configure to progressively receive over the network voice media received from the second client communication device and to cooperate with the storage module to progressively store in the first persistent storage element the received voice media pertaining to the conversation from the second client communication device;

a render module configured to selectively render the voice media received over the network (a) progressively as the voice media is received over the network in a real-time mode and (b) progressively by retrieving and rendering the received voice media out of the first persistent storage element in a time-shifted mode; and

a synchronization module configured to synchronize the transmitted and received voice media of the conversation between the first client communication device and the second client communication device, the synchronization effort occurring in situations where there is voice media loss on the network and the synchronization effort including retransmission efforts for voice media continuing beyond a time period where the voice media can be rendered in real-time to ensure that the first and second client communication devices each obtain and persistently store complete copies of the transmitted and received voice media of the conversation for rendering in the time-shifted mode.

2. The application of claim 1, wherein the transmission module is further configured to:

(i) define a transmission loop; and

(ii) transmit the voice media available during the transmission loop.

3. The application of claim 2, wherein the transmission module is further configured (iii) define successive transmission loops and perform (ii) for each successive transmission loop.

4. The application of claim 1, wherein the synchronization module ascertains the voice media transmitted by the second client communication device but not received at the first client communication device, the synchronizing element including a data quality store module that is configured to note missing, corrupted or reduced bit rate versions of the voice media transmitted by the second client communication device to the first communication device.

5. The application of claim 4, wherein the synchronization module further comprises a request for retransmission module which generates requests for retransmission of the voice media noted by the data quality store module.

6. The application of claim 1, wherein the synchronization module of the first client communication device ascertains the voice media transmitted by the second client communication device but not received at the first client communication device and retransmission requests are used to obtain complete copies of the voice media at the first client communication device and wherein complete copies of the voice media further comprises complete copies of the voice media as originally encoded when created.

7. The application of claim 1, wherein the conversation, besides voice media, includes one or more of the following media types:

video, photos, text, sensor data, position or GPS information, radio signals, or any combination thereof.

8. The application of claim 1, wherein the first client communication device intended to run the application comprises one of the following:

land-line phone, wireless phone, cellular phone, satellite phone, computer, radio, server, satellite radio, tactical radio or tactical phone.

9. An application embedded in a non-transitory computer readable medium and intended to run on a first client communication device, the application comprising:

a storage module configured to progressively store in a first persistent storage element on the first client communication device voice media pertaining to a conversation and created using the first communication device as the voice media is created while using the first client communication device;

a transmission module configured to progressively transmit the voice media to a second client communication device over a network as the voice media is created and stored in the first storage element;

a network receive module configure to progressively receive over the network voice media received from the second client communication device and to cooperate with the storage module to progressively store in the first persistent storage element the received voice media pertaining to the conversation from the second client communication device;

a render module configured to selectively render the voice media received over the network (a) progressively as the voice media is received over the network in a real-time mode and (b) progressively by retrieving and rendering the received voice media out of the first persistent storage element in a time-shifted mode; and

a synchronization module configured to synchronize the transmitted and received voice media of the conversation between the first client communication device and the second client communication device, the synchronization effort occurring in situations where there is voice media loss on the network and continuing beyond a time period where the voice media can be rendered in real-time so ensure that the first and second client communication devices each obtain and persistently store complete copies of the transmitted and received voice media of the conversation for rendering in the time-shifted mode wherein the synchronization module is further configured to;

i) receive a notification from the second client device reporting on a status of all locally created voice media originating at the second client device pertaining to the conversation;

(ii) ascertain the voice media transmitted by the second client communication device but not received at the first client communication device; and

(iii) request retransmission of the ascertained voice media.

10. The application of claim 9, wherein the synchronization module is further configured to:

i) provide a notification to the second client device reporting on a status of all locally created voice media originating at the first client device pertaining to the conversation;

ii) receive retransmission requests from the second client communication device for voice media of the conversation that was transmitted by the first client communication device but not received at the second client communication device; and

iii) cooperate with the transmission module to retransmit the voice media of the conversation that was transmitted by the first client communication device but not received at the second client communication device.

11. An application embedded in a non-transitory computer readable medium and intended to run on a first client communication device, the application comprising:

a storage module configured to progressively store in a first persistent storage element on the first client communication device voice media pertaining to a conversation and created using the first communication device as the voice media is created while using the first client communication device;

a transmission module configured to progressively transmit the voice media to a second client communication device over a network as the voice media is created and stored in the first storage element;

a network receive module configure to progressively receive over the network voice media received from the second client communication device and to cooperate with the storage module to progressively store in the first persistent storage element the received voice media pertaining to the conversation from the second client communication device;

a render module configured to selectively render the voice media received over the network (a) progressively as the voice media is received over the network in a real-time mode and (b) progressively by retrieving and rendering the received voice media out of the first persistent storage element in a time-shifted mode; and

a synchronization module configured to synchronize the transmitted and received voice media of the conversation between the first client communication device and the second client communication device, the synchronization effort occurring in situations where there is voice media loss on the network and continuing beyond a time period where the voice media can be rendered in real-time so ensure that the first and second client communication devices each obtain and persistently store complete copies of the transmitted and received voice media of the conversation for rendering in the time-shifted mode;

wherein the transmission module is further configured to;

(i) define a transmission loop;

(ii) transmit the voice media available during the transmission loop; and

(iii) transmit a reduced bit rate representation of the voice media during the transmission loop if the usable bit rate on the network is not sufficient for transmitting the available voice media as originally encoded when created.

12. The application of claim 11, wherein the transmission module is further configured to transmit the reduced bit rate representation of the voice media by one of the following:

(iv) using fewer bits per unit of time when packetizing the reduced bit rate representation of the available voice media relative to packetizing the available media as originally encoded;

(v) increasing the packetization interval when transmitting the available voice media;

(vi) adjusting the rate at which packets containing the available voice media are transmitted;

or

(vii) any combination of (iv) through (vi).

13. The application of claim 11, wherein the transmission module is further configured to transmit the reduced bit rate representation of the voice media by using one of the following:

(a) one or more different codec settings;

(b) one or more different codecs;

(c) a compression algorithm;

or

(d) any combination of (a) through (c).

14. The application of claim 11, further comprising performing (iii) for each loop when the usable bit rate on the network is not sufficient for transmitting the available voice media as originally encoded when created.

15. The application of claim 11, wherein the transmission module, in cooperation with the synchronization module, is further configure to retransmit the voice media transmitted using the reduced bit rate representation so that the second client communication device eventually receives the complete copy of the voice media.

16. A first client communication device, comprising:

a storage element configured to progressively store in a first persistent storage element voice media pertaining to a conversation and created using the first client communication device as the voice media is created;

a transmission element configured to progressively transmit the voice media to a second client communication device over a network as the voice media is created and stored in the first storage element;

a network receive element configure to progressively receive over the network voice media received from the second client communication device and to cooperate with the storage element to progressively store in the first persistent storage element the received voice media pertaining to the conversation from the second client communication device;

a render element configured to selectively render the voice media received over the network (a) progressively as the voice media is received over the network in a real-time mode and (b) progressively by retrieving and rendering the received voice media out of the first persistent storage element in a time-shifted mode; and

a synchronization element configured to synchronize the transmitted and received voice media of the conversation between the first client communication device and the second client communication device, the synchronization effort occurring in situations where there is voice media loss on the network and the synchronization effort including retransmission efforts for voice media continuing beyond a time period where the voice media can be rendered in real-time to ensure that the first and second client communication devices each obtain and persistently store complete copies of the transmitted and received voice media of the conversation for rendering in the time-shifted mode.

17. The device of claim 16, wherein the transmission element is further configured to:

(i) define a transmission loop; and

(ii) transmit the voice media available during the transmission loop.

18. The device of claim 17, wherein the transmission element is further configured to (iii) define successive transmission loops and perform (ii) for each successive transmission loop.

19. The device of claim 16, wherein the synchronization element ascertains the voice media transmitted by the second client communication device but not received at the first client communication device, the synchronizing element including a data quality store element that is configured to note missing, corrupted or reduced bit rate versions of the voice media transmitted by the second client communication device to the first client communication device.

20. The device of claim 19, wherein the synchronization element further comprises a request for retransmission element which generates requests for retransmission of the voice media noted by the data quality store element.

21. The device of claim 16, wherein the synchronization element of the first client communication device ascertains the voice media transmitted by the second client communication device but not received at the first client communication device and retransmission requests are used to obtain complete copies of the voice media at the first client communication device, wherein the voice media further comprises complete copies of the voice media as originally encoded when created.

22. The device of claim 16, wherein the conversation, besides voice media, includes one or more of the following media types:

video, photos, text, sensor data, position or GPS information, radio signals, or any combination thereof.

23. The device of claim 16, wherein the first client communication device comprises one of the following:

land-line phone, wireless phone, cellular phone, satellite phone, computer, radio, server, satellite radio, tactical radio or tactical phone.

24. A first client communication device, comprising:

a storage element configured to progressively store in a first persistent storage element voice media pertaining to a conversation and created using the first client communication device as the voice media is created;

a transmission element configured to progressively transmit the voice media to a second client communication device over a network as the voice media is created and stored in the first storage element;

a network receive element configure to progressively receive over the network voice media received from the second client communication device and to cooperate with the storage element to progressively store in the first persistent storage element the received voice media pertaining to the conversation from the second client communication device;

a render element configured to selectively render the voice media received over the network (a) progressively as the voice media is received over the network in a real-time mode and (b) progressively by retrieving and rendering the received voice media out of the first persistent storage element in a time-shifted mode; and

a synchronization element configured to synchronize the transmitted and received voice media of the conversation between the first client communication device and the second client communication device, the synchronization effort occurring in situations where there is voice media loss on the network and continuing beyond a time period where the voice media can be rendered in real-time so that the first and second client communication devices each obtain and persistently store complete copies of the transmitted and received voice media of the conversation for rendering in the time-shifted mode;

wherein the synchronization element is further configured to;

i) receive notification from the second client device of the status of all locally created voice media originating at the second client device pertaining to the conversation;

ii) ascertain the voice media transmitted by the second client communication device but not received at the first client communication device; and

iii) request retransmission of the ascertained voice media.

25. The device of claim 24, wherein the synchronization element is further configured to:

(i) provide notification to the second client device of the status of all locally created voice media originating at the first client device pertaining to the conversation;

(ii) receive retransmission requests from the second client communication device for voice media of the conversation that was transmitted by the first client communication device but not received at the second client communication device; and

(iii) cooperate with the transmission element to retransmit the voice media of the conversation that was transmitted by the first client communication device but not received at the second client communication device.

26. A first client communication device, comprising:

a storage element configured to progressively store in a first persistent storage element voice media pertaining to a conversation and created using the first client communication device as the voice media is created;

a transmission element configured to progressively transmit the voice media to a second client communication device over a network as the voice media is created and stored in the first storage element;

a network receive element configure to progressively receive over the network voice media received from the second client communication device and to cooperate with the storage element to progressively store in the first persistent storage element the received voice media pertaining to the conversation from the second client communication device;

a render element configured to selectively render the voice media received over the network (a) progressively as the voice media is received over the network in a real-time mode and (b) progressively by retrieving and rendering the received voice media out of the first persistent storage element in a time-shifted mode; and

a synchronization element configured to synchronize the transmitted and received voice media of the conversation between the first client communication device and the second client communication device, the synchronization effort occurring in situations where there is voice media loss on the network and continuing beyond a time period where the voice media can be rendered in real-time so that the first and second client communication devices each obtain and persistently store complete copies of the transmitted and received voice media of the conversation for rendering in the time-shifted mode;

wherein the transmission element is further configured to;

(i) define a transmission loop; and

(ii) transmit the voice media available during the transmission loop; and

(iii) transmit a reduced bit rate representation of the voice media during the transmission loop if the usable bit rate on the network is not sufficient for transmitting the available voice media as originally encoded when created.

27. The device of claim 26, wherein the transmission element is further configured to transmit the reduced bit rate representation of the voice media by one of the following:

(iv) using fewer bits per unit of time when packetizing the reduced bit rate representation of the available voice media relative to packetizing the available media as originally encoded;

(v) increasing the packetization interval when transmitting the available voice media;

(vi) adjusting the rate at which packets containing the available voice media are transmitted;

or

(vii) any combination of (iv) through (vi).

28. The device of claim 26, wherein the transmission element is further configured to transmit the reduced bit rate representation of the voice media by using one of the following:

(a) one or more different codec settings;

(b) one or more different codecs;

(c) a compression algorithm;

or

(d) any combination of (a) through (c).

29. The device of claim 26, further comprising performing (iii) for each loop when the usable bit rate on the network is not sufficient for transmitting the available voice media as originally encoded when created.

30. The device of claim 26, wherein the transmission element, in cooperation with the synchronization module, is further configure to retransmit the voice media transmitted using the reduced bit rate representation so that the second communication device eventually receives the complete copy of the voice media.