Method and apparatus for near real-time synchronization of voice communications

• US 8,250,181 B2
• Filed: 10/17/2008
• Issued: 08/21/2012
• Est. Priority Date: 10/19/2007
• Status: Expired due to Fees

• Current Assignee: Voxer IP LLC (Voxer, Inc.)
• Original Assignee: Voxer IP LLC (Voxer, Inc.)
• Inventors: Katis, Thomas E., Panttaja, James T., Panttaja, Mary G., Ranney, Matthew J.
• Primary Examiner(s): Burgess, Barbara
• Application Number:
  US12/253,820
• Publication Number:
  US 20090168759A1
• Time in Patent Office: 1,404 Days
• Field of Search: 709/203, 709/204, 709/205, 709/219, 709/231, 709/248
• US Class Current: 709/219
• CPC Class Codes: H04L 12/1831 Tracking arrangements for later retrieval, e.g. recording contents, participants activities or behavior, network status
• CPC Class Codes: H04L 51/04 Real-time or near real-time messaging, e.g. instant messaging [IM]
• CPC Class Codes: H04L 65/1083 In-session procedures
• CPC Class Codes: H04L 65/1094 Inter-user-equipment sessions transfer or sharing
• CPC Class Codes: H04L 65/4015 where at least one of the additional parallel sessions is real time or time sensitive, e.g. white board sharing, collaboration or spawning of a subconference
• CPC Class Codes: H04L 65/752 adapting media to network capabilities
• CPC Class Codes: H04L 65/764 at the destination reformatting of additional data in video clients H04N21/4355

Abstract

A method and system for synchronizing in real-time the voice media of a conversation conducted over a network between a first communication device and a second communication. The method includes at each of the first and second communication devices progressively storing in first and second storage elements and transmitting the voice media created using the first and second communication devices to the other communication device respectively. Both the first and second communication devices store in the first and second storage elements the progressively received media from the other device respectively. A mechanism to continually review, ascertain and request the media stored in the first storage element, but not the second storage element, and vice-versa is provided to ensure that the two storage elements contain the same voice media. As a result, the first and second storage elements each maintain real-time synchronized copies of the voice media of the conversation respectively.

Claims

1. A first communication device configured to synchronize the voice media of a conversation conducted over a network, the first communication device comprising:

a first storage element configured to progressively store voice media created using the first communication device as the voice media is created using the first communication device;

a transmitting element configured to progressively transmit the voice media created using the first communication device to a second communication device over the network;

a receiver element configured to progressively receive and store in the first storage element voice media pertaining to the conversation generated at the second communication device, stored in a second storage element associated with the second communication device, and transmitted from the second communication device to the first communication device over the network; and

a synchronization element configured to synchronize the voice media stored in the first storage element so that first storage element and the second storage element each contain stored complete copies of the voice media of the conversation, the synchronization element including;

a media ascertaining element configured to ascertain any voice media pertaining to the conversation transmitted by the second communication device but not received by the first communication device; and

a request generator configured to send to the second communication device one or more requests for retransmission of the ascertained voice media,

the receiver element further configured to store in the first storage element the media received at the first communication device from the second communication device in response to the requests for retransmission.

2. The first communication device of claim 1, wherein the first storage element and the receiver element cooperate so that the voice media transmitted by the second communication device is progressively stored as the voice media is progressively received over the network.

3. The first communication device of claim 1, wherein the first storage element and the transmitting element are further configured to progressively store and progressively transmit the voice media created using the first communication device as the media is created.

4. The first communication device of claim 1, wherein the transmitting element progressively transmits the voice media created using the first communication device to the second communication device over the network by:

(i) defining a transmission loop;

(ii) ascertaining the available voice media created using the first communication device during the defined transmission loop;

(iii) ascertaining the usable bit rate available on the network during the defined transmission loop;

(iv) determining if the ascertained usable bit rate available on the network during the transmission loop is sufficient to transmit a first bit rate representation of the ascertained available voice media as originally encoded; and

either;

(v) transmitting the first bit rate presentation of the available media if the ascertained available bit rate is sufficient;

or

(vi) generating a reduced bit rate version of the available voice media relative to the first bit rate representation when the ascertained available bit rate on the network is not sufficient to transmit the first bit rate representation of the available voice media, the reduced bit rate version of the available voice media enhancing the ability for the first communication device and the second communication device to engage in the conversation in near real-time; and

(vii) transmitting the reduced bit rate version of the available voice media when the ascertained usable bit rate available on the network during the transmission loop is not sufficient.

5. The first communication device of claim 4, wherein the transmitting element progressively transmits the voice media created using the first communication device to the second communication device over the network by:

defining successive transmission loops; and

performing (i) through (iii) and either (v) or (vi) and (vii) depending on if the ascertained usable bit rate on the network is sufficient or not for each defined successive loop respectively.

6. The first communication device of claim 4, wherein the transmitting element is further configured to generate the reduced bit rate version of the available voice media by using fewer bits per unit of time when packetizing the available voice media for transmission.

7. The first communication device of claim 4, wherein the transmitting element is further configured to generate the reduced bit rate version of the available voice media by using (a) one or more different codec settings, (b) one or more different codecs, (c) a compression algorithm;

or (d) any combination of (a) through (c) when packetizing the available voice media for transmission.

8. The first communication device of claim 4, wherein the transmitting element is further configured to generate the reduced bit rate version of the available voice media by increasing the packetization interval used to transmit the available voice media.

9. The first communication device of claim 4, wherein the transmitting element is further configured to generate the reduced bit rate version of the available voice media by adjusting the transmission rate at which packets containing the available voice media are transmitted.

10. The first communication device of claim 4, wherein the transmitting element is further configured to generate the reduced bit rate version of the available voice media by performing one or more of the following:

(i) using fewer bits per unit of time when packetizing the reduced bi rate representation of the available voice media relative to the first bit rate representation;

(ii) increasing the packetization interval when transmitting the available voice media;

(iii) adjusting the rate at which packets containing the available voice media are transmitted;

or

(iv) any combination of (i) through (iii).

11. The first communication device of claim 1, wherein the first communication device further comprises a media time-sensitivity ascertaining element configured to ascertain if the voice media created using the first communication device is either time-sensitive or not time-sensitive, the media time-sensitivity ascertaining element ascertaining the sensitivity of the voice media as either:

(i) time-sensitive if a user of the second communication device is progressively rendering the voice media upon receipt;

or

(ii) not time-sensitive if the second communication device is not rendering the voice media upon receipt.

12. The first communication device of claim 11, wherein the media time-sensitivity ascertaining element and the transmitting element cooperate to:

(iii) transmit the voice media ascertained as not time-sensitive to the second communication device using a first transmission technique, the first transmission technique relying on packets having a first packetization interval;

or

(iv) transmit the voice media ascertained as time-sensitive to the second communication device using a second transmission technique, the second transmission technique relying on packets having a second packetization interval, where the first packetization interval is larger than the second packetization interval.

13. The first communication device of claim 1, wherein the first communication device further comprises a data quality store configured to note any missing, corrupted or reduced bit rate versions of the voice media pertaining to the conversation transmitted by the second communication device to the first communication device.

14. The first communication device of claim 13, wherein the media ascertaining element is configured to ascertain any voice media pertaining to the conversation stored in the second storage element but not in the first storage element by continually reviewing the voice media noted in the data quality store.

15. The first communication device of claim 14, wherein the request generator is further configured to generate and send to the second communication device the one or more requests for retransmission of the ascertained voice media noted in the data quality store.

16. The first communication device of claim 15, wherein the data quality store is further configured to remove the notation for the voice media noted in the data quality store as the corresponding one or more requests for retransmission are satisfied.

17. The first communication device of claim 1, further comprising a rendering element associated with the first communication device and configured to optionally render the voice media of the conversation either (i) as the media is received or (ii) after an arbitrary delay by retrieving the voice media from the first storage element, wherein the arbitrary delay is defined by a user of the first communication device.

18. The first communication device of claim 1, wherein the conversation includes one or more of the following types of media besides voice:

video, photo, text, sensor data, position or GPS information, radio signals, or any combination thereof.

19. The first communication device of claim 1, wherein the communication device comprises one of the following:

a land-line phone, a wireless phone, a cellular phone, a satellite phone, a computer, a radio, a server, a satellite radio, a tactical radio or a tactical phone.

20. The first communication device of claim 1, wherein the voice media is indexed using one of the following indexing schemes:

(i) time-based indexing;

(ii) bit or byte based indexing;

(iii) hashing type indexing;

(iv) binary-tree type indexing;

(v) file directory based indexing;

or

(vi) any combination of (i) through (v).

21. An application embedded in a non-transitory computer readable medium and intended to run on a first communication device, the application configured to synchronize voice media of a conversation conducted over a network, comprising:

a first storage module configured to progressively store voice media created using the first communication device as the voice media is created using the first communication device;

a transmitting module configured to progressively transmit the voice media created using the first communication device to a second communication device over the network;

a receiver module configured to progressively receive and store in the first storage module voice media pertaining to the conversation generated at the second communication device, stored in a second storage module associated with the second communication device, and transmitted from the second communication device to the first communication device over the network; and

a synchronization module configured to synchronize the voice media stored in the first storage module so that first storage module and the second storage module each contain stored complete copies of the voice media of the conversation, the synchronization module including;

a media ascertaining module configured to ascertain any voice media pertaining to the conversation transmitted by the second communication device but not received by the first communication device; and

a request generator module configured to send to the second communication device one or more requests for retransmission of the ascertained voice media;

the receiver module further configured to store in the first storage module the media received at the first communication device from the second communication device in response to the requests for retransmission.

22. The application of claim 21, wherein the first storage module and the receiver module cooperate so that the voice media transmitted by the second communication device is progressively stored as the voice media is progressively received over the network.

23. The application of claim 21, wherein the first storage module and the transmitting module are further configured to progressively store and progressively transmit the voice media created using the first communication device as the media is created.

24. The application of claim 21, wherein the transmitting module progressively transmits the voice media created using the first communication device to the second communication device over the network by:

(i) defining a transmission loop;

(ii) ascertaining the available voice media created using the first communication device during the defined transmission loop;

(iii) ascertaining the usable bit rate available on the network during the defined transmission loop; and

(iv) determining if the ascertained usable bit rate available on the network during the transmission loop is sufficient to transmit a first bit rate representation of the ascertained available voice media as originally encoded.

25. The application of claim 24, wherein the transmitting module progressively transmits the voice media created using the first communication device to the second communication device over the network by:

(v) transmitting the first bit rate presentation of the available media if the ascertained available bit rate is sufficient.

26. The application of claim 25, wherein the transmitting module progressively transmits the voice media created using the first communication device to the second communication device over the network by:

(vi) generating a reduced bit rate version of the available voice media relative to the first bit rate representation when the ascertained available bit rate on the network is not sufficient to transmit the first bit rate representation of the available voice media, the reduced bit rate version of the available voice media enhancing the ability for the first communication device and the second communication device to engage in the conversation in near real-time; and

(vii) transmitting the reduced bit rate version of the available voice media when the ascertained usable bit rate available on the network during the transmission loop is not sufficient.

27. The application of claim 26, wherein the transmitting module progressively transmits the voice media created using the first communication device to the second communication device over the network further by:

defining successive transmission loops; and

performing (i) through (iii) and either (v) or (vi) and (vii) depending on if the ascertained usable bit rate on the network is sufficient or not for each defined successive loop respectively.

28. The application of claim 26, wherein the transmitting module is further configured to generate the reduced bit rate version of the available voice media by performing one or more of the following:

(i) using fewer bits per unit of time when packetizing the reduced bit rate representation of the available voice media relative to the first bit rate representation;

(ii) increasing the packetization interval when transmitting the available voice media;

(iii) adjusting the rate at which packets containing the available voice media are transmitted;

or

(iv) any combination of (i) through (iii).

29. The application of claim 21, further comprising a data quality store module configured to note any missing, corrupted or reduced bit rate versions of the voice media pertaining to the conversation transmitted by the second communication device to the first communication device.

30. The application of claim 23, wherein the media ascertaining module is configured to ascertain any voice media pertaining to the conversation stored in the second storage module but not in the first storage module by continually reviewing the voice media noted by the data quality store module.

31. The application of claim 30, wherein the request generator module is further configured to generate and send to the second communication device the one or more requests for retransmission of the ascertained voice media noted in the data quality store.

32. The application of claim 31, wherein the data quality store module is further configured to remove the notation as the corresponding one or more requests for retransmission are satisfied.

33. The application of claim 21, further comprising a rendering module associated with the first communication device and configured to optionally render the voice media of the conversation either (i) as the media is received or (ii) after an arbitrary delay by retrieving the voice media from the first storage module, wherein the arbitrary delay is defined by a user of the first communication device.

34. The application of claim 21, wherein the conversation includes one or more of the following types of media besides voice:

video, photo, text, sensor data, position or GPS information, radio signals, or any combination thereof.

35. The application of claim 21, wherein the first communication device comprises one of the following:

a land-line phone, a wireless phone, a cellular phone, a satellite phone, a computer, a radio, a server, a satellite radio, a tactical radio or a tactical phone.

36. The application of claim 21, further comprising an index module for indexing the voice media of the conversation using one of the following indexing schemes:

(i) time-based indexing;

(ii) bit or byte based indexing;

(iii) hashing type indexing;

(iv) binary-tree type indexing;

(v) file directory based indexing;

or

(vi) any combination of (i) through (v).

37. A method performed on a first communication device to synchronize voice media of a conversation conducted over a network, the method comprising:

progressively storing in a first storage element on the first communication device voice media created using the first communication device as the voice media is created using the first communication device;

progressively transmitting the voice media created using the first communication device to a second communication device over the network;

progressively receiving and storing in the first storage element voice media pertaining to the conversation generated at the second communication device, stored in a second storage element associated with the second communication device, and transmitted from the second communication device to the first communication device over the network; and

synchronizing the voice media stored in the first storage element so that first storage element and the second storage element each contain stored complete copies of the voice media of the conversation, the synchronization performed by;

ascertaining any voice media pertaining to the conversation transmitted by the second communication device but not received by the first communication device; and

sending to the second communication device one or more requests for retransmission of the ascertained voice media; and

receiving and storing in the first storage element the media received at the first communication device from the second communication device in response to the requests for retransmission.

38. The method of claim 37, further comprising:

progressively transmitting the voice media created using the first communication device as the voice media is created and progressively stored; and

progressively storing the voice media transmitted by the second communication device as the voice media is progressively received over the network.

39. The method of claim 37, wherein progressively transmitting the voice media created using the first communication device to the second communication device over the network further comprises:

(i) defining a transmission loop;

(ii) ascertaining the available voice media created using the first communication device during the defined transmission loop;

(iii) ascertaining the usable bit rate available on the network during the defined transmission loop; and

(iv) determining if the ascertained usable bit rate available on the network during the transmission loop is sufficient to transmit a first bit rate representation of the ascertained available voice media as originally encoded.

40. The method of claim 39, wherein progressively transmitting the voice media created using the first communication device to the second communication device over the network further comprises:

(v) transmitting the first bit rate presentation of the available media if the ascertained available bit rate is sufficient.

41. The method of claim 40, wherein progressively transmitting the voice media created using the first communication device to the second communication device over the network further comprises:

(vi) generating a reduced bit rate version of the available voice media relative to the first bit rate representation when the ascertained available bit rate on the network is not sufficient to transmit the first bit rate representation of the available voice media, the reduced bit rate version of the available voice media enhancing the ability for the first communication device and the second communication device to engage in the conversation in near real-time; and

(vii) transmitting the reduced bit rate version of the available voice media when the ascertained usable bit rate available on the network during the transmission loop is not sufficient.

42. The method of claim 41, wherein progressively transmitting the voice media created using the first communication device to the second communication device over the network further comprises:

defining successive transmission loops; and

performing (i) through (iii) and either (v) or (vi) and (vii) depending on if the ascertained usable bit rate on the network is sufficient or not for each defined successive loop respectively.

43. The method of claim 41, wherein generating the reduced bit rate version of the available voice media comprises one of the following:

(i) using fewer bits per unit of time when packetizing the reduced bit rate representation of the available voice media relative to the first bit rate representation;

(ii) increasing the packetization interval when transmitting the available voice media;

(iii) adjusting the rate at which packets containing the available voice media are transmitted;

or

(iv) any combination of (i) through (iii).

44. The method of claim 37, further comprising noting any missing, corrupted or reduced bit rate versions of the voice media pertaining to the conversation transmitted by the second communication device to the first communication device.

45. The method of claim 44, further comprising generating and sending to the second communication device the one or more requests for retransmission of the voice media noted as missing, corrupted or the reduced bit rate versions.

46. The method of claim 45, further comprising removing the notation for the voice media as the corresponding one or more requests for retransmission are satisfied.

47. The method of claim 37, further comprising optionally rendering the voice media of the conversation on the first communication device either (i) as the media is received or (ii) after an arbitrary delay by retrieving the voice media from the first storage element, wherein the arbitrary delay is defined by a user of the first communication device.

48. The method of claim 37, wherein the conversation includes one or more of the following types of media besides voice:

video, photo, text, sensor data, position or GPS information, radio signals, or any combination thereof.

49. The method of claim 37, wherein the first communication device for performing the method comprising one of the following:

a land-line phone, a wireless phone, a cellular phone, a satellite phone, a computer, a radio, a server, a satellite radio, a tactical radio or a tactical phone.

50. The method of claim 37, further comprising indexing the voice media of the conversation using one of the following indexing schemes:

(i) time-based indexing;

(ii) bit or byte based indexing;

(iii) hashing type indexing;

(iv) binary-tree type indexing;

(v) file directory based indexing;

or

(vi) any combination of (i) through (v).