Telecommunication and multimedia management method and apparatus

• US 8,090,867 B2
• Filed: 08/15/2008
• Issued: 01/03/2012
• Est. Priority Date: 10/19/2007
• Status: Expired due to Fees

• Current Assignee: Voxer IP LLC (Voxer, Inc.)
• Original Assignee: Voxer IP LLC (Voxer, Inc.)
• Inventors: Katis, Thomas E., Panttaja, James T., Panttaja, Mary G., Ranney, Matthew J.
• Primary Examiner(s): BATES, KEVIN T
• Application Number:
  US12/192,890
• Publication Number:
  US 20090103521A1
• Time in Patent Office: 1,236 Days
• Field of Search: 709/234, 709/235, 370/468, 370/470
• US Class Current: 709/234
• CPC Class Codes: H04L 51/04 Real-time or near real-time messaging, e.g. instant messaging [IM]
• CPC Class Codes: H04L 51/066 Format adaptation, e.g. format conversion or compression
• CPC Class Codes: H04L 51/226 Delivery according to priorities
• CPC Class Codes: H04L 65/1083 In-session procedures
• CPC Class Codes: H04L 65/1094 Inter-user-equipment sessions transfer or sharing
• CPC Class Codes: H04L 65/4015 where at least one of the additional parallel sessions is real time or time sensitive, e.g. white board sharing, collaboration or spawning of a subconference
• CPC Class Codes: H04L 65/4053 without floor control
• CPC Class Codes: H04L 65/75 Media network packet handling
• CPC Class Codes: H04L 65/752 adapting media to network capabilities
• CPC Class Codes: H04L 65/764 at the destination reformatting of additional data in video clients H04N21/4355
• CPC Class Codes: H04L 65/80 Responding to QoS

Abstract

An improved communication method for sending media between a sending node and receiving node during a conversation. When network bandwidth is insufficient to transmit a full bit rate representation of time-sensitive media, then a reduced bit rate representation of the media is transmitted for the purpose of increasing the ability of the recipient to review the media upon receipt and continue the conversation in the real-time mode when the bandwidth on the network is insufficient to support the transmission of the full bit rate representation. Media that is ascertained as not time-sensitive on the other hand is transmitted when bandwidth in excess of what is needed for time-sensitive media becomes available. When the media ascertained as not time-sensitive is transmitted, the rate of transmission is adjusted at the sending node based on network conditions, the adjusted rate of transmission being set for network efficiency and reliable delivery of the media ascertained as not time-sensitive relative to the timeliness of the delivery of the media ascertained as time-sensitive.

Claims

1. A method, comprising:

ascertaining at a sending node if media to be transmitted from the sending node to a receiving node is time sensitive or not time-sensitive, the time-sensitivity of the media determined if the media is part of a conversation and is consumed in either a real-time mode or a time-shifted mode respectively;

transmitting the media ascertained as time-sensitive from the sending node to the receiving node, the media initially transmitted at a first bit rate and at a first packetization interval, the first bit rate and the first packetization interval being sufficient for a full bit rate representation of the media to be reviewed upon receipt, the full bit rate representation of the media being derived from when the media was originally encoded;

ascertaining if bandwidth on the network is sufficient to support the transmission of the full bit rate representation of the media ascertained as time-sensitive at the first bit rate and first packetization interval;

generating a reduced bit rate representation of the media ascertained as time-sensitive when the bandwidth is insufficient;

transmitting the reduced bit rate representation from the sending node to the receiving node, the reduced bit rate representation enhancing the timeliness of the delivery of the media ascertained as time sensitive so as to increase the ability of the recipient to review the media upon receipt and continue the the consumption of the media in the real-time mode when the bandwidth on the network is insufficient;

transmitting the media ascertained as not time sensitive from the sending node to the receiving node when bandwidth is available in excess of what is needed for the transmission of time-sensitive media; and

adjusting the rate of transmission of the media ascertained as not time-sensitive at the sending node based on network conditions by setting a second packetization interval for transmitting packets of the media ascertained as not time-sensitive for network efficiency and adjusting the rate at which the packets are transmitted based on network conditions,

whereby the adjusted rate of transmission of the media ascertained as not time-sensitive is set for network efficiency and reliable delivery relative to the timeliness of the delivery of the media ascertained as time-sensitive.

2. The method of claim 1, further comprising:

(i) defining a transmission loop;

(ii) determining the media ascertained as time-sensitive and the media ascertained as not time-sensitive available for transmission during the defined transmission loop;

(iii) comparing the ascertained bandwidth on the network with the bandwidth necessary to transmit both the media ascertained as time-sensitive and not time-sensitive in the defined transmission loop; and

either;

(a) transmitting the available media ascertained as time-sensitive and the media ascertain as not time-sensitive if the ascertained bandwidth is sufficient for transmitting both types of media;

or

(b) determining if the ascertained bandwidth is sufficient for transmitting only the available media ascertained as time-sensitive during the transmission loop; and

(c) transmitting only the available media ascertained as time-sensitive if the ascertained bandwidth on the network is determined as sufficient for transmitting only the available media ascertained as time-sensitive during the transmission loop.

3. The method of claim 2, wherein transmitting only the available media ascertained as time-sensitive further comprises transmitting the full bit rate representation of the media ascertained as time-sensitive at the first bit rate and first packetization interval when the ascertained bandwidth on the network is sufficient for transmitting the full bit rate representation.

4. The method of claim 3, further comprising:

(d) determining if the ascertained bandwidth is insufficient for transmitting only the full bit rate representation of the media ascertained as time-sensitive at the first bit rate and first packetization interval;

(e) generating the reduced bit rate representation of the available media ascertained as time-sensitive only if the ascertained bandwidth is determined to be insufficient.

5. The method of claim 4, further comprising continually defining transmission loops and performing (ii), (iii), and either (a) or (b) through (e) for each transmission loop.

6. The method of claim 1, wherein generating the reduced bit rate representation of the media ascertained as time-sensitive further comprises:

defining the first packetization interval to transmit the full bit rate representation of the media ascertained as time-sensitive sufficient for the media to be reviewed upon receipt;

determining a number of bits in the packet payloads for packetizing the full bit representation of the media ascertained as time-sensitive at the defined first packetization interval; and

purposely reducing the packet payload with fewer bits than the determined number of bits when packetizing the media ascertained as time-sensitive.

7. The method of claim 6, wherein purposely reducing the packet payload with fewer bits in the packet payloads further comprises progressively lowering the number of bits.

8. The method of claim 7, further comprising;

ascertaining when progressively lowering the number of bits crosses a predetermined minimum bit threshold;

stopping the progressive lowering of the number of bits representing the packet payloads when the predetermined minimum threshold is crossed; and

increasing the first packetization interval used to packetize the media ascertained as time-sensitive.

9. The method of claim 8, further comprising forcing the conversation out of the real-time mode when the reduction of the bit representation of the media and the latency associated with the increased first packetization interval results in the conversation no longer being able to be conducted in the real-time mode.

10. The method of claim 9, further comprising continuing the transmission of the media ascertained as time-sensitive as network conditions permit after the forcing the conversation out of the real-time mode.

11. The method of claim 10, further comprising:

ascertaining when conditions on the network sufficiently improve to resume the conversation in the real-time mode; and

resuming the conversation in the real-time mode after ascertaining when conditions on the network improve by either (a) decreasing the first packetization interval, (b) increasing the packet payloads with more bits, or (c) performing both (a) and (b).

12. The method of claim 1, wherein generating the reduced bit representation of the media ascertained as time-sensitive further comprises:

(i) using different codec settings;

(ii) using different codecs;

(iii) using a compression algorithm;

or

(iv) any combination of (i) through (iii).

13. The method of claim 1, wherein ascertaining if bandwidth on the network is sufficient to support the transmission of the full bit rate representation of the media ascertained as time-sensitive at the first bit rate and the first packetization interval further comprises:

determining packet loss on the network;

comparing the determined packet loss with an acceptable packet loss threshold; and

ascertaining that the bandwidth is sufficient if the ascertained packet loss is less than or equal to the acceptable packet loss threshold.

14. The method of claim 13, wherein generating the reduced bit rate representation of the media ascertained as time-sensitive further comprises reducing the bit rate used to represent the media until the determined packet loss on the network meets the acceptable packet loss threshold.

15. The method of claim 1, wherein adjusting the rate of transmission of the media ascertained as not time-sensitive further comprises:

continuously ascertaining available bandwidth on the network in excess of bandwidth needed for transmitting media ascertained as time-sensitive; and

setting a packetization interval for transmitting packets of the media ascertained as not time-sensitive for network efficiency and adjusting the rate at which the packets are transmitted based on the ascertained available bandwidth,

whereby the adjusted rate of transmission of the media ascertained as not time-sensitive is set for network efficiency and reliable delivery relative to the timeliness of the delivery of the media ascertained as time-sensitive.

16. The method of claim 15, wherein the continuously ascertained bandwidth is the instantaneous maximum bandwidth on the network in excess of what is needed for the transmission of media ascertained as time-sensitive.

17. The method of claim 1, wherein the second packetization interval is larger than a first packetization interval.

18. The method of claim 1, wherein ascertaining bandwidth on the network further comprises ascertaining the bandwidth on the network from receipt reports sent by the receiving node to the sending node, the receipt reports including measured network parameters including one or more of the following:

throughput, loss and jitter.

19. The method of claim 1, further comprising generating receipt reports at the receiving node by:

noting missing, corrupted and reduced bit rate representations of the media as the media is received at the receiving node;

maintaining a data quality store at the receiving node for the noted media;

scanning the data quality store; and

including in the generated receipt reports requests for retransmission of the media noted in the data quality store.

20. The method of claim 19, further comprising:

receiving the retransmitted noted media at the receiving node; and

removing the notation of the received retransmitted media noted in the data quality store.

21. The method of claim 19, wherein the requests for retransmission of the noted media further comprises:

identifying the media ascertained as not time-sensitive and noted in the data quality store;

generating a low priority request for retransmission of the identified media; and

including the low priority request for retransmission in the receipt reports.

22. The method of claim 19, wherein the requests for retransmission of the noted media further comprises:

identifying the reduced bit rate representation media ascertained as time-sensitive and noted in the data quality store;

generating a high priority request for retransmission of the identified media if the reduced bit representation of the identified media is reduced to a degree that the conversation can no longer be conducted in the real-time mode; and

including the high priority request for retransmission of the identified media in the receipt reports.

23. The method of claim 19, wherein the requests for retransmission of the media further comprises:

identifying the reduced bit rate representation media ascertained as time-sensitive and noted in the data quality store; and

generating a low priority request for retransmission of the identified media so long as the reduced bit rate representation is sufficient to maintain the conversation in the real-time mode; and

including the low priority request for retransmission in the receipt reports, the retransmission of the identified media occurring when bandwidth on the network in excess of what is needed for media ascertained as time-sensitive becomes available.

24. The method of claim 19, further comprising deferring the retransmission of the full bit representation of the identified media until bandwidth in excess of what is needed to transmit media ascertained as time sensitive becomes available in response to low priority request for retransmission contained in the receipt reports.

25. The method of claim 1 further comprising:

ascertaining when the conversation transitions between the real-time mode and the time-shifted mode; and

transmitting the media of the conversation as either time-sensitive or not time-sensitive depending on if the conversation is in the real-time or the time-shifted modes respectively.

26. The method of claim 1, wherein determining if the conversation is in the real-time or the time-shifted mode is based on an explicitly declared intent of a recipient of the media to review the media either immediately upon receipt or at the later time after receipt.

27. The method of claim 1, wherein determining if the conversation is in the real-time or the time-shifted mode is based on an inferred intent based on the behavior of a recipient of the media to review the media either immediately upon receipt or at the later time after receipt.

28. The method of claim 1, wherein the sending node comprises a communication device, the communication device consisting of one of the following:

land-line phone, wireless phone, cellular phone, satellite phone, computer, radio, server, satellite radio, tactical radio or tactical phone.

29. The method of claim 1, wherein the receiving node comprises a communication device, the communication device consisting of one of the following:

land-line phone, wireless phone, cellular phone, satellite phone, computer, radio, server, satellite radio, tactical radio or tactical phone.

30. The method of claim 1, wherein the media is locally generated at the sending node.

31. The method of claim 1, wherein the sending node is a hop on the network between a remote device transmitting the media and the receiving node.

32. The method of claim 1, wherein the receiving node is a hop on the network between the sending node and the recipient.

33. The method claim 1, wherein the media further comprises one or more of the following types of media:

voice, video, text, sensor data, position or GPS information, radio signals, or a combination thereof.

34. The method of claim 1, wherein the recipient of the transmitted media is a person.

35. The method of claim 1, wherein the recipient of the transmitted media is a service running on a device connected to the network.