TELECOMMUNICATION AND MULTIMEDIA MANAGEMENT METHOD AND APPARATUS

• US 20090103560A1
• Filed: 08/15/2008
• Published: 04/23/2009
• Est. Priority Date: 10/19/2007
• Status: Active Grant

• Current Assignee: Voxer IP LLC (Voxer, Inc.)
• Original Assignee: Rebelvox LLC (Voxer, Inc.)
• Inventors: Katis, Thomas E., Panttaja, Mary G., Ranney, Matthew J., Panttaja, James T.
• Granted Patent: US 8,111,713 B2
• Time in Patent Office: Days
• US Class Current: 370/465
• CPC Class Codes: H04L 51/04 Real-time or near real-time messaging, e.g. instant messaging [IM]
• CPC Class Codes: H04L 51/066 Format adaptation, e.g. format conversion or compression
• CPC Class Codes: H04L 51/226 Delivery according to priorities
• CPC Class Codes: H04L 51/23 Reliability checks, e.g. acknowledgments or fault reporting
• CPC Class Codes: H04L 65/1083 In-session procedures
• CPC Class Codes: H04L 65/4015 where at least one of the additional parallel sessions is real time or time sensitive, e.g. white board sharing, collaboration or spawning of a subconference
• CPC Class Codes: H04L 65/4053 without floor control
• CPC Class Codes: H04L 65/752 adapting media to network capabilities
• CPC Class Codes: H04L 65/764 at the destination reformatting of additional data in video clients H04N21/4355

Abstract

An improved communication apparatus for transmitting media between a sending node and a receiving node during a conversation. When network bandwidth is insufficient to transmit a full bit rate representation of time-sensitive media, the sending node generates and transmits a reduced bit rate representation of the media for the purpose of increasing the ability of the recipient to review the media upon receipt and continue the conversation in the real-time mode when the bandwidth on the network is insufficient to support the transmission of the full bit rate representation. The sending node also transmits media that is ascertained as not time-sensitive only when bandwidth in excess of what is needed for time-sensitive media becomes available. When the media ascertained as not time-sensitive is transmitted, the rate of transmission is adjusted at the sending node based on network conditions, the adjusted rate of transmission being set for network efficiency and reliable delivery of the media ascertained as not time-sensitive relative to the timeliness of the delivery of the media ascertained as time-sensitive.

Claims

1. An apparatus, comprising:

a sending node connected to a network, the sending node configured to send media to a receiving node over the network, the sending node including;

a media ascertaining element configured to ascertain if media to be transmitted from the sending node to the receiving node is time sensitive or not time-sensitive, the time-sensitivity of the media being determined if the media is part of a conversation in either a real-time mode or a time-shifted mode respectively;

a transmitting element configured to transmit the media ascertained as time-sensitive from the sending node to the receiving node, the media initially transmitted at a first bit rate and at a first packtization interval being sufficient for a full bit rate representation of the media to be reviewed upon receipt, the full bit rate representation of the media being derived from when the media was originally encoded;

a bandwidth ascertaining element configured to ascertain if bandwidth on the network is sufficient to support the transmission of the full bit rate representation of the media ascertained as time-sensitive at the first bit rate and packetization interval sufficient for the media to be reviewed upon receipt;

a media bit rate reduction element configured to generate a reduced bit rate representation of the media ascertained as time-sensitive when the bandwidth is insufficient;

the transmitting element further configured to transmit the reduced bit rate representation from the sending node to the receiving node, the reduced bit rate representation enhancing the timeliness of the delivery of the media ascertained as time sensitive so as to increase the ability of the recipient to review the media upon receipt and continue the conversation in the real-time mode when the bandwidth on the network is insufficient to support the transmission of the full bit rate representation of the media ascertained as time-sensitive;

the transmitting element further configured to transmit the media ascertained as not time sensitive by the media ascertaining element from the sending node to the receiving node when bandwidth is available in excess of what is needed for the transmission of time-sensitive media as determined by the bandwidth ascertaining element; and

the transmitting element adjusting the rate of transmission of the media ascertained as not time-sensitive at the sending node based on network conditions by setting a second packetization interval for transmitting packets of the media ascertained as not time-sensitive for network efficiency and adjusting the rate at which the packets are transmitted based on network conditions

whereby the adjusted rate of transmission of the media ascertained as not time-sensitive is set for network efficiency and reliable delivery relative to the timeliness of the delivery of the media ascertained as time-sensitive.

2. The apparatus of claim 1, wherein the transmitting element performs a transmission sequence to define the media to be transmitted, the transmission sequence comprising:

(i) defining a transmission loop;

(ii) determining the media ascertained as time-sensitive and the media ascertained as not time-sensitive available for transmission during the defined transmission loop;

(iii) comparing the ascertained bandwidth on the network with the bandwidth necessary to transmit the available media ascertained as time-sensitive and not time-sensitive in the defined transmission loop; and

either;

(a) scheduling the transmitter to transmit the available media ascertained as time-sensitive and the media ascertain as not time-sensitive if the ascertained bandwidth is sufficient for transmitting both types of media;

or

(b) determining if the ascertained bandwidth is sufficient for transmitting only the available media ascertained as time-sensitive during the transmission loop; and

(c) scheduling the transmission of only the available media ascertained as time-sensitive if the ascertained bandwidth on the network is determined as sufficient for transmitting only the media ascertained as time-sensitive during the transmission loop.

3. The apparatus of claim 2, wherein scheduling the transmission of only the available media ascertained as time-sensitive further comprises transmitting the full bit rate representation of the media ascertained as time-sensitive at the first bit rate and first packetization interval when the ascertained bandwidth on the network is sufficient for transmitting the full bit rate representation.

4. The apparatus of claim 3, wherein the transmission sequence further comprises:

(d) determining if the ascertained bandwidth is insufficient for transmitting only the full bit rate representation of the media ascertained as time-sensitive at the first bit rate and the first packetization interval; and

(e) transmitting the reduced bit rate representation of the media as generated by the media bit rate reduction element if the ascertained bandwidth is determined to be insufficient for transmitting the full bit rate representation of the media ascertained as time-sensitive.

5. The apparatus of claim 4, wherein the transmission sequence performed by the transmission element continually defines transmission loops when there is media to be transmitted and performs (ii), (iii), and either (a) or (b) through (e) for each transmission loop.

6. The apparatus of claim 1, wherein the media bit rate reduction element further comprises:

a packetizer for defining the first packetization interval to transmit the full bit rate representation of the media ascertained as time-sensitive sufficient for the media to be reviewed upon receipt and for determining a number of bits in the packet payloads for packetizing the full bit representation of the media ascertained as time-sensitive at the defined first packetization interval; and

a bit reducing element configured to purposely reduce the packet payload with fewer bits than the determined number of bits when packetizing the media ascertained as time-sensitive.

7. The apparatus of claim 6, wherein the bit reducing element progressively reduces the number of bits when representing the packet payload with fewer bits in the packet payloads.

8. The apparatus of claim 7, wherein the packetizer of the media bit rate reduction element is further configured to increase the first packetization interval used to packetize the media ascertained as time-sensitive after progressively lowering the number of bits.

9. The apparatus of claim 1, wherein the transmitting element is further configured to transmit the media ascertained as time-sensitive as network conditions permit.

10. The apparatus of claim 1, wherein the media bit rate reduction element generates the reduced bit representation of the media ascertained as time-sensitive by:

(i) using different codec settings;

(ii) using different codecs;

(iii) using a compression algorithm;

or

(iv) any combination of (i) through (iii).

11. The apparatus of claim 1, wherein the bandwidth ascertaining element ascertains if bandwidth on the network is sufficient to support the transmission of the full bit rate representation of the media ascertained as time-sensitive at the first bit rate and the first packetization interval further comprises:

determining packet loss on the network;

comparing the determined packet loss with an acceptable packet loss threshold; and

ascertaining that the bandwidth is sufficient if the ascertained packet loss is less than the acceptable packet loss threshold.

12. The apparatus of claim 11, wherein the media bit rate reduction element is further configured to generate the reduced bit rate representation of the media ascertained as time-sensitive by reducing the bit rate used to represent the media until the determined packet loss on the network meets the acceptable packet loss threshold.

13. The apparatus of claim 1, wherein the transmission element adjusts the rate of transmission of the media ascertained as not time-sensitive by:

continuously ascertaining available bandwidth on the network in excess of bandwidth needed for transmitting media ascertained as time-sensitive;

setting a packetization interval for transmitting packets of the media ascertained as not time-sensitive for network efficiency and adjusting the rate at which the packets are transmitted based on the ascertained available bandwidth,

whereby the adjusted rate of transmission of the media ascertained as not time-sensitive is set for network efficiency and reliable delivery relative to the timeliness of the delivery of the media ascertained as time-sensitive.

14. The apparatus of claim 13, wherein the continuously ascertained bandwidth is the instantaneous maximum bandwidth on the network in excess of what is needed for the transmission of media ascertained as time-sensitive.

15. The apparatus of claim 1, wherein the second packetization interval is larger than a first packetization interval.

16. The apparatus of claim 1, wherein bandwidth ascertaining element ascertains bandwidth on the network from receipt reports sent by the receiving node to the sending node, the receipt reports including measured network parameters including one or more of the following:

throughput, loss and jitter.

17. The apparatus of claim 16, wherein the receipt reports further include either high or low priority requests for retransmission of one or more of the following:

(i) missing media;

(ii) corrupted media;

or

(iii) full bit rate representations of previously transmitted reduced bit rate representations of the media.

18. The apparatus of claim 17, wherein the transmitting element is further configured to retransmit the missing, corrupted or full bit representations of the media when bandwidth on the network in excess of what is needed for media ascertained as time-sensitive is available in response to the low priority requests.

19. The apparatus of claim 18, wherein the transmitting element is further configured to prioritize the retransmission of the missing, corrupted or full bit representations of the media ahead of the transmission of media ascertained as time-sensitive and media ascertained as not time-sensitive in response to the high priority requests.

20. The apparatus of claim 1, wherein the receiving node further comprises a receipt report generator, the receipt report generator generating receipt reports by:

noting missing, corrupted and reduced bit rate representations of the media as the media is received at the receiving node;

maintaining a data quality store at the receiving node for the noted media;

scanning the data quality store; and

including in the generated receipt reports requests for retransmission of the media noted in the data quality store.

21. The apparatus of claim 20, wherein the receiving node is further configured to:

receive the retransmitted noted media at the receiving node; and

remove the notation of the received retransmitted media noted in the data quality store.

22. The apparatus of claim 20, wherein the receipt report generator is further configured to generate the receipt reports with requests for retransmission of noted media including either high priority requests for retransmission or low priority requests for retransmission.

23. The apparatus of claim 22, wherein the transmitter at the sending node is further configured to defer the retransmission of the noted media until bandwidth in excess of what is needed to transmit media ascertained as time sensitive becomes available in response to the low priority request for retransmission.

24. The apparatus of claim 1 further comprising:

a conversation mode detection element configured to ascertain when the conversation transitions between the real-time mode and the time-shifted mode, the transmitting element further configured to transmit the media of the conversation as either time-sensitive or not time-sensitive depending on if the conversation is in the real-time or the time-shifted modes respectively.

25. The apparatus of claim 24, wherein the conversation mode detection element determines if the conversation is in the real-time or the time-shifted mode based on either an explicitly declared intent of a recipient of the media to review the media either immediately upon receipt or at the later time after receipt.

26. The apparatus of claim 24, wherein the conversation mode detection element determines if the conversation is in the real-time or the time-shifted mode based on an inferred intent based on the behavior of a recipient of the media to review the media either immediately upon receipt or at the later time after receipt.

27. The apparatus of claim 1, wherein the sending node comprises a communication device, the communication device consisting of one of the following:

land-line phone, wireless phone, cellular phone, satellite phone, computer, radio, server, satellite radio, tactical radio or tactical phone.

28. The apparatus of claim 1, wherein the receiving node comprises a communication device, the communication device consisting of one of the following:

land-line phone, wireless phone, cellular phone, satellite phone, computer, radio, server, satellite radio, tactical radio or tactical phone.

29. The apparatus of claim 1, wherein the media is locally generated at the sending node.

30. The apparatus of claim 1, wherein the sending node is a hop on the network between a remote device transmitting the media and the receiving node.

31. The apparatus of claim 1, wherein the receiving node is a hop on the network between the sending node and the recipient.

32. The apparatus claim 1, wherein the media further comprises one or more of the following types of media:

voice, video, text, sensor data, position or GPS information, radio signals, or a combination thereof.

33. The apparatus of claim 1, wherein the recipient of the transmitted media is a person.

34. The apparatus of claim 1, wherein the recipient of the transmitted media is a service running on a device connected to the network.