Telecommunication and multimedia management method and apparatus

• US 10,511,557 B2
• Filed: 10/16/2018
• Issued: 12/17/2019
• Est. Priority Date: 06/28/2007
• Status: Active Grant

• Current Assignee: Voxer IP LLC (Voxer, Inc.)
• Original Assignee: Voxer IP LLC (Voxer, Inc.)
• Inventors: Katis, Thomas E., Panttaja, James T., Panttaja, Mary G., Ranney, Matthew J.
• Primary Examiner(s): Ramakrishnaiah, Melur
• Application Number:
  US16/161,474
• Publication Number:
  US 20190052590A1
• Time in Patent Office: 427 Days
• Field of Search: 379 9301, 348 1401- 1416
• CPC Class Codes: E04H 3/126 Foldable, retractable or tiltable tribunes E04H3/123 takes precedence
• CPC Class Codes: G06F 15/16 Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs coordinating program control therefor G06F9/52; in regulating and control system G05B
• CPC Class Codes: H04L 12/1831 Tracking arrangements for later retrieval, e.g. recording contents, participants activities or behavior, network status
• CPC Class Codes: H04L 51/04 Real-time or near real-time messaging, e.g. instant messaging [IM]
• CPC Class Codes: H04L 51/10 Multimedia information
• CPC Class Codes: H04L 51/216 Handling conversation history, e.g. grouping of messages in sessions or threads
• CPC Class Codes: H04L 51/226 Delivery according to priorities
• CPC Class Codes: H04L 51/42 Mailbox-related aspects, e.g. synchronisation of mailboxes
• CPC Class Codes: H04L 61/2503 Translation of Internet protocol [IP] addresses
• CPC Class Codes: H04L 65/1083 In-session procedures
• CPC Class Codes: H04L 65/4015 where at least one of the additional parallel sessions is real time or time sensitive, e.g. white board sharing, collaboration or spawning of a subconference
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04L 65/752 adapting media to network capabilities
• CPC Class Codes: H04L 65/764 at the destination reformatting of additional data in video clients H04N21/4355
• CPC Class Codes: H04L 67/54 Presence management, e.g. monitoring or registration for receipt of user log-on information, or the connection status of the users
• CPC Class Codes: H04M 1/64 Automatic arrangements for answering calls; Automatic arrangements for recording messages for absent subscribers; Arrangements for recording conversations centralised dictation systems H04M11/10
• CPC Class Codes: H04M 11/00 Telephonic communication systems specially adapted for combination with other electrical systems
• CPC Class Codes: H04M 3/53 Centralised arrangements for recording incoming messages , i.e. mailbox systems
• CPC Class Codes: H04M 3/5307 for recording messages comprising any combination of audio and non-audio components
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/15 : Conference systems
• CPC Class Codes: H04W 4/12 : Messaging; Mailboxes; Announcements

Abstract

A telecommunication and multimedia management apparatus and method that supports voice and other media communications and that enables users to: (i) participate in multiple conversation modes, including live phone calls, conference calls, instant voice messaging or tactical communications; (ii) review the messages of conversations in either a live mode or a time-shifted mode and to seamlessly transition back and forth between the two modes; (iii) participate in multiple conversations either concurrently or simultaneously; (iv) archive the messages of conversations for later review or processing; and (v) persistently store media either created or received on the communication devices of users. The latter feature enables users to generate or review media when either disconnected from the network or network conditions are poor and to optimize the delivery of media over the network based on network conditions and the intention of the users participating in conversations.

Claims

1. A method for operating a video message service infrastructure on a network, the method comprising:

receiving, at the video message service infrastructure, a video message transmitted by a sending device, the video message containing video media, the video media being transmitted as the video media is created;

generating, by the video message service infrastructure, two or more degraded versions of the video media of the video message, where each of the two or more degraded versions is created by degrading the quality of the video media by a different amount;

selecting one of the degraded versions of the video media based on an ascertained bandwidth on the network for delivering the video media from the video message service infrastructure to a recipient device on the network; and

transmitting the selected degraded version of the video media from the video message service infrastructure to the recipient device so that the recipient device can render at least a portion of the video media on the recipient device, the video message service infrastructure operating so that the rendering occurs;

(i) while the video media of the video message is being created and transmitted over the network by the sending device; and

(ii) without having to first establish an end-to-end connection over the network between the sending device and the recipient device before the sending device begins to transmit the video media of the video message.

2. The method of claim 1, wherein at least one of the two or more degraded versions of the video media of the video message is generated by reducing a bit rate used to represent the video media relative to a complete bit rate representation of the video media as received by the video message service infrastructure.

3. The method of claim 1, further comprising transmitting one or more of the two or more degraded versions to two or more additional recipient devices, wherein the transmitted one or more degraded versions are selected at least partially based on being commensurate with ascertained bandwidths on the network along delivery paths for delivering the video message to the two or more additional recipient devices respectively.

4. The method of claim 1, wherein generating the two or more degraded versions of the video media further comprises trans-coding the video media from one video codec into one of two or more second video codec(s).

5. The method of claim 4, wherein each of the two or more second video codec(s) defines a different tradeoff between a transmission bit rate versus quality.

6. The method of claim 1, wherein generating a first of the two or more degraded versions further comprises increasing a packetization interval for packets containing the video media when transmitting the first degraded version.

7. The method of claim 1, further comprising storing, by the video message service infrastructure, each of the degraded versions of the video media of the video message, as the video message is received.

8. The method of claim 1, wherein the video message is received at the video message service infrastructure using a first real-time streaming protocol, and the selected degraded version of the video media is transmitted from the video message service infrastructure to the recipient device using a second real-time streaming protocol.

9. The method of claim 8, wherein the first real-time streaming protocol is different from the second real-time streaming protocol.

10. The method of claim 8, wherein the first and second real-time streaming protocols are higher-level protocols that operate over a lower-level transmission protocol.

11. The method of claim 10, wherein the lower-level transmission protocol is either:

(i) Transmission Control Protocol (TCP);

or

(ii) User Datagram Protocol (UDP).

12. The method of claim 1, wherein the video message service infrastructure is further configured to perform one of the following:

(i) trans-code the video message;

(ii) encrypt and authenticate the video message;

or

(iii) both (i) and (ii).

13. The method of claim 1, wherein the selected degraded version of the video media improves the ability of the recipient device to render at least the portion of the video message as the video media is being transmitted by the sending device when network conditions are insufficient to support transmission of a complete version of the video media as received by the video message service infrastructure.

14. The method of claim 1, further comprising:

ascertaining a first codec that was used by the sending device to encode the video media of the video message when sent;

ascertaining a second codec to be used by the recipient device to decode the video media of the video message when received;

trans-coding the video media of the video message from the first codec to the second codec; and

transmitting, to the recipient device, the trans-coded video media.

15. The method of claim 1, further comprising:

storing the video media of the video message in a storage location accessible by the video message service infrastructure; and

providing from the storage location to the recipient device a complete copy of the video media of the video message for rendering in a time-shifted mode, wherein the complete copy is the same quality as received by the video message service infrastructure from the sending device.

16. The method of claim 1, further comprising:

receiving, at the video message service infrastructure, a second video message transmitted by a second sending device, the second video message including second video media, wherein the second video media being transmitted by the second sending device as the second video media is created;

transmitting the second video media from the video message service infrastructure over the network to the recipient device so that the recipient device can render at least a portion of the second video media, the video message service infrastructure operating so that the rendering occurs;

(i) as the second video media of the second video message is created and transmitted over the network by the second sending device;

(ii) without having to first establish an end-to-end connection over the network between the second sending device and the recipient device before the second sending device begins to transmit the second video media of the second video message; and

(iii) simultaneously with the rendering of the video media of the video message.

17. The method of claim 1, further comprising providing an application for execution on the sending device, the application enabling a sender using the sending device to transmit the video media of the video message as the video media is created and encoded.

18. The method of claim 1, further comprising providing an application for execution on the recipient device, the application enabling a recipient using the recipient device to selectively render the video media of the video message:

at least partially as the video media of the video message is created and transmitted by the sending device and delivered by the video message service infrastructure; and

in a time-shifted mode by rendering the video media of the video message out of storage.

19. The method of claim 1, wherein the sending device is arranged to initiate transmission of the video message to the video message service infrastructure independently of the video message service infrastructure ascertaining a location of the recipient device or any other location(s) of other recipient device(s) on the network.

20. The method of claim 1, wherein the sending device is arranged to transmit the video message to the video message service infrastructure regardless of whether the recipient device or any other recipient device(s) is/are available or not available on the network.

21. The method of claim 1 further configured to provide a notification of the video message to the recipient device.

22. The method of claim 1, further comprising:

receiving an identifier from the sending device, the identifier identifying the recipient of the video message;

resolving the identifier into an IP address on the network, the IP address identifying the recipient device on the network; and

;

using the IP address to transmit the video media of the video message to the recipient device.

23. The method of claim 1, further comprising:

configuring the video message service infrastructure to include a plurality of servers located on the network, the plurality of servers collectively configured for performing;

receiving the video message;

ascertaining locations on the network for multiple recipient devices including the recipient device, the locations derived from multiple identifiers provided by the sending device;

transmitting the video message to the multiple recipient devices so that each of the multiple recipient devices can render at least a portion of the video message as the sending device is sending the video message;

storing the video message as the video media is received by the video message service infrastructure; and

providing the video media of the video message to any of the multiple recipient devices for rendering in a time-shifted mode.

24. A method for operating a video message service infrastructure on a network, the method comprising:

receiving a video message sent by a sending device, the video message containing video media that is transmitted over the network by the sending device as the video media is being created;

ascertaining bandwidth over the network for delivering the video media of the video message from video message service infrastructure to a recipient device on the network;

generating an adjusted bit rate representation of the video media of the video message commensurate with the ascertained bandwidth; and

transmitting the adjusted bit rate representation of the video media over the network to the recipient device so that a recipient can render at least a portion of the video message on the recipient device, the video message service infrastructure operating so that rendering occurs;

(i) while the video media of the video message is being created and transmitted over the network by the sending device; and

(ii) without having to first establish an end-to-end connection over the network between the sending device and the recipient device before the sending device begins to transmit the video media of the video message,

whereby generating the adjusted the bit rate representation of the video media of the video message improves the ability of the recipient to render the portion of the video message on the recipient device as the video media is being transmitted by the sending device when network conditions are insufficient to support transmission of a complete version of the video media, wherein the complete version is the same quality as received by the video message service infrastructure.

25. The method of claim 24, further comprising:

ascertaining bandwidth over the network for delivering the video media of the video message to each of multiple recipient devices on the network, the multiple recipient devices including the recipient device;

generating within the video message service infrastructure one or more adjusted bit rate representations of the video media of the video message;

ascertaining a selected bit rate representation, among the one or more adjusted bit rate representation, for each of the multiple recipient devices based on the ascertained bandwidth for each of the multiple recipient devices respectively;

transmitting over the network each of the selected adjusted bit rate representations of the video media from the video message service infrastructure to the multiple recipient devices respectively, the

wherein the transmission of the one or more adjusted bit rate representations of the video media of the video message enables each of the multiple recipient devices to render at least portions of the video media of the video message respectively,

the video message service infrastructure operating so that rendering occurs;

(i) while the video media of the video message is being created and transmitted over the network by the sending device; and

(ii) without having to first establish an end-to-end connection over the network between the sending device and any of the multiple recipient devices before the sending device begins to transmit the video media of the video message.

26. The method of claim 24, wherein generating the adjusted bit rate representation of the video media of the video message comprises degrading the quality of the video media of the video message.

27. The method of claim 24, wherein generating the adjusted bit rate representation of the video media of the video message comprises selecting one of multiple codecs, each of the multiple codecs defining a different tradeoff between a transmission bit rate versus quality, and the selection of one of the multiple codecs is at least partially based on the ascertained bandwidth.

28. The method of claim 24, wherein generating the adjusted bit rate representation of the video media of the video message comprises increasing a packetization interval for packets containing the video media when transmitting the video message.

29. The method of claim 24, further comprising storing, by the video message service infrastructure, the adjusted bit rate representation of the video media of the video message.

30. The method of claim 24, wherein the video message is received at the video message service infrastructure using a first real-time streaming protocol, and the adjusted bit rate representation of the video media of the video message is transmitted from the video message service infrastructure to the recipient device using a second real-time streaming protocol.

31. The method of claim 30, wherein the first real-time streaming protocol is different from the second real-time streaming protocol.

32. The method of claim 30, wherein the first and second real-time streaming protocols are higher-level protocols that operate over a lower-level transmission protocol.

33. The method of claim 32, wherein the lower-level transmission protocol is TCP.

34. The method of claim 32, wherein the lower-level transmission protocol is UDP.

35. The method of claim 24, wherein the video message service infrastructure is further configured to perform one of the following:

(i) trans-code the video message;

(ii) encrypt and authenticate the video message;

or

(iii) both (i) and (ii).

36. The method of claim 24, further comprising:

ascertaining a first codec that was used by the sending device to encode the video media of the video message when transmitted;

ascertaining a second codec to be used by the recipient device to decode the video media of the video message when received;

trans-coding the video media of the video message from the first codec to the second codec; and

delivering the trans-coded video media when transmitting the adjusted bit rate representation of the video media of the video message to the recipient device.

37. The method of claim 24, further comprising:

storing the video media message in a storage location accessible by the video message service infrastructure; and

providing from the storage location to the recipient device a complete copy of the video message for rendering in a time-shifted mode, wherein the complete copy is the same quality of the video media as received by the video message service infrastructure from the sending device.

38. The method of claim 24, further comprising providing an application for execution on the sending device, the application enabling a sender using the sending device to transmit the video media of the video message as the video media is created and encoded.

39. The method of claim 24, further comprising providing an application for execution on the receiving device, the application enabling the recipient using the recipient device to selectively render the video media of the video message:

at least partially as the video media of the video message is being created and transmitted by the sending device and delivered by the video message service infrastructure; and

in a time-shifted mode by rendering the video media of the video message received from storage.

40. The method of claim 24, further comprising configuring the sending device to initiate transmission of the video message to the video message service infrastructure independently of the video message service infrastructure ascertaining a location of the recipient device and any other recipient device(s) on the network.

41. The method of claim 24, further comprising configuring the sending device to transmit the video message to the video message service infrastructure regardless of whether the recipient device and any other recipient device(s) are available or not available on the network.

42. The method of claim 24, further comprising providing a notification of the video message to the recipient device.

43. The method of claim 24, comprising configuring the video message service infrastructure for:

receiving an identifier, the identifier identifying the recipient for receiving the video message;

resolving the identifier into an IP address on the network, the IP addresses identifying the recipient device on the network; and

using the IP address for transmitting the video media of the video message from the video service infrastructure to the recipient device.

44. The method of claim 24, further comprising configuring the video message service infrastructure to include a plurality of servers located on the network, the plurality of servers collectively configured for performing the following:

receiving the video message;

ascertaining multiple locations on the network for multiple recipient devices including the receiving device, the multiple locations derived from one or more identifiers provided by the sending device for identifying multiple recipients including the recipient for receipt of the video message;

transmitting the video message to the multiple recipient devices so that portions of the video message can be respectively rendered by the multiple recipient devices at the same time as the sending device is transmitting the video message;

storing the video message in a storage location accessible by the video message service infrastructure; and

providing the stored video message to one or more of the multiple recipient devices for rendering in a time-shifted mode.

45. A method for operating a video message service infrastructure on a network, the method comprising:

receiving, at the video message service infrastructure, a first video message transmitted by a first sending device, the first video message containing first video media, the first video media being transmitted as the first video media is created;

receiving, at the video message service infrastructure, a second video message transmitted by a second sending device, the second video message containing second video media, the second video media being transmitted as the second video media is created;

transmitting from the video message service infrastructure over the network;

(a) the first video media of the first video message to a recipient device; and

(b) the second video media of the second video message to the recipient device;

the transmission of (a) and (b) occurring so that the recipient device can simultaneously render at least a first portion of the first video message and at least a second portion of the second video message, the simultaneous rendering occurring;

(c) as the first video media of the first video message and the second video media of the second video message are each being created and transmitted over the network by the first and the second sending devices respectively; and

(d) without having to first establish an end-to-end connection over the network between either of;

(i) the first sending device and the recipient device before the first sending device begins to transmit the first video media of the first video message; and

(ii) the second sending device and the recipient device before the second sending device begins to transmit the second video media of the second video message.

46. The method of claim 45, further comprising providing an application for execution on the first sending device and the second sending device,

the application enabling a first sender using the first sending device to:

(e) identify on the first sending device a first group of one or more recipients for receipt of the first video message; and

(f) transmit the first video message as the first video media is being created; and

the application enabling a second sender using the second sending device to;

(g) identify on the second sending device a second group of one or more recipients; and

(d) transmit the second video message as the second video media is being created,

wherein the recipient device receives both the first media and the second media of the first and second video messages because the recipient is included in both the first and second groups respectively.

47. The method of claim 45, further comprising providing an application for execution on the recipient device, the application enabling a recipient using the recipient device to selectively render:

(e) the at least first and second portions simultaneously; and

(f) the first video message and the second video message each selectively out of storage in a time-shifted mode.

48. The method of claim 45, wherein the first sending device is arranged for transmitting the first video message to the video message service infrastructure both:

(e) independently of the video message service infrastructure ascertaining a location of the recipient device or any other recipient devices on the network; and

(f) regardless of whether the recipient device or any other recipient devices are available or not available on the network.

49. The method of claim 45, further comprising providing a first notification and a second notification at the recipient device for notifying the recipient of the first video message and the second video message respectively.

50. The method of claim 45, wherein the video message service infrastructure is further arranged for:

receiving a first identifier, the first identifier identifying a recipient user of the recipient device;

receiving the identifier with the second video message;

resolving an IP address for the recipient device from the identifier; and

using the IP address for transmitting, from the video message service infrastructure to the recipient device, the first video media of the first video message and the second video media of the second video message.

51. The method of claim 45, further comprising configuring the video message service infrastructure to include a plurality of servers located on the network, the plurality of servers collectively configured for performing:

receiving the first video message and the second video message;

transmitting the first video message to first multiple recipient devices, including the recipient device, the first multiple recipient devices each capable of rendering first portions of the first video message as the first sending device is transmitting the first video message; and

transmitting the second video message to second multiple recipient devices, including the recipient device, the second multiple recipient devices each capable of rendering second portions of the second video message as the second sending device is transmitting the second video message.

52. The method of claim 45, further comprising degrading the quality of the first video media of the first video message for the purpose of enhancing the ability of a recipient to render the first video message on the recipient device when network conditions are insufficient to support transmission of a complete representation of the first video media as received by the video message service infrastructure from the first sending device.

53. The method of claim 45, further comprising degrading the quality of the first video media of the first video message by reducing a bit rate used to represent the first video media, wherein the bit rate is reduced relative to a complete bit representation of the first video media as received by the video message service infrastructure from the first sending device.

54. The method of claim 45, further comprising generating two or more bit rate versions used to represent the first video media of the first video message, each of the two or more bit rate versions representing different degraded quality versions of the first video media of the first video message.

55. The method of claim 45, further comprising degrading the quality of the first video media of the first video message by selecting one of multiple codecs, each of the multiple codecs defining a different tradeoff of transmission bit rate versus quality, wherein the selection of one of the multiple codecs is at least partially based on available bandwidth on at least a portion of a delivery path over the network for delivering the first video message to the recipient device.

56. The method of claim 45, further comprising degrading a quality of the first video media of the first video message by increasing a packetization interval for packets containing the first video media when transmitting the first video message from the video message service infrastructure to the recipient device.

57. The method of claim 45, further comprising simultaneously transmitting from the video message service infrastructure over the network to the recipient device the first portion of the first video message and the second portion of the second video message so that the recipient device can mix and simultaneously render the first and second portions of the two video messages.

58. The method of claim 45, further comprising configuring the video message service infrastructure to include a plurality of servers located on the network, the plurality of servers collectively configured for performing the following:

receiving the first video media of the first video message and the second video media of the second video message;

storing the first video media of the first video message and the second video media of the second video message; and

simultaneously transmitting first packets containing the first video media of the first video message and second packets containing the second video media of the second video message to the recipient device so that at least the first and second portions of the first video message and the second video message can be mixed and simultaneously rendered as the first and second sending devices are transmitting the first video message and the second video message.

59. The method of claim 45, further comprising configuring the video message service infrastructure to include a plurality of servers located on the network, the plurality of servers collectively configured for performing one of the following:

(i) trans-coding the first video media of the first video message and the second video media of the second video message; and

(ii) encrypting the first video media of the first video message and the second video media of the second video message; and

(iii) authenticating the first video message and the second video message;

or

(iv) any combination of (i) through (iii).