Telecommunication and multimedia management method and apparatus

• US 10,841,261 B2
• Filed: 10/31/2019
• Issued: 11/17/2020
• Est. Priority Date: 06/28/2007
• Status: Active Grant

• Current Assignee: Voxer IP LLC (Voxer, Inc.)
• Original Assignee: Voxer IP LLC (Voxer, Inc.)
• Inventors: Katis, Thomas E., Panttaja, James T., Panttaja, Mary G., Ranney, Matthew J.
• Primary Examiner(s): Ramakrishnaiah, Melur
• Application Number:
  US16/669,865
• Publication Number:
  US 20200067859A1
• Time in Patent Office: 383 Days
• Field of Search: 379 9301, 379 9317, 348 1401- 1416, 709204, 709205
• CPC Class Codes: E04H 3/126 Foldable, retractable or tiltable tribunes E04H3/123 takes precedence
• CPC Class Codes: G06F 15/16 Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs coordinating program control therefor G06F9/52; in regulating and control system G05B
• CPC Class Codes: H04L 12/1831 Tracking arrangements for later retrieval, e.g. recording contents, participants activities or behavior, network status
• CPC Class Codes: H04L 51/04 Real-time or near real-time messaging, e.g. instant messaging [IM]
• CPC Class Codes: H04L 51/10 Multimedia information
• CPC Class Codes: H04L 51/216 Handling conversation history, e.g. grouping of messages in sessions or threads
• CPC Class Codes: H04L 51/226 Delivery according to priorities
• CPC Class Codes: H04L 51/42 Mailbox-related aspects, e.g. synchronisation of mailboxes
• CPC Class Codes: H04L 61/2503 Translation of Internet protocol [IP] addresses
• CPC Class Codes: H04L 65/1083 In-session procedures
• CPC Class Codes: H04L 65/4015 where at least one of the additional parallel sessions is real time or time sensitive, e.g. white board sharing, collaboration or spawning of a subconference
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04L 65/752 adapting media to network capabilities
• CPC Class Codes: H04L 65/764 at the destination reformatting of additional data in video clients H04N21/4355
• CPC Class Codes: H04L 67/54 Presence management, e.g. monitoring or registration for receipt of user log-on information, or the connection status of the users
• CPC Class Codes: H04M 1/64 Automatic arrangements for answering calls; Automatic arrangements for recording messages for absent subscribers; Arrangements for recording conversations centralised dictation systems H04M11/10
• CPC Class Codes: H04M 11/00 Telephonic communication systems specially adapted for combination with other electrical systems
• CPC Class Codes: H04M 3/53 Centralised arrangements for recording incoming messages , i.e. mailbox systems
• CPC Class Codes: H04M 3/5307 for recording messages comprising any combination of audio and non-audio components
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/15 : Conference systems
• CPC Class Codes: H04W 4/12 : Messaging; Mailboxes; Announcements

Abstract

A telecommunication and multimedia management apparatus and method that supports voice and other media communications and that enables users to: (i) participate in multiple conversation modes, including live phone calls, conference calls, instant voice messaging or tactical communications; (ii) review the messages of conversations in either a live mode or a time-shifted mode and to seamlessly transition back and forth between the two modes; (iii) participate in multiple conversations either concurrently or simultaneously; (iv) archive the messages of conversations for later review or processing; and (v) persistently store media either created or received on the communication devices of users. The latter feature enables users to generate or review media when either disconnected from the network or network conditions are poor and to optimize the delivery of media over the network based on network conditions and the intention of the users participating in conversations.

Claims

1. A method for operating a video message service infrastructure on a network, the method comprising:

receiving, at the video message service infrastructure, a video message transmitted by a sending device, video media of the video message transmitted as the video media is created;

degrading the quality of the video media of the video message based on an ascertained bandwidth on the network for delivering the video media from the video message service infrastructure to a recipient device on the network;

transmitting the degraded quality of the video media of the video message from the video message service infrastructure to the recipient device so that a recipient can render at least a portion of the video media of the video message on the recipient device, the video message service infrastructure operating so that the rendering occurring;

(i) while the video media of the video message is being created and transmitted over the network by the sending device; and

(ii) without having to first establish an end-to-end connection over the network between the sending device and the recipient device before the sending device begins to transmit the video media of the video message.

2. The method of claim 1, wherein degrading the quality of the video media of the video message comprises reducing a bit rate used to represent the video media relative a complete bit rate representation of the video media as received by the video message infrastructure.

3. The method of claim 1, further comprising generating two or more bit rate versions used to represent the video media of the video message, each of the two or more bit rate versions representing different degraded quality versions of the video media of the video message.

4. The method of claim 3, further comprising transmitting one or more of the two or more bit rate versions to two or more recipient devices, wherein the one or more transmitted bit rate versions are each commensurate with bandwidth on the network along delivery paths for delivering the video message to the two or more recipient devices respectively.

5. The method of claim 1, wherein degrading the quality of the video media further comprises encoding the video media using one of multiple codecs, each of the multiple codecs defining a different tradeoff between a transmission bit rate versus quality.

6. The method of claim 5, wherein degrading the quality of the video media further comprises increasing a packetization interval for packets containing the video media when transmitting the video message to the recipient device.

7. The method of claim 1, wherein degrading the quality of the video media further comprises lowering a transmission rate of bits representing the video media when transmitting the video message.

8. The method of claim 1, wherein degrading the quality of the video media improves the ability of the recipient to render at least the portion of the video message on the recipient device as the video media is being transmitted by the sending device when network conditions are insufficient to support transmission of a complete version of the video media as received by the video message infrastructure.

9. The method of claim 1, further comprising:

ascertaining a first codec that was used by the sending device to encode the video media of the video message when sent;

ascertaining a second codec to be used by the recipient device to decode the video media of the video message when received;

trans-coding the video media of the video message from the first codec to the second codec; and

delivering the trans-coded video media when transmitting the degraded quality of the video media of the video message to the recipient device.

10. The method of claim 1, further comprising:

storing the video media of the video message in a storage location accessible by the video message service infrastructure; and

providing from the storage location to the recipient device a complete copy of the video media of the video message for rendering in a time-shifted mode, wherein the complete copy is the same quality as received by the video message infrastructure.

11. The method of claim 1, further comprising:

receiving, over the network at the video message service infrastructure, a second video message transmitted by a second sending device, second video media of the second video message transmitted as the second video media is created;

transmitting the second video media from the video message service infrastructure over the network to the recipient device so that the recipient can render at least a portion of the second video media on the recipient device, the video message service infrastructure operating so that the rendering occurs;

(i) as the second video media of the second video message is created and transmitted over the network by the second sending device;

(ii) without having to first establish an end-to-end connection over the network between the second sending device and the recipient device before the second sending device begins to transmit the second video media of the second video message; and

(iii) simultaneously with the video media of the video message.

12. The method of claim 1, further comprising providing an application for execution on the sending device, the application enabling a sender to create and transmit the video media of the video message as the video media is created and encoded.

13. The method of claim 1, further comprising providing an application for execution on the recipient device, the application enabling the recipient to selectively render the video media of the video message and delivered by video message service infrastructure:

at least partially as the video media of the video message is created, transmitted by the sending device and delivered by the video message service infrastructure; and

in a time-shifted mode by rendering the video media of the video message out of storage.

14. The method of claim 1, wherein the sending device is arranged to initiate transmission of the video message to the video message service infrastructure independently of the video message service infrastructure ascertaining a location of any of one or more recipient devices on the network.

15. The method of claim 1, wherein the sending device is arranged to transmit the video message to the video message service infrastructure regardless of whether any of one or more recipient devices are available or not available on the network.

16. The method of claim 1, further configured to provide a notification of the video message to one or more recipient devices respectively.

17. The method of claim 1, further comprising:

receiving one or more identifiers with the received video message, the one or more identifiers associated with one or more recipients of the video message;

resolving the one or more identifiers into one or more IP addresses on the network, each of the one or more IP addresses associated with one or more recipient devices associated with the one or more recipients respectively; and

using the one or more IP addresses for routing the video media over the network when transmitting the video message from the video message service infrastructure to the one or more recipient devices.

18. The method of claim 1, further comprising:

configuring the video message service infrastructure to include a plurality of servers located on the network, the plurality of servers collectively configured for performing;

receiving the video message;

ascertaining one or more locations on the network for one or more recipient devices, the one or more locations derived from one or more identifiers received with the video message and for identifying the one or more recipients of the video message;

transmitting the video message to the one or more recipient devices so that at least a portion of the video message can be rendered at the same time as the sending device is sending the video message;

trans-code thing video message;

encrypting and authenticating the video message; and

storing the video message and transmitting the video message to the one or more recipient devices out of storage so that the video message can be rendered in a time-shifted mode.

19. A method for operating a video message service infrastructure on a network, the method comprising:

receiving a video message sent by a sending device, video media of the video message transmitted over the network by the sending device as the video media of the video message is created;

creating an adjusted bit rate representation of the video media of the video message; and

transmitting the adjusted bit rate representation of the video media over the network to the recipient device so that a recipient can render at least a portion of the video message on the recipient device, the video message service infrastructure operating so that rendering occurs;

(i) while the video media of the video message is being created and transmitted over the network by the sending device; and

(ii) without having to first establish an end-to-end connection over the network between the sending device and the recipient device before the sending device begins to transmit the video media of the video message.

20. The method of claim 19, further comprising:

generating multiple adjusted bit rate representations of the video media of the video message; and

transmitting over the network one or more of the multiple adjusted bit rate representations of the video media of the video message from the video message service infrastructure to recipient devices, each transmission commensurate with an expected bandwidth on the network for transmission to one of the recipient devices,

wherein the transmission to each of the recipient devices enables rendering of at least a portion of the video media of the video message, the video message service infrastructure operating so that rendering occurs;

(i) while the video media of the video message is being created and transmitted over the network by the sending device; and

(ii) without having to first establish an end-to-end connection over the network between the sending device and any of the recipient devices before the sending device begins to transmit the video media of the video message.

21. The method of claim 19, further comprises degrading the quality of the video media of the video message when creating the adjusted bit rate representation.

22. The method of claim 19, wherein creating the adjusted bit rate representation of the video media of the video message comprises selecting one of multiple codecs, each of the multiple codecs defining a different tradeoff between a transmission bit rate versus quality, wherein the selection of one of the multiple codecs is at least partially based on the ascertained bandwidth.

23. The method of claim 19, wherein creating the adjusted bit rate representation of the video media of the video message further comprises increasing a packetization interval for packets containing the video media when transmitting the video message.

24. The method of claim 19, wherein creating the adjusted bit rate representation of the video media of the video message further comprises lowering a transmission rate of bits representing the video media when transmitting the video message.

25. The method of claim 19, wherein the adjusted bit rate representation of the video media of the video message improves the ability of the recipient to render at least the portion of the video message on the recipient device as the video media is being transmitted by the sending device when network conditions are insufficient to support transmission of a complete version of the video media, wherein the complete version is the same quality as received by video message service infrastructure.

26. The method of claim 19, further comprising:

ascertaining a first codec that was used by the sending device to encode the video media of the video message when sent;

ascertaining a second codec to be used by the recipient device to decode the video media of the video message when received;

trans-coding the video media of the video message from the first codec to the second codec; and

delivering the trans-coded video media when transmitting the video media of the video message to the recipient device.

27. The method of claim 19, further comprising:

storing the video media message in a storage location accessible by the video message service infrastructure; and

providing from the storage location to the recipient device a complete copy of the video media of the video message when requested for rendering in a time-shifted mode, wherein the complete copy is the same quality as received by video message service infrastructure.

28. The method of claim 19, further comprising providing an application for execution on the sending device, the application enabling a sender using the sending device to transmit the video media of the video message as the video media is created and encoded.

29. The method of claim 19, further comprising providing an application for execution on the receiving device, the application enabling the recipient to selectively render the video media of the video message:

at least partially as the video media of the video message is being created, transmitted by the sending device and delivered by the video message service infrastructure; and

in a time-shifted mode by rendering the video media of the video message out of storage.

30. The method of claim 19, wherein the sending device is arranged to initiate transmission of the video message to the video message service infrastructure independently of the video message service infrastructure ascertaining a location of any of one or more recipient devices on the network.

31. The method of claim 19, wherein the sending device is arranged to transmit the video message to the video message service infrastructure regardless of whether any of one or more recipient devices are available or not available on the network.

32. The method of claim 19, further configured to provide a notification of the video message to one or more recipient devices respectively.

33. The method of claim 19, wherein the video message service infrastructure is further arranged for:

receiving one or more identifiers with the received video message, the one or more identifiers associated with one or more recipients;

resolving the one or more identifiers into one or more IP addresses on the network, each of the one or more IP addresses associated with one or more recipient devices associated with the one or more recipients respectively; and

using the one or more IP addresses for routing the video media of the video message over the network to the one or more recipient devices.

34. The method of claim 19, further comprising configuring the video message service infrastructure to include a plurality of servers located on the network, the plurality of servers collectively configured for performing the following:

receiving the video message;

ascertaining one or more locations on the network for one or more recipient devices, the one or more locations derived from one or more identifiers received with the video message and identifying the one or more recipients of the video message;

storing the video message and transmitting the video message to the one or more recipient devices out of storage so that the video message can be rendered in a time-shifted mode;

transmitting the video message to the one or more recipient devices so that at least a portion of the video message can be rendered at the same time as the sending device is transmitting the video message;

trans-coding the video message; and

encrypting and authenticating the video message.

35. A method for operating a video message service infrastructure on a network, the method comprising:

receiving, at the video message service infrastructure, a first video message transmitted by a first sending device, first video media of the first video message transmitted as the first video media is created;

receiving, at the video message service infrastructure, a second video message transmitted by a second sending device, second video media of the second video message transmitted as the second video media is created;

transmitting from the video message service infrastructure over the network;

(a) the first video media of the first video message to a recipient device; and

(b) the second video media of the second video message to the recipient device;

the transmission of (a) and (b) occurring so that a recipient can simultaneously render on the recipient device at least a portion of the first video message and at least a portion of the second video message, the simultaneous rendering occurring;

(c) as the first video media of the first video message and the second video media of the second video message are each created and transmitted over the network by the first and the second sending devices respectively; and

(d) without having to first establish an end-to-end connection over the network between either of;

(i) the first sending device and the recipient device before the first sending device begins to transmit the first video media of the first video message; and

(ii) the second sending device and the recipient device before the second sending device begins to transmit the second video media of the second video message,

wherein the video message service infrastructure is further configured to either or both trans-code and/or degrade the quality of the first video media of the first video message and the second media of the second video message as needed when transmitting to the recipient device.

36. The method of claim 35, further comprising providing an application for execution on each of the first sending device and the second sending device,

the application enabling a first sender to:

(e) identify on the first sending device a first group of one or more first recipients for receipt of the first video message; and

(f) transmit the first video message as the first video media is being created; and

the application enabling a second sender to;

(g) identify on the second sending device a second group of one or more second recipients for receipt of the second video message; and

(d) transmit the second video message as the second video media is being created,

wherein the recipient is included in both the first group and the second group.

37. The method of claim 35, further comprising providing an application for execution on the recipient device, the application enabling the recipient to selectively render:

(a) at least the portions of both the first video message and the second video message as simultaneously delivered by the video message service infrastructure over the network; and

(b) each of the first video message and the second video message out of storage in a time-shifted mode.

38. The method of claim 35, wherein the first sending device is arranged for transmitting the first video message to the video message service infrastructure both:

(a) independently of the video message service infrastructure ascertaining a location of any of one or more recipient devices on the network; and

(b) regardless of whether any of the one or more recipient devices are available or not available on the network.

39. The method of claim 35, further comprising providing a first notification and a second notification at the recipient device for notifying the recipient of the first video message and the second video message respectively.

40. The method of claim 35, wherein the video message service infrastructure is further arranged for:

receiving an identifier with the first video message, the identifier identifying the recipient user;

receiving the identifier with the second video message;

resolving, within the video message service infrastructure, an IP address for the recipient device from the identifier; and

using the IP address for routing over the network the first video media of the first video message and the second video media of the second video message when transmitting the first video message and the second video message to the recipient device.

41. The method of claim 35, further comprising configuring the video message service infrastructure to include a plurality of servers located on the network, the plurality of servers collectively configured for performing the following:

receiving the first video message and the second video message;

transmitting the first video message to one or more recipient devices so that at least a portion of the first video message can be rendered at the same time as the first sending device is transmitting the first video message;

transmitting the second video message to one or more second recipient devices so that at least a portion of the second video message can be rendered at the same time as the second sending device is transmitting the second video message;

storing and transmitting the first video message and the second video message out of storage so that the first video message and the second video message can each be selectively rendered in a time-shifted mode by the one or more first and second recipient devices respectively;

trans-code the first video message and the second video message; and

provide encryption and authentication of the first video message and the second video message.

42. The method of claim 35, further comprising degrading the quality of the first video media of the first video message for the purpose of enhancing the ability of the recipient to render at least the portion of the first video message on the recipient device when network conditions are insufficient to support transmission of a complete representation of the first video media as received by the video message service infrastructure.

43. The method of claim 35, further comprising degrading the quality of the first video media of the first video message by reducing a bit rate used to represent the first video media, wherein the bit rate is reduced relative to a complete bit representation of the first video media as received by the video message service infrastructure.

44. The method of claim 35, further comprising generating two or more bit rate versions used to represent the first video media of the first video message, each of the two or more bit rate versions representing different degraded quality versions of the first video media of the first video message.

45. The method of claim 35, further comprising degrading the quality of the first video media of the first video message by using one of multiple codecs, each of the multiple codecs defining a different tradeoff of transmission bit rate versus quality, wherein a selection of one of the multiple codecs is at least partially based on available bandwidth on at least a portion of the delivery path over the network for delivering the first video message to the recipient device.

46. The method of claim 35, further comprising degrading a quality of the first video media of the first video message by increasing a packetization interval for packets containing the first video media when transmitting the first video message to the recipient device.

47. The method of claim 35, further comprising degrading the quality of the first video media of the first video message by lowering a transmission rate of bits representing the first video media when transmitting the first video message to the recipient device.

48. A method for operating a video message service infrastructure on a network, the method comprising:

receiving a video message sent by a sending device as video media of the video message is created and transmitted by the sending device over the network;

ascertaining a first codec that was used by the sending device to encode the video media of the video message when transmitted;

ascertaining a second codec to be used by a recipient device to decode the video media of the video message when received at the recipient device;

trans-coding the video media of the video message from the first codec to the second codec; and

transmitting the trans-coded video media of the video message over the network to the recipient device so that a recipient can render at least a portion of the video media of the video message on the recipient device, the video message service infrastructure operating so that rendering occurring;

(i) while the video media of the video message is being created and transmitted over the network by the sending device; and

(ii) without having to first establish an end-to-end connection over the network between the sending device and the recipient device before the sending device begins to transmit the video media of the video message.

49. The method of claim 48, further comprising degrading the quality of the trans-coded video media based on an ascertained bandwidth on the network for delivering the video media of the video message from the video message service infrastructure to the recipient device.

50. The method of claim 49, wherein degrading the quality of the trans-coded video media comprises reducing a bit rate used to represent the trans-coded video media of the video message, wherein the bit rate is reduced relative to a complete representation of the first video media as received by the video message service infrastructure.

51. The method of claim 49, wherein degrading the quality of the trans-coded video media further comprising generating two or more bit rate versions used to represent the trans-coded video media of the video message, each of the two or more bit rate versions representing different degraded quality versions of the trans-coded video media of the video message.

52. The method of claim 51, further comprising transmitting one or more of the two or more bit rate versions representing different degraded quality versions of the trans-coded video media of the video message to two or more recipient devices, wherein:

(a) the two or more recipient devices each use the second codec to decode the trans-coded video media; and

(b) the one or more bit rate versions are each commensurate with bandwidth on the network along delivery paths to the two or more recipient devices respectively.

53. The method of claim 49, wherein degrading the quality of the trans-coded video media further comprises using one of multiple codecs, each of the multiple codecs defining a different tradeoff between a transmission bit rate versus quality, wherein a selection of one of the multiple codecs is at least partially based on available bandwidth on at least a portion of the delivery path over the network for delivering the video message to the recipient device.

54. The method of claim 49, wherein degrading the quality of the trans-coded video media further comprises increasing a packetization interval for packets containing the trans-coded video media when transmitting the video message to the recipient device.

55. The method of claim 48, wherein degrading the quality of the trans-coded video media further comprises lowering a transmission rate of bits representing the video media when transmitting the video message to the recipient device.

56. The method of claim 48, wherein degrading the quality of the trans-coded video media enhances the ability of the recipient to render at least the portion of the video message as the video media is being transmitted by the sending device when network conditions are insufficient to support transmission of a complete representation of the first video media as received by the video message service infrastructure.

57. The method of claim 48, further comprising:

receiving an identifier with the received video message, the identifier associated with the recipient;

resolving the identifier into an IP address on the network, the IP address associated with the recipient device; and

using the IP addresses for routing the trans-coded video media of the video message when transmitting the video message to the recipient device.

58. The method of claim 48, further comprising configuring the video message service infrastructure to include a plurality of servers located on the network, the plurality of servers collectively configured for performing the following:

receiving the video message;

ascertaining a location on the network for the recipient device, the location derived from an identifier received with the video message and identifying the recipient of the video message;

transmitting the video message to the one or more recipient devices so that at least a portion of the video message can be rendered at the same time as the sending device is sending the video message;

providing encryption and authentication of the video message; and

storing the video message and transmitting the video message out of storage to the one or more recipient devices so that the video message can be rendered in a time-shifted mode.