System for intelligible audio conversation over unreliable digital transmission media

• US 9,680,905 B2
• Filed: 12/23/2014
• Issued: 06/13/2017
• Est. Priority Date: 12/23/2013
• Status: Active Grant

• Current Assignee: David Arthur Yost
• Original Assignee: David Arthur Yost
• Inventors: Yost, David Arthur
• Primary Examiner(s): Patel, Parth
• Application Number:
  US14/580,661
• Publication Number:
  US 20150180932A1
• Time in Patent Office: 903 Days
• CPC Class Codes: H04L 65/80 Responding to QoS

Abstract

During unfavorable network conditions, a system provides a fallback mode that ensures conversation intelligibility and eliminates dropouts. In the fallback mode, the system uses data retransmission and mitigates retransmission delay by breaking input and output audio into segments for flow control and by involving the users in flow control. On the receive end, when necessary, the system pauses audio output and indicates the pause to the receiving user to inform the receiving user that the pause is a system artifact rather than silence from the transmitting user. On the transmit end, when necessary to avoid accumulation of excessive end-to-end delay, the system suspends audio input and indicates the suspension to the transmitting user to inform the transmitting user that audio input is temporarily being ignored. The transmit end of the system decides when to suspend audio input based on current end-to-end delay, which is continually communicated from the receive end.

Claims

1. A system for limiting accumulation of end-to-end delay (EED), which represents a time interval between a time of acceptance of input data from a transmitting user and a time of outputting of the data to a receiving user utilizing a remote system, after transmission of the data over a network, the system comprising:

a first transmission subsystem (TS) configured to transmit and receive data to and from the remote system over a network;

an audio input subsystem (AIS) configured to accept input data from a

transmitting user and transmit the accepted input data to the remote system via the first TS; and

an input flow-control subsystem (IFCS) configured to

continually receive EED updates from the remote system,

suspend audio input and indicate the suspension to the transmitting user via a suspension notification to inform the transmitting user that audio input is temporarily being ignored in order to avoid accumulation of excessive EED when the EED is determined to exceed the threshold based on the received EED updates, and

inform the transmitting user when the suspension notification is transmitted to the AIS,

wherein the AIS, upon receiving the suspension notification, is configured to stop accepting data from the transmitting user.

2. The system of claim 1, wherein the IFCS is configured to transmit a resume notification to the AIS after the EED is determined to no longer exceed the threshold, while the AIS is not accepting data from the transmitting user after receiving the suspension notification,

wherein the AIS is configured to resume receipt of data from the transmitting user upon receiving the resume notification, and

wherein the IFCS informs the transmitting user when the resume notification is transmitted to the AIS.

3. The system of claim 1, wherein the IFCS is configured to cease transmitting the suspension notification to the AIS after the EED is determined to no longer exceed the threshold, while the AIS is not accepting data from the transmitting user after receiving the suspension notification,

wherein the AIS is configured to resume acceptance of data from the transmitting user upon the IFCS ceasing transmission of the suspension notification, and

wherein the IFCS informs the transmitting user when transmission of the suspension notification to the AIS ceases.

4. The system of claim 1, comprising:

a second TS configured to transmit and receive data to and from the remote system over the network; and

an audio output subsystem (AOS) configured to

receive data from the first TS via the network and to output the received data to the receiving user,

calculate, at predetermined intervals, an EED representing a time interval between a time of acceptance of input data from the transmitting user by the remote system and the current time at which the system is outputting the data to the receiving user, and

generate EED updates of the calculated interval, and

transmit the generated EED updates to the first TS via the network.

5. A system for limiting accumulation of end-to-end delay (EED), which represents a time interval between a time of acceptance of input data from a transmitting user utilizing a remote system and a time of outputting of the data to a receiving user, after transmission of the data over a network, the system comprising:

a transmission subsystem (TS) configured to transmit and receive data to and from the remote system and a receiving user over the network; and

an audio output subsystem configured to

receive data from remote system via the TS and to output the received data to the receiving user,

calculate, at predetermined intervals, an EED representing a time interval between a time of acceptance of input data from the transmitting user by the remote system and the current time at which the system is outputting the data to the receiving user, and

generate EED updates of the calculated interval,

transmit the generated EED updates to the remote system via the TS; and

suspend audio input and indicate the suspension to the transmitting user via a suspension notification to inform the transmitting user that audio input is temporarily being ignored in order to avoid accumulation of excessive EED when the EED is determined to exceed the threshold based on the received EED updates.

6. A system for providing intelligible audio conversation over a transmission medium, the system comprising:

an audio input subsystem (AIS);

an audio output subsystem (AOS); and

a transmission subsystem configured to allow the AIS and AOS to communicate with each other utilizing a reliable message transport protocol (RMTP),

wherein the transmission subsystem provides a fallback mode that, during a time of network unreliability in which the network cannot reliably and timely deliver communications between the AIS and AOS, the transmission subsystem configures the

AIS and AOS to utilize data retransmission and mitigates retransmission delay by breaking input and output audio into segments for flow control and by involving a receiving user of the AOS and a transmitting user of the AIS in flow control,

wherein, during the time of network unreliability, the AOS is configured to pause audio output and indicate the pause to a receiving user via auditory and/or visual cues to inform the receiving user that the pause is a system artifact rather than silence from a transmitting user of the AIS,

wherein the AIS, in order to avoid accumulation of excessive end-to-end delay during the time of network unreliability, is configured to suspend audio input and indicate the suspension to the transmitting user via auditory and/or visual cues to inform the transmitting user that audio input is temporarily being ignored, and

wherein the AIS is configured to decide when to suspend audio input based on the current end-to-end delay, which is continually communicated from the AOS.