Method and system for mitigating delay in receiving audio stream during production of sound from audio stream

• US 9,978,395 B2
• Filed: 03/15/2013
• Issued: 05/22/2018
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Vocollect, Inc. (Honeywell International Inc.)
• Original Assignee: Vocollect, Inc. (Honeywell International Inc.)
• Inventors: Braho, Keith, Barr, Russell A., Karabin, George Joshue
• Primary Examiner(s): Fischer, Mark
• Application Number:
  US13/835,638
• Publication Number:
  US 20140270196A1
• Time in Patent Office: 1,894 Days
• CPC Class Codes: G10L 13/08 Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
• CPC Class Codes: G10L 21/047 characterised by the type of waveform to be thinned out or inserted
• CPC Class Codes: H04R 2201/107 Monophonic and stereophonic headphones with microphone for two-way hands free communication

Abstract

A communication component modifies production of an audio waveform at determined modification segments to thereby mitigate the effects of a delay in processing and/or receiving a subsequent audio waveform. The audio waveform and/or data associated with the audio waveform are analyzed to identify the modification segments based on characteristics of the audio waveform and/or data associated therewith. The modification segments show where the production of the audio waveform may be modified without substantially affecting the clarity of the sound or audio. In one embodiment, the invention modifies the sound production at the identified modification segments to extend production time and thereby mitigate the effects of delay in receiving and/or processing a subsequent audio waveform for production.

Claims

1. An apparatus comprising:

transceiver circuitry configured to receive an audio stream, the audio stream including an audio waveform;

a memory configured to store the received audio stream;

audio production circuitry configured to produce sound using the audio waveform;

processing circuitry configured to;

analyze the received audio stream and identify a modification segment of the audio waveform, the modification segment being a segment of the audio waveform where production of the audio waveform may be modified to mitigate a delay in receiving the audio stream by temporally extending the modification segment without substantially affecting clarity of the produced sound, and

drive production of sound using the audio waveform based at least in part on the modification segment that was identified;

wherein the audio stream includes metadata associated with the audio waveform that indicates a position of a specific type of sound included in the audio waveform, and the processing circuitry is configured to analyze the associated metadata to identify the modification segment having the position within the specific type of sound; and

wherein the specific type of sound is phonemes having natural pauses, phonemes having voiceless glottal plosives, phonemes related to vowels, phonemes related to fricatives, quasi-stationary audio waveform segments of phonemes, middle audio waveform segments of phonemes, lip positions having natural pauses, or lip positions having voiceless glottal plosives.

2. The apparatus of claim 1 wherein the processing circuitry is configured to extend the audio waveform at the identified modification segment.

3. The apparatus of claim 2 wherein the processing circuitry is configured to analyze remaining time to produce sound using a received audio waveform and the expected time to receive a subsequent portion of an audio stream and to determine the duration for the extension of the audio waveform.

4. The apparatus of claim 2, wherein the processing circuitry is configured to extend the audio waveform by pausing production of sound at the identified modification segment of the audio waveform for a desired time period.

5. The apparatus of claim 2, wherein the processing circuitry is configured to extend the audio waveform by repeating the identified modification segment to extend the sound represented by the identified modification segment.

6. The apparatus of claim 1, wherein the identified modification segment corresponds to a segment of low amplitude in the audio waveform.

7. The apparatus of claim 1 wherein the processing circuitry is configured to drive production of sound by delaying production of sound until the modification segment is identified.

8. The apparatus of claim 1, wherein the identified modification segment corresponds to a segment of the audio waveform where the audio waveform is quasi-stationary.

9. The apparatus of claim 1, the processing circuitry being further configured to:

determine whether production of sound using the audio waveform will end before a subsequent portion of the audio stream is expected to be received, and drive production of sound using the audio waveform based at least in part on the processing circuitry determining that the production of sound using the audio waveform will end before a subsequent portion of the audio stream is expected to be received.

10. The apparatus of claim 1, the processing circuitry being further configured to:

determine whether production of sound using the audio waveform will end before a subsequent portion of the audio stream with an identified modification segment is expected to be received, and

drive production of sound using the audio waveform based at least in part on the processing circuitry determining that the production of sound using the audio waveform will end before a subsequent portion of the audio stream with an identified modification segment is expected to be received.

11. A system comprising:

a transmitting device for transmitting an audio stream including an audio waveform;

a receiving device for receiving the audio stream including audio production circuitry configured to produce sound using the audio waveform of the audio stream;

processing circuitry of the transmitting device configured to analyze the audio stream and identify a modification segment of the audio waveform, the modification segment being a segment of the audio waveform where production of the audio waveform may be modified to mitigate a delay when the receiving device receives the audio stream by temporally extending the modification segment without substantially affecting clarify of the produced sound; and

processing circuitry of the receiving device configured for driving the production of sound using the audio waveform based at least in part on the modification segment that was identified;

wherein the audio stream includes metadata associated with the audio waveform that indicates a position of a specific type of sound included in the audio waveform;

wherein the processing circuitry of the transmitting device is configured to analyze the associated metadata and identify modification segment having the position within the specific type of sound; and

wherein the specific type of sound is phonemes having natural pauses, phonemes having voiceless glottal plosives, phonemes related to vowels, phonemes related to fricatives, quasi-stationary audio waveform segments of phonemes, middle audio waveform segments of phonemes, lip positions having natural pauses, or lip positions having voiceless glottal plosives.

12. The system of claim 11 wherein the processing circuitry of the receiving device is configured to extend the audio waveform at the identified modification segment.

13. The system of claim 12, wherein the processing circuitry is configured to extend the audio waveform by pausing production of sound at the identified modification segment of the audio waveform for a desired time period.

14. The system of claim 12, wherein the processing circuitry is configured to extend the audio waveform by repeating the identified modification segment to extend the sound represented by the identified modification segment.

15. The system of claim 11, wherein the identified modification segment corresponds to a segment of low amplitude in the audio waveform.

16. The system of claim 11, wherein the identified modification segment corresponds to a segment of the audio waveform where the audio waveform is quasi-stationary.

17. A method of producing sound from an audio waveform, the audio waveform being included in a received audio stream, the method comprising:

analyzing the audio stream to identify a modification segment of the audio waveform, the modification segment being a segment of the audio waveform where production of the audio waveform may be modified to mitigate a delay in receiving the received the audio stream by temporally extending the modification segment without substantially affecting clarity of the produced sound;

producing sound using the audio waveform based at least in part on the modification segment that was identified;

wherein the audio stream includes metadata associated with the audio waveform that indicates a position of a specific type of sound included in the audio waveform;

analyzing the associated metadata; and

identifying the modification segment having the position within the specific type of sound, the specific type of sound being phonemes having natural pauses, phonemes having voiceless glottal plosives, phonemes related to vowels, phonemes related to fricatives, quasi-stationary audio waveform segments of phonemes, middle audio waveform segments of phonemes, lip positions having natural pauses, or lip positions having voiceless glottal plosives.

18. The method of claim 17 further comprising extending the audio waveform at the identified modification segment and producing sound using the extended audio waveform.

19. The method of claim 18 further comprising analyzing remaining time to produce sound using the received audio waveform and the expected time to receive a subsequent portion of an audio stream and to determine the duration for the extension of the audio waveform.

20. The method of claim 18 including pausing production of sound at the identified modification portion of the audio waveform for a desired time period to extend the audio waveform.

21. The method of claim 18 including extending the waveform by repeating the identified modification segment to extend the sound represented by the identified modification segment.

22. The method of claim 17, wherein analyzing the audio stream includes analyzing the audio waveform to determine a segment of the audio waveform having a low amplitude, and identifying a segment of low amplitude as a modification segment.

23. The method of claim 17, wherein analyzing the audio stream includes analyzing the audio waveform to determine a segment of the audio waveform where the audio waveform is quasi-stationary, and identifying a quasi-stationary segment as a modification segment.

24. The method of claim 17, further comprising:

determining whether production of sound using the audio waveform of the received audio stream will end before a subsequent portion of the audio stream is expected to be received; and

producing the sound using the audio waveform based at least in part on whether production of sound using the audio waveform will end before a subsequent portion of the audio stream is expected to be received.