Method and system for adjusting user speech in a communication session

• US 9,558,756 B2
• Filed: 10/29/2013
• Issued: 01/31/2017
• Est. Priority Date: 10/29/2013
• Status: Active Grant

• Current Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: Wohlert, Randolph, Bangor, Aaron, Stockert, Mark
• Primary Examiner(s): Armstrong, Angela A
• Application Number:
  US14/065,903
• Publication Number:
  US 20150120293A1
• Time in Patent Office: 1,190 Days
• US Class Current: 1/1
• CPC Class Codes: G09B 21/00 Teaching, or communicating with, the blind, deaf or mute audible presentation of material to be studied G09B5/04; devices or methods for replacing direct visual or auditory perception by another kind of perception A61F9/08, A61F11/04; audible indication of meter readings or of colour G01D7/12; watches for blind persons G04B25/02; speech analysis, speech recognition G10L; sound-recording or reproducing, per se G11B
• CPC Class Codes: G09B 21/006 using audible presentation of the information
• CPC Class Codes: G09B 21/009 Teaching or communicating with deaf persons G09B21/04 takes precedence; deaf and dumb persons G09B21/00
• CPC Class Codes: G10L 13/02 Methods for producing synthetic speech; Speech synthesisers
• CPC Class Codes: G10L 13/033 Voice editing, e.g. manipulating the voice of the synthesiser
• CPC Class Codes: G10L 15/00 Speech recognition G10L17/00 takes precedence
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 21/003 Changing voice quality, e.g. pitch or formants
• CPC Class Codes: G10L 21/02 Speech enhancement, e.g. noise reduction or echo cancellation reducing echo effects in line transmission systems H04B3/20; echo suppression in hands-free telephones H04M9/08
• CPC Class Codes: G10L 21/0364 for improving intelligibility
• CPC Class Codes: G10L 21/10 Transforming into visible information
• CPC Class Codes: H04L 67/306 User profiles
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34

Abstract

A system that incorporates the subject disclosure may include, for example, receive user speech captured at a second end user device during a communication session between the second end user device and a first end user device, apply speech recognition to the user speech, identify an unclear word in the user speech based on the speech recognition, adjust the user speech to generate adjusted user speech by replacing all or a portion of the unclear word with replacement audio content, and provide the adjusted user speech to the first end user device during the communication session. Other embodiments are disclosed.

Claims

1. A method, comprising:

receiving, by a system including a processor, user speech captured at a second end user device during a communication session between the second end user device and a first end user device, wherein the user speech is received by the system from the second end user device;

applying, by the system, speech recognition to the user speech;

identifying, by the system, an unclear word in the user speech based on the speech recognition;

adjusting, by the system, the user speech to generate adjusted user speech by replacing a portion of the unclear word with replacement audio content without replacing a remainder of the unclear word, wherein the replacing of the portion of the unclear word includes replacing a dropped syllable of the unclear word; and

providing, by the system, the adjusted user speech to the first end user device during the communication session.

2. The method of claim 1, wherein the replacement audio content comprises synthesized speech and further comprising:

accessing, by the system, a speech profile for a second user of the second end user device; and

detecting, by the system, an undesirable speech trait associated with the second user according to the speech profile,

wherein the speech recognition is applied to the user speech responsive to the detecting of the undesirable speech trait.

3. The method of claim 2, comprising:

generating, by the system, the speech profile based on an analysis of a previous communication session associated with the second user, wherein the system comprises an application programming interface accessible by a remote device that is operated by a third party that is different from a service provider operating the system, and wherein the adjusting of the user speech is according to instructions stored and executed by the system which are received from the remote device.

4. The method of claim 1, comprising identifying, by the system, another unclear word in the user speech based on the speech recognition, wherein the adjusting of the user speech to generate the adjusted user speech includes replacing the other unclear word in its entirety with one of synthesized speech or recorded speech.

5. The method of claim 1, wherein the replacement audio content comprises recorded speech.

6. The method of claim 1, comprising:

accessing, by the system, a speech profile for a second user of the second end user device;

accessing, by the system, an impairment profile for a first user of the first end user device; and

detecting, by the system, at least one of an undesirable speech trait associated with the second user according to the speech profile or an impairment associated with the first user according to the impairment profile,

wherein the speech recognition is applied to the user speech responsive to the detecting of at least one of the undesirable speech trait or the impairment, and wherein the speech recognition is not applied to the user speech responsive to the undesirable speech trait and the impairment not being detected.

7. The method of claim 1, wherein the adjusting of the user speech to generate the adjusted user speech includes amplifying selective frequencies of the user speech.

8. The method of claim 7, comprising accessing, by the system, an impairment profile for a first user of the first end user device, wherein the impairment profile includes an audiogram for the first user, and wherein the selective frequencies that are amplified are selected based on the audiogram.

9. The method of claim 8, further comprising:

providing a hearing test at a communication device of the first user prior to the communication session; and

generating the audiogram based on the hearing test.

10. The method of claim 8, comprising obtaining the audiogram from a remote source operated by a third party that is different from a service provider operating the system.

11. The method of claim 1, comprising:

analyzing the user speech to generate undesirable speech data;

accessing a speech profile for the second user; and

adjusting the speech profile to indicate an undesirable speech trait associated with the second user according to the undesirable speech data.

12. A system, comprising:

a memory that stores executable instructions; and

a processor coupled with the memory, wherein the processor, responsive to executing the executable instructions, performs operations comprising;

receiving user speech captured at a second end user device during a communication session between a second user of the second end user device and a first user of a first end user device;

detecting at least one of an undesirable speech trait associated with the second user or an impairment associated with the first user;

applying speech recognition to the user speech responsive to the detecting of at least one of the undesirable speech trait or the impairment;

identifying an unclear word in the user speech based on the speech recognition;

storing instructions for executing a group of adjustment techniques for modifying the user speech, wherein the group of adjustment techniques includes providing replacement audio content and translating the other user speech into sign language images;

selecting an adjustment technique from among the group of adjustment techniques according to the at least one of the undesirable speech trait or the impairment;

responsive to the selected adjustment technique including the providing of the replacement audio content, adjusting the user speech to generate adjusted user speech by replacing at least a portion of the unclear word with the replacement audio content; and

providing the adjusted user speech to the first end user device during the communication session.

13. The system of claim 12, wherein the replacement audio content comprises synthesized speech, wherein the adjusting of the user speech comprises replacing a syllable of the unclear word with the synthesized speech without replacing a remainder of the unclear word, and wherein the processor comprises a plurality of processors operating in a distributed processing environment.

14. The system of claim 12, wherein the replacement audio content comprises recorded speech and wherein the operations further comprise obtaining the recorded speech from a previous communication session associated with the second user.

15. The system of claim 12, wherein the operations further comprise:

accessing a second profile for the second user of the second end user device; and

accessing a first profile for the first user of the first end user device, wherein the detecting of the at least one of the undesirable speech trait associated with the second user or the impairment associated with the first user is based on the second and first profiles, respectively, wherein the first profile includes an audiogram for the first user and wherein the operations further comprise;

generating the second profile based on an analysis of a previous communication session associated with the second user; and

obtaining the audiogram from one of a remote source or a hearing test performed by a communication device of the first user, wherein the remote source is operated by a third party that is different from a service provider operating the system.

16. The system of claim 12, comprising an application programming interface connected with the processor, wherein the application programming interface is accessible by a remote device that is operated by a third party that is different from a service provider operating the system, and wherein the adjusting of the user speech is according to instructions stored and executed by the processor which are received from the remote device.

17. The system of claim 12, wherein the operations further comprise:

determining another impairment of the second user;

receiving other user speech captured at the first end user device during the communication session;

selecting a second adjustment technique from among the group of adjustment techniques according to the other impairment of the second user;

adjusting the other user speech according to the second adjustment technique to generate adjusted other user speech; and

providing the adjusted other user speech to the second end user device during the communication session.

18. A computer-readable storage device comprising computer instructions which, responsive to being executed by a processor of a first end user device, causes the processor to perform operations comprising:

receiving adjusted user speech from a system that includes an application server during a communication session between the first end user device and a second end user device, wherein the adjusted user speech is generated from a modification of user speech captured at the second end user device during the communication session, wherein the modification is responsive to a detection of at least one of an impairment of a first user of the first end user device or an undesirable speech trait of a second user of the second end user device, wherein the modification includes identifying an unclear word utilizing speech recognition and replacing a portion of the unclear word with replacement audio content without replacing a remainder of the unclear word;

presenting the adjusted user speech at the first end user device;

capturing other user speech at the first end user device; and

providing the other user speech to the system to enable the system to generate adjusted other user speech responsive to a detection of another impairment of the second user of the second end user device and based on an adjustment technique selected by the system from among a group of adjustment techniques, wherein the system provides the adjusted other user speech to the second end user device for presentation during the communication session, wherein instructions are stored by the system for selectively executing the group of adjustment techniques, and wherein the group of adjustment techniques includes providing replacement audio content and translating the other user speech into sign language images.

19. The computer-readable storage device of claim 18, wherein the replacement audio content comprises recorded speech and wherein the recorded speech is obtained from a previous communication session of the second user.

20. The computer-readable storage device of claim 18, wherein the operations further comprise translating the other user speech into text, and wherein the processor comprises a plurality of processors operating in a distributed processing environment.