Determining customer service quality through digitized voice characteristic measurement and filtering

• US 10,244,113 B2
• Filed: 04/26/2016
• Issued: 03/26/2019
• Est. Priority Date: 04/26/2016
• Status: Active Grant

• Current Assignee: FMR LLC
• Original Assignee: FMR LLC
• Inventors: Kao, Jason, Sheng, Xinxin, Omidfar, Bahram, Zheng, Erkang
• Primary Examiner(s): Phan, Joseph T
• Application Number:
  US15/139,188
• Publication Number:
  US 20170310820A1
• Time in Patent Office: 1,064 Days
• CPC Class Codes: G10L 25/21 the extracted parameters being power information
• CPC Class Codes: G10L 25/51 for comparison or discrimination
• CPC Class Codes: G10L 25/63 for estimating an emotional state
• CPC Class Codes: G10L 25/87 Detection of discrete points within a voice signal
• CPC Class Codes: G10L 25/90 Pitch determination of speech signals
• CPC Class Codes: H04M 2201/40 using speech recognition speech recognition per se G10L15/00
• CPC Class Codes: H04M 2203/2038 Call context notifications
• CPC Class Codes: H04M 2203/555 Statistics, e.g. about subscribers but not being call statistics
• CPC Class Codes: H04M 2203/556 Statistical analysis and interpretation
• CPC Class Codes: H04M 3/5175 Call or contact centers supervision arrangements

Abstract

Methods and apparatuses are described for determining customer service quality through digitized voice characteristic measurement and filtering. A voice analysis module captures a first digitized voice segment corresponding to speech submitted by a user of a remote device. The voice analysis module extracts a first set of voice features from the first voice segment, and determines an emotion level of the user based upon the first set of voice features. The voice analysis module captures a second digitized voice segment corresponding to speech submitted by the user. The voice analysis module extracts a second set of voice features from the second voice segment, and determines a change in the emotion level of the user by comparing the first set of voice features to the second set of voice features. The module normalizes the change in the emotion level of the user using emotion influence factors, and generates a service score.

Claims

1. A computerized method for determining customer service quality through digitized voice characteristic measurement and filtering, the method comprising:

capturing, by a voice analysis module of a computing device, a first bitstream containing a first digitized voice segment from a remote device, the first digitized voice segment corresponding to speech submitted by a user of the remote device during a voice call;

adjusting, by the voice analysis module, compression of the first bitstream containing the first digitized voice segment to enhance audio quality of the first bitstream;

extracting, by the voice analysis module, a first set of voice features of the user from the first digitized voice segment by analyzing one or more of a spectrogram or a waveform of the first digitized voice segment to determine quantifiable characteristics associated with the user'"'"'s voice and assigning the determined quantifiable characteristics as the first set of voice features;

determining, by the voice analysis module, an emotion level of the user by comparing the first set of voice features to one or more voice features associated with a baseline voice profile previously generated for the user;

capturing, by the voice analysis module, a second bitstream containing a second digitized voice segment from the remote device, the second digitized voice segment corresponding to speech submitted by the user during the voice call;

adjusting, by the voice analysis module, compression of the second bitstream containing the second digitized voice segment to enhance audio quality of the second bitstream;

extracting, by the voice analysis module, a second set of voice features of the user from the second digitized voice segment by analyzing one or more of a spectrogram or a waveform of the second digitized voice segment to determine quantifiable characteristics associated with the user'"'"'s voice and assigning the determined quantifiable characteristics as the second set of voice features;

determining, by the voice analysis module, a change in the emotion level of the user by comparing the first set of voice features to the second set of voice features;

normalizing, by the voice analysis module, the change in the emotion level of the user using one or more environmental emotion influence factors by;

determining, by the voice analysis module, whether one or more of the environmental emotion influence factors applies to the voice call;

determining, by the voice analysis module, an emotion influence value attributable to the applied environmental emotion influence factors; and

adjusting, by the voice analysis module, the change in the emotion level of the user based upon the emotion influence value;

generating, by the voice analysis module, a customer service score for the voice call based upon the normalized change in the emotion level of the user; and

displaying, by the voice analysis module on a display device coupled to the computing device periodically during the voice call, a current emotion level of the user based upon the determined change in the emotion level.

2. The method of claim 1, wherein the first bitstream is captured from the remote device at a first time during the voice call and the second bitstream is captured from the remote device at a second time during the voice call.

3. The method of claim 2, wherein the first time is the beginning of the voice call and the second time is the end of the voice call.

4. The method of claim 1, further comprising:

capturing, by the voice analysis module, a plurality of other bitstreams containing other digitized voice segments from the remote device the plurality of voice segments each corresponding to speech submitted by a user of the remote device during a prior voice call;

adjusting, by the voice analysis module, compression of the plurality of bitstreams containing the digitized voice segments to enhance audio quality of the plurality of bitstreams;

extracting, by the voice analysis module, voice features of the user from each of the digitized voice segments by analyzing one or more of a spectrogram or a waveform of each of the digitized voice segments to determine quantifiable characteristics associated with the user'"'"'s voice and assigning the determined quantifiable characteristics as the second set of voice features; and

generating, by the voice analysis module, the baseline voice profile for the user based upon the extracted voice features.

5. The method of claim 1, wherein the first set of voice features and the second set of voice features include pitch contour, jitter, loudness contour, shimmer contour, utterance duration, audible portions, inaudible portions, and combinations thereof.

6. The method of claim 5, wherein the emotions represented by the emotion level of the user include anger, sadness, happiness, boredom, and nervousness.

7. The method of claim 1, wherein the voice analysis module determines whether one or more of the environmental emotion influence factor applies to the voice call based upon one or more of:

a date of the voice call, an identity of a customer service representative participating in the voice call, a topic of the voice call, or an external event to the voice call.

8. The method of claim 1, wherein the emotion influence value is determined based upon analyzing a plurality of other voice calls over a predefined time period that share one or more of the environmental emotion influence factors applied to the voice call.

9. The method of claim 1, wherein the remote device is an interactive voice response system.

10. A system for determining customer service quality through digitized voice characteristic measurement and filtering, the system comprising a computing device having a memory for storing a voice analysis module and a processor that executes the voice analysis module to:

capture a first bitstream containing a first digitized voice segment from a remote device, the first digitized voice segment corresponding to speech submitted by a user of the remote device during a voice call;

adjust compression of the first bitstream containing the first digitized voice segment to enhance audio quality of the first bitstream;

extract a first set of voice features of the user from the first digitized voice segment by analyzing one or more of a spectrogram or a waveform of the first digitized voice segment to determine quantifiable characteristics associated with the user'"'"'s voice and assigning the determined quantifiable characteristics as the first set of voice features;

determine an emotion level of the user by comparing the first set of voice features to one or more voice features associated with a baseline voice profile previously generated for the user;

capture a second bitstream containing a second digitized voice segment from the remote device, the second digitized voice segment corresponding to speech submitted by the user during the voice call;

adjust compression of the second bitstream containing the first digitized voice segment to enhance audio quality of the second bitstream;

extract a second set of voice features of the user from the second digitized voice segment by analyzing one or more of a spectrogram or a waveform of the second digitized voice segment to determine quantifiable characteristics associated with the user'"'"'s voice and assigning the determined quantifiable characteristics as the second set of voice features;

determine a change in the emotion level of the user by comparing the first set of voice features to the second set of voice features;

normalize the change in the emotion level of the user using one or more environmental emotion influence factors by;

determining whether one or more of the environmental emotion influence factors applies to the voice call;

determining an emotion influence value attributable to the applied environmental emotion influence factors; and

adjusting the change in the emotion level of the user based upon the emotion influence value;

generate a customer service score for the voice call based upon the normalized change in the emotion level of the user; and

display, on a display device coupled to the computing device periodically during the voice call, a current emotion level of the user based upon the determined change in the emotion level.

11. The system of claim 10, wherein the first bitstream is captured from the remote device at a first time during the voice call and the second bitstream is captured from the remote device at a second time during the voice call.

12. The system of claim 11, wherein the first time is the beginning of the voice call and the second time is the end of the voice call.

13. The system of claim 10, wherein the processor further executes the voice analysis module to:

capture a plurality of other bitstreams containing other digitized voice segments from the remote device the plurality of voice segments each corresponding to speech submitted by a user of the remote device during a prior voice call;

adjust compression of the plurality of bitstreams containing the digitized voice segments to enhance audio quality of the plurality of bitstreams;

extract voice features of the user from each of digitized voice segments by analyzing one or more of a spectrogram or a waveform of each of the digitized voice segments to determine quantifiable characteristics associated with the user'"'"'s voice and assigning the determined quantifiable characteristics as the second set of voice features; and

generating, by the voice analysis module, the baseline voice profile for the user based upon the extracted voice features;

analyze each of the digitized voice segments for voice features of the user; and

determine a change in the emotion level of the user by comparing the voice features of the user for each of the digitized voice segments.

14. The system of claim 10, wherein the first set of voice features and the second set of voice features include pitch contour, jitter, loudness contour, shimmer contour, utterance duration, audible portions, inaudible portions, and combinations thereof.

15. The system of claim 14, wherein emotions represented by the emotion level of the user include anger, sadness, happiness, boredom, and nervousness.

16. The system of claim 10, wherein the voice analysis module determines whether one or more of the environmental emotion influence factors applies to the voice call based upon one or more of:

a date of the voice call, an identity of a customer service representative participating in the voice call, a topic of the voice call, or an external event to the voice call.

17. The system of claim 10, wherein the voice analysis module determines the emotion influence value based upon analyzing a plurality of other voice calls over a predefined time period that share one or more of the environmental emotion influence factors applied to the voice call.

18. The system of claim 10, wherein the remote device is an interactive voice response system.