DETERMINING CUSTOMER SERVICE QUALITY THROUGH DIGITIZED VOICE CHARACTERISTIC MEASUREMENT AND FILTERING

• US 20170310820A1
• Filed: 04/26/2016
• Published: 10/26/2017
• Est. Priority Date: 04/26/2016
• Status: Active Grant

• Current Assignee: FMR LLC
• Original Assignee: FMR LLC
• Inventors: Kao, Jason, Sheng, Xinxin, Omidfar, Bahram, Zheng, Erkang
• Granted Patent: US 10,244,113 B2
• Time in Patent Office: Days
• CPC Class Codes: G10L 25/21 the extracted parameters being power information
• CPC Class Codes: G10L 25/51 for comparison or discrimination
• CPC Class Codes: G10L 25/63 for estimating an emotional state
• CPC Class Codes: G10L 25/87 Detection of discrete points within a voice signal
• CPC Class Codes: G10L 25/90 Pitch determination of speech signals
• CPC Class Codes: H04M 2201/40 using speech recognition speech recognition per se G10L15/00
• CPC Class Codes: H04M 2203/2038 Call context notifications
• CPC Class Codes: H04M 2203/555 Statistics, e.g. about subscribers but not being call statistics
• CPC Class Codes: H04M 2203/556 Statistical analysis and interpretation
• CPC Class Codes: H04M 3/5175 Call or contact centers supervision arrangements

Abstract

Methods and apparatuses are described for determining customer service quality through digitized voice characteristic measurement and filtering. A voice analysis module captures a first digitized voice segment corresponding to speech submitted by a user of a remote device. The voice analysis module extracts a first set of voice features from the first voice segment, and determines an emotion level of the user based upon the first set of voice features. The voice analysis module captures a second digitized voice segment corresponding to speech submitted by the user. The voice analysis module extracts a second set of voice features from the second voice segment, and determines a change in the emotion level of the user by comparing the first set of voice features to the second set of voice features. The module normalizes the change in the emotion level of the user using emotion influence factors, and generates a service score.

Claims

1. A computerized method for determining customer service quality through digitized voice characteristic measurement and filtering, the method comprising:

capturing, by a voice analysis module of a computing device, a first digitized voice segment from a remote device, the first digitized voice segment corresponding to speech submitted by a user of the remote device during a voice call;

extracting, by the voice analysis module, a first set of voice features of the user from the first digitized voice segment;

determining, by the voice analysis module, an emotion level of the user based upon the first set of voice features;

capturing, by the voice analysis module, a second digitized voice segment from the remote device, the second digitized voice segment corresponding to speech submitted by the user during the voice call;

extracting, by the voice analysis module, a second set of voice features of the user from the second digitized voice segment;

determining, by the voice analysis module, a change in the emotion level of the user by comparing the first set of voice features to the second set of voice features;

normalizing, by the voice analysis module, the change in the emotion level of the user using one or more emotion influence factors; and

generating, by the voice analysis module, a customer service score for the voice call based upon the normalized change in the emotion level of the user.

2. The method of claim 1, wherein the first digitized voice segment is captured from the remote device at a first time during the voice call and the second digitized voice segment is captured from the remote device at a second time during the voice call.

3. The method of claim 2, wherein the first time is the beginning of the voice call and the second time is the end of the voice call.

4. The method of claim 1, further comprising:

capturing, by the voice analysis module, a plurality of digitized voice segments, including the first digitized voice segment and the second digitized voice segment, continuously during the voice call;

extracting, by the voice analysis module, voice features of the user from each of the digitized voice segments; and

determining, by the voice analysis module, a change in the emotion level of the user by comparing the voice features of the user for each of the digitized voice segments.

5. The method of claim 1, wherein the first set of voice features and the second set of voice features include pitch contour, jitter, loudness contour, shimmer contour, utterance duration, audible portions, inaudible portions, and combinations thereof.

6. The method of claim 5, wherein the emotions represented by the emotion level of the user include anger, sadness, happiness, boredom, and nervousness.

7. The method of claim 1, further comprising:

calibrating, by the voice analysis module, the first digitized voice segment based upon a baseline voice profile for the user.

8. The method of claim 7, wherein the baseline voice profile is generated using the first digitized voice segment.

9. The method of claim 7, wherein the step of normalizing the change in the emotion level of the user using one or more emotion influence factors comprises:

determining, by the voice analysis module, whether an emotion influence factor applies to the voice call;

determining, by the voice analysis module, an emotion influence value attributable to the applied emotion influence factor; and

adjusting, by the voice analysis module, the change in the emotion level of the user based upon the emotion influence value.

10. The method of claim 9, wherein the voice analysis module determines whether an emotion influence factor applies to the voice call based upon one or more of:

a date of the voice call, an identity of the user, an identity of a customer service representative participating in the voice call, a topic of the voice call, and an external event to the voice call.

11. The method of claim 9, wherein the emotion influence value is determined based upon analyzing a plurality of other voice calls over a predefined time period that share the emotion influence factor.

12. The method of claim 1, wherein the remote device is an interactive voice response system.

13. A system for determining customer service quality through digitized voice characteristic measurement and filtering, the system comprising a voice analysis module of a computing device, the voice analysis module configured to

capture a first digitized voice segment from a remote device, the first digitized voice segment corresponding to speech submitted by a user of the remote device during a voice call;

analyze the first digitized voice segment for a first set of voice features of the user;

determine an emotion level of the user based upon the first set of voice features;

capture a second digitized voice segment from the remote device, the second digitized voice segment corresponding to speech submitted by the user during the voice call;

analyze the second digitized voice segment for a second set of voice features of the user;

determine a change in the emotion level of the user by comparing the first set of voice features to the second set of voice features;

normalize the change in the emotion level of the user using one or more emotion influence factors; and

generate a customer service score for the voice call based upon the normalized change in the emotion level of the user.

14. The system of claim 13, wherein the first digitized voice segment is captured from the remote device at a first time during the voice call and the second digitized voice segment is captured from the remote device at a second time during the voice call.

15. The system of claim 14, wherein the first time is the beginning of the voice call and the second time is the end of the voice call.

16. The system of claim 13, wherein the voice analysis module is further configured to

capture a plurality of digitized voice segments, including the first digitized voice segment and the second digitized voice segment, continuously during the voice call;

analyze each of the digitized voice segments for voice features of the user; and

determine a change in the emotion level of the user by comparing the voice features of the user for each of the digitized voice segments.

17. The system of claim 13, wherein the first set of voice features and the second set of voice features include pitch contour, jitter, loudness contour, shimmer contour, utterance duration, audible portions, inaudible portions, and combinations thereof.

18. The system of claim 17, wherein emotions represented by the emotion level of the user include anger, sadness, happiness, boredom, and nervousness.

19. The system of claim 13, wherein the voice analysis module is further configured to calibrate the first digitized voice segment based upon a baseline voice profile for the user.

20. The system of claim 19, wherein the baseline voice profile is generated using the first digitized voice segment.

21. The system of claim 13, wherein the voice analysis module normalizes the change in the emotion level of the user using one or more emotion influence factors by determining whether an emotion influence factor applies to the voice call;

determining an emotion influence value attributable to the applied emotion influence factor; and

adjusting the change in the emotion level of the user based upon the emotion influence value.

22. The system of claim 21, wherein the voice analysis module determines whether an emotion influence factor applies to the voice call based upon one or more of:

a date of the voice call, an identity of the user, an identity of a customer service representative participating in the voice call, a topic of the voice call, and an external event to the voice call.

23. The system of claim 21, wherein the voice analysis module determines the emotion influence value based upon analyzing a plurality of other voice calls over a predefined time period that share the emotion influence factor.

24. The system of claim 13, wherein the remote device is an interactive voice response system.

25. A computer program product, tangibly embodied in a non-transitory computer readable storage device, for determining customer service quality through digitized voice characteristic measurement and filtering, the computer program product including instructions operable to cause a voice analysis module of a computing device to

capture a first digitized voice segment from a remote device, the first digitized voice segment corresponding to speech submitted by a user of the remote device during a voice call;

analyze the first digitized voice segment for a first set of voice features of the user;

determine an emotion level of the user based upon the first set of voice features;

capture a second digitized voice segment from the remote device, the second digitized voice segment corresponding to speech submitted by the user during the voice call;

analyze the second digitized voice segment for a second set of voice features of the user;

determine a change in the emotion level of the user by comparing the first set of voice features to the second set of voice features;

normalize the change in the emotion level of the user using one or more emotion influence factors; and

generate a customer service score for the voice call based upon the normalized change in the emotion level of the user.