System and method for video analysis

• US 10,223,581 B2
• Filed: 12/27/2015
• Issued: 03/05/2019
• Est. Priority Date: 12/27/2015
• Status: Active Grant

• Current Assignee: Genesys Telecommunications Laboratories Incorporated (Genesys Cloud Services Incorporated)
• Original Assignee: Interactive Intelligence Group Incorporated (Genesys Cloud Services Incorporated)
• Inventors: Walsh, Richard J., Ertel, II, David Allan
• Primary Examiner(s): Mehta, Bhavesh M
• Assistant Examiner(s): Lemieux, Ian L
• Application Number:
  US14/979,414
• Publication Number:
  US 20170185829A1
• Time in Patent Office: 1,164 Days
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: G06V 40/25 Recognition of walking or running movements, e.g. gait recognition
• CPC Class Codes: G09B 19/04 Speaking with audible presentation of the material to be studied G09B5/04
• CPC Class Codes: G10L 25/57 for processing of video signals
• CPC Class Codes: G10L 25/63 for estimating an emotional state

Abstract

A system and method are presented for video analysis. In an embodiment, video analysis may be applied in a contact center environment to aid contact center agents in improving the effectiveness of interactions. Feedback may be provided in real time to an agent on their interaction. Body mechanics may be examined and rated. In an embodiment, body mechanics may be combined with indicators which inform an agent on their performance. Tips may also be provided to an agent on how to improve performance in the form of audio overlay, video overlay, reports, or any combination of these. Feedback may also be used to improve the routing of communications within the contact center. Static and dynamic visual characteristics may also be used to improve routing decisions.

Claims

1. A method for analysis of a video stream in a contact center system, the method comprising:

a. receiving, by a video server, a first video stream and a second video stream;

b. providing, by the video server, the second video stream to a video analysis system comprising;

an independent video processing system configured to analyze individual frames of video streams in real-time and extract a plurality of visual characteristics of a communicant in a video stream; and

a voice analysis component configured to analyze audio accompanying a video stream and extract a plurality of audio characteristics of a communicant in a video stream, and

wherein the video analysis system is configured to send events based on the analysis back to the contact center system;

c. analyzing, automatically by the video analysis system, using a real-time video processing algorithm, each of a plurality of extracted visual and audio characteristics of a communicant in the second video stream, as the communicant in the second video stream interacts with a communicant from the first video stream, and automatically combining the plurality of characteristics to determine combined characteristics of the communicant in the second video stream; and

d. providing, by the video analysis system, real-time feedback regarding the characteristics of the communicant in the second video stream to the communicant in the second video stream based on the combined characteristics of the communicant in the second video stream.

2. The method of claim 1, wherein the characteristics comprise at least one of:

eye contact, head positioning, posture, facial expressions, speech amplitude, background, clothing, aesthetics of the communicant, speech cadence, speech pitch, and speech tone.

3. The method of claim 1, wherein the communicant in the second video stream comprises a contact center agent.

4. The method of claim 1, wherein the first video stream comprises a customer'"'"'s outgoing video stream.

5. The method of claim 1, wherein the second video stream comprises a contact center agent'"'"'s video stream.

6. The method of claim 1, wherein the analyzing further comprises comparing the characteristics to desired characteristics that have been previously identified as successful.

7. The method of claim 1, wherein the feedback is provided to the communicant in the second video stream via at least one of:

via a GUI, a feedback system, an email, a text message, or other communication.

8. The method of claim 7, wherein the feedback comprises one or more of:

indicator bars, skeleton overlays, and geometric mask.

9. The method of claim 1, wherein the method further comprises providing the feedback to a third party.

10. The method of claim 1, wherein the method further comprises automatically re-routing a communicant in the first video stream to a third communicant based on the analyzing of the communicant from the second video stream in step (c).

11. A method for analysis of a video stream in a contact center system, the method comprising:

a. receiving, by a video server, a plurality of video streams at the video server;

b. providing, by the video server, the plurality of video streams to a video analysis system;

c. analyzing, by the video analysis system, using a real-time video processing algorithm, characteristics of a communicant in each of the plurality of video streams extracted from the plurality of video streams;

d. providing, by the video analysis system, real-time feedback regarding the characteristics of the communicant in each of the plurality of video streams to at least one of the communicants; and

e. determining, by the contact center system, a routing decision of the communicant in a first video stream to an agent based on the analyzing of the communicant in each of the plurality of video streams in step (c);

f. automatically routing, by the contact center system, the communicant in the first video stream to the agent identified based on the analyzing of the communicant in each of the plurality of video streams in step (c).

12. The method of claim 11, wherein the characteristics comprise at least one of:

eye contact, head positioning, posture, facial expressions, speech amplitude, background, clothing, aesthetics of the communicant, speech cadence, speech pitch, and speech tone.

13. The method of claim 11, wherein the video analysis system comprises one or more of:

an independent video processing system that is capable of analyzing individual frames of the video streams in real-time and a voice analysis component, and wherein the video analysis system is configured to send events based on the analysis back to the contact center system.

14. The method of claim 11, wherein the communicant in at least one of the plurality of video streams comprises a contact center agent.

15. The method of claim 11, wherein at least one of the plurality of video streams comprises a customer'"'"'s outgoing video stream.

16. The method of claim 11, wherein at least one of the plurality of video streams comprises a contact center agent'"'"'s video stream.

17. The method of claim 11, wherein the analyzing further comprises comparing the characteristics to desired characteristics that have been previously identified as successful.

18. The method of claim 11, wherein the feedback is provided to the communicant via at least one of:

via a GUI, a feedback system, an email, a text message, or other communication.

19. The method of claim 18, wherein the feedback comprises one or more of:

indicator bars, skeleton overlays, and geometric mask.

20. The method of claim 11, wherein the method further comprises providing the feedback to a third party.