Emotion recognition in video conferencing
First Claim
1. A computer-implemented method for video conferencing, the method comprising:
- receiving a video including a sequence of images;
detecting at least one object of interest in one or more of the images;
locating feature reference points of the at least one object of interest;
aligning a virtual face mesh to the at least one object of interest in one or more of the images based at least in part on the feature reference points;
finding over the sequence of images at least one deformation of the virtual face mesh, wherein the at least one deformation is associated with at least one face mimic;
determining that the at least one deformation refers to a facial emotion selected from a plurality of reference facial emotions;
generating an emotional status of an individual based on the facial emotion selected from the plurality of reference facial emotions;
determining that the facial emotion is a negative facial emotion; and
generating a communication bearing data associated with the negative facial emotion only where the facial emotion is determined to be a negative facial emotion.
4 Assignments
0 Petitions
Accused Products
Abstract
Methods and systems for videoconferencing include recognition of emotions related to one videoconference participant such as a customer. This ultimately enables another videoconference participant, such as a service provider or supervisor, to handle angry, annoyed, or distressed customers. One example method includes the steps of receiving a video that includes a sequence of images, detecting at least one object of interest (e.g., a face), locating feature reference points of the at least one object of interest, aligning a virtual face mesh to the at least one object of interest based on the feature reference points, finding over the sequence of images at least one deformation of the virtual face mesh that reflect face mimics, determining that the at least one deformation refers to a facial emotion selected from a plurality of reference facial emotions, and generating a communication bearing data associated with the facial emotion.
-
Citations
22 Claims
-
1. A computer-implemented method for video conferencing, the method comprising:
-
receiving a video including a sequence of images; detecting at least one object of interest in one or more of the images; locating feature reference points of the at least one object of interest; aligning a virtual face mesh to the at least one object of interest in one or more of the images based at least in part on the feature reference points; finding over the sequence of images at least one deformation of the virtual face mesh, wherein the at least one deformation is associated with at least one face mimic; determining that the at least one deformation refers to a facial emotion selected from a plurality of reference facial emotions; generating an emotional status of an individual based on the facial emotion selected from the plurality of reference facial emotions; determining that the facial emotion is a negative facial emotion; and generating a communication bearing data associated with the negative facial emotion only where the facial emotion is determined to be a negative facial emotion. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20)
-
-
21. A system, comprising:
a computing device including at least one processor and a memory storing processor-executable codes, which, when implemented by the at least one processor, cause to perform the steps of; receiving a video including a sequence of images; detecting at least one object of interest in one or more of the images; locating feature reference points of the at least one object of interest; aligning a virtual face mesh to the at least one object of interest in one or more of the images based at least in part on the feature reference points; finding over the sequence of images at least one deformation of the virtual face mesh, wherein the at least one deformation is associated with at least one face mimic; determining that the at least one deformation refers to a facial emotion selected from a plurality of reference facial emotions; generating an emotional status of an individual based on the facial emotion selected from the plurality of reference facial emotions; determining that the facial emotion is a negative facial emotion; and generating a communication bearing data associated with the negative facial emotion only where the facial emotion is determined to be a negative facial emotion.
-
22. A non-transitory processor-readable medium having instructions stored thereon, which when executed by one or more processors, cause the one or more processors to implement a method, comprising:
-
receiving a video including a sequence of images; detecting at least one object of interest in one or more of the images; locating feature reference points of the at least one object of interest; aligning a virtual face mesh to the at least one object of interest in one or more of the images based at least in part on the feature reference points; finding over the sequence of images at least one deformation of the virtual face mesh, wherein the at least one deformation is associated with at least one face mimic; determining that the at least one deformation refers to a facial emotion selected from a plurality of reference facial emotions; generating an emotional status of an individual based on the facial emotion selected from the plurality of reference facial emotions; determining that the facial emotion is a negative facial emotion; and generating a communication bearing data associated with the negative facial emotion only where the facial emotion is determined to be a negative facial emotion.
-
Specification