Multi-modal modeling of temporal interaction sequences
First Claim
Patent Images
1. A method for assessing an interaction involving at least two participants, at least one of the participants being a person, the method comprising, with a computing system:
- detecting, from multi-modal data captured by at least one sensing device during the interaction, a plurality of different behavioral cues expressed by the participants;
analyzing the detected behavioral cues with respect to a plurality of different time scales, each of the time scales being defined by a time interval whose size is compared to the size of other time intervals of the interaction, wherein the plurality of different time scales comprise at least two of a short term time scale, a medium term time scale, and a long term time scale;
recognizing, using machine learning and based on the analysis of the detected behavioral cues, a temporal interaction sequence comprising a pattern of the behavioral cues corresponding to one or more of the time scales; and
deriving, from the temporal interaction sequence, an assessment of the efficacy of the interaction;
wherein at least one indication of the assessment is provided through a device to a user or an application of the computing system.
1 Assignment
0 Petitions
Accused Products
Abstract
A multi-modal interaction modeling system can model a number of different aspects of a human interaction across one or more temporal interaction sequences. Some versions of the system can generate assessments of the nature or quality of the interaction or portions thereof, which can be used to, among other things, provide assistance to one or more of the participants in the interaction.
10 Citations
24 Claims
-
1. A method for assessing an interaction involving at least two participants, at least one of the participants being a person, the method comprising, with a computing system:
-
detecting, from multi-modal data captured by at least one sensing device during the interaction, a plurality of different behavioral cues expressed by the participants; analyzing the detected behavioral cues with respect to a plurality of different time scales, each of the time scales being defined by a time interval whose size is compared to the size of other time intervals of the interaction, wherein the plurality of different time scales comprise at least two of a short term time scale, a medium term time scale, and a long term time scale; recognizing, using machine learning and based on the analysis of the detected behavioral cues, a temporal interaction sequence comprising a pattern of the behavioral cues corresponding to one or more of the time scales; and deriving, from the temporal interaction sequence, an assessment of the efficacy of the interaction; wherein at least one indication of the assessment is provided through a device to a user or an application of the computing system. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23)
-
-
24. A method for assessing an interaction involving at least two participants, at least one of the participants being a person, the method comprising, with a computing system:
-
detecting, from multi-modal data captured by at least one sensing device, a plurality of different behavioral cues expressed by the participants during the interaction, the behavioral cues comprising one or more non-verbal cues and verbal content; recognizing, using machine learning, a temporal interaction sequence comprising a pattern of the behavioral cues occurring over a time interval during the interaction, wherein the time interval corresponds to at least one time scale, the at least one time scale being one of a short term time scale, a medium term time scale, and a long term time scale; and deriving, from the temporal interaction sequence, an assessment of the efficacy of the interaction; wherein at least one indication of the assessment is provided through a device to a user or an application of the computing system.
-
Specification