MULTI-MODAL MODELING OF TEMPORAL INTERACTION SEQUENCES
First Claim
Patent Images
1. A method for assessing an interaction involving at least two participants, at least one of the participants being a person, the method comprising, with a computing system:
- detecting, from multi-modal data captured by at least one sensing device during the interaction, a plurality of different behavioral cues expressed by the participants;
analyzing the detected behavioral cues with respect to a plurality of different time scales, each of the time scales being defined by a time interval whose size can be compared to the size of other time intervals of the interaction;
recognizing, based on the analysis of the detected behavioral cues, a temporal interaction sequence comprising a pattern of the behavioral cues corresponding to one or more of the time scales; and
deriving, from the temporal interaction sequence, an assessment of the efficacy of the interaction.
1 Assignment
0 Petitions
Accused Products
Abstract
A multi-modal interaction modeling system can model a number of different aspects of a human interaction across one or more temporal interaction sequences. Some versions of the system can generate assessments of the nature or quality of the interaction or portions thereof, which can be used to, among other things, provide assistance to one or more of the participants in the interaction.
-
Citations
37 Claims
-
1. A method for assessing an interaction involving at least two participants, at least one of the participants being a person, the method comprising, with a computing system:
-
detecting, from multi-modal data captured by at least one sensing device during the interaction, a plurality of different behavioral cues expressed by the participants; analyzing the detected behavioral cues with respect to a plurality of different time scales, each of the time scales being defined by a time interval whose size can be compared to the size of other time intervals of the interaction; recognizing, based on the analysis of the detected behavioral cues, a temporal interaction sequence comprising a pattern of the behavioral cues corresponding to one or more of the time scales; and deriving, from the temporal interaction sequence, an assessment of the efficacy of the interaction. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23)
-
-
24. A method for assessing an interaction involving at least two participants, at least one of the participants being a person, the method comprising, with a computing system:
-
detecting, from multi-modal data captured by at least one sensing device, a plurality of different behavioral cues expressed by the participants during the interaction, the behavioral cues comprising one or more non-verbal cues and verbal content; recognizing a temporal interaction sequence comprising a pattern of the behavioral cues occurring over a time interval during the interaction; and deriving, from the temporal interaction sequence, an assessment of the efficacy of the interaction.
-
-
25. A method for assessing a person'"'"'s emotional state during an interaction involving the person and at least one other participant, the method comprising, with a computing system:
-
detecting, from multi-modal data captured by one or more sensing devices, a plurality of different behavioral cues expressed by the participants during the interaction; recognizing a plurality of temporal interaction sequences, each temporal interaction sequence comprising a pattern of the behavioral cues occurring over a time interval during the interaction, and at least one of the temporal interaction sequences involving behavioral cues of the person and behavioral cues of at least one other participant; assessing the person'"'"'s emotional state during each of the temporal interaction sequences based on the behavioral cues involved in the temporal interaction sequence; determining a relative significance of each of the temporal interaction sequences to the interaction; and assessing the person'"'"'s emotional state during the interaction as a whole based on the person'"'"'s assessed emotional state during each of the temporal interaction sequences and the determined relative significance of each of the temporal interaction sequences to the interaction as a whole. - View Dependent Claims (26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37)
-
Specification