MULTI-MODAL MODELING OF TEMPORAL INTERACTION SEQUENCES
First Claim
Patent Images
1. A method for predicting a behavioral event to occur in an interaction involving at least two participants, at least one of the participants being a person, the method comprising, with a computing system:
- detecting, from multi-modal data captured by one or more sensing devices, a plurality of different behavioral cues expressed by the participants during the interaction;
recognizing a plurality of temporal interaction sequences, each of the temporal interaction sequences occurring over a time interval during the interaction, and involving a pattern of the behavioral cues, and at least two of the participants being involved in at least one of the temporal interaction sequences;
determining the nature of the interaction based on the recognized temporal interaction sequences; and
predicting the behavioral event to occur in the interaction based on the determined nature of the interaction.
1 Assignment
0 Petitions
Accused Products
Abstract
A multi-modal interaction modeling system can model a number of different aspects of a human interaction across one or more temporal interaction sequences. Some versions of the system can generate assessments of the nature or quality of the interaction or portions thereof, which can be used to, among other things, provide assistance to one or more of the participants in the interaction.
46 Citations
18 Claims
-
1. A method for predicting a behavioral event to occur in an interaction involving at least two participants, at least one of the participants being a person, the method comprising, with a computing system:
-
detecting, from multi-modal data captured by one or more sensing devices, a plurality of different behavioral cues expressed by the participants during the interaction; recognizing a plurality of temporal interaction sequences, each of the temporal interaction sequences occurring over a time interval during the interaction, and involving a pattern of the behavioral cues, and at least two of the participants being involved in at least one of the temporal interaction sequences; determining the nature of the interaction based on the recognized temporal interaction sequences; and predicting the behavioral event to occur in the interaction based on the determined nature of the interaction. - View Dependent Claims (2, 3, 4, 5, 6)
-
-
7. An interaction assistant embodied in one or more machine-readable storage media and accessible by a computing device to assist with an interaction involving a natural-language dialog between a person and the computing device, by:
-
detecting, from multi-modal data captured by at least one sensing device, a plurality of different behavioral cues expressed by the person during the natural-language dialog; recognizing a temporal interaction sequence comprising a pattern of the behavioral cues occurring over a time interval during the natural-language dialog; deriving, from the temporal interaction sequence, an assessment of a portion of the natural-language dialog involving the person; and formulating a portion of the natural-language dialog involving the computing device based on the assessment. - View Dependent Claims (8, 9, 10, 11, 12, 13)
-
-
14. A method for assessing a person'"'"'s emotional state during an interaction involving the person and at least one other participant, the method comprising, with a computing system:
-
detecting, from multi-modal data captured by one or more sensing devices, a plurality of different behavioral cues expressed by the participants during the interaction; recognizing a plurality of temporal interaction sequences, each temporal interaction sequence comprising a pattern of the behavioral cues occurring over a time interval during the interaction, and at least one of the temporal interaction sequences involving the person and at least one other participant; assessing the person'"'"'s emotional state during each of the temporal interaction sequences based on the behavioral cues involved in the temporal interaction sequence; detecting changes in the participant'"'"'s behavior over a period of time defining the duration of the interaction; and evaluating the participant'"'"'s behavior over the period of time based on the detected changes in behavior and the time intervals in which they occurred. - View Dependent Claims (15, 16, 17, 18)
-
Specification