Systems and methods for determining and using interaction models

• US 20050182618A1
• Filed: 03/23/2004
• Published: 08/18/2005
• Est. Priority Date: 02/18/2004
• Status: Active Grant

• Current Assignee: Fuji Xerox
• Original Assignee: Fuji Xerox Company Limited (Xerox Holdings Corp.)
• Inventors: Polanyi, Livia, Van Den Berg, Martin H., Thione, Giovanni L., Azara, Misty
• Granted Patent: US 7,415,414 B2
• Time in Patent Office: Days
• US Class Current: 704/9
• CPC Class Codes: G06F 40/35 Discourse or dialogue representation
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 25/51 for comparison or discrimination
• CPC Class Codes: G10L 25/90 Pitch determination of speech signals

Abstract

Techniques are provided for determining and using interaction models. Discourse functions, prosodic features and turn information are determined from the speech information in a training corpus. Statistics, decision trees, rules and/or various other methods are used to determine a predictive interaction model based on the discourse functions, the prosodic features and the turn information. Predictive interaction models are optionally determined for individual users, genres, languages and/or other characteristics of the speech information. The predictive interaction model is useable to predict turns in a dialogue based on the discourse functions and prosodic features identified in the speech information. Speech information is presented and/or received based on the predictive interaction model.

Claims

1. A method of determining user interactions comprising the steps of:

determining speech information;

determining discourse functions and prosodic features in the speech information;

determining a predictive interaction model; and

determining an interaction turn based on the predictive interaction model and the determined discourse functions and prosodic features.

2. The method of claim 1, in which the discourse functions are determined from automatically recognized speech information.

3. The method of claim 1, in which the discourse functions are determined based on a theory of discourse analysis.

4. The method of claim 3, in which the theory of discourse analysis is at least one of:

the Linguistic Discourse Model, the Unified Linguistic Discourse Model, Rhetorical Structures Theory, Discourse Structure Theory and Structured Discourse Representation Theory.

5. The method of claim 1, further comprising the step of scheduling an interaction event based on the turn prediction.

6. The method of claim 1, in which the prosodic features include at least one of:

a silence preceding a discourse functions;

a silence following a discourse function;

rate of speech;

pitch frequency;

changes in pitch frequency and volume.

17. The method of claim 1, in which the speech information is at least one of:

verbal natural language information and non-verbal natural language information.

18. The method of claim 17, in which the non-verbal information is at least one of:

sign language gestures, pen gestures, hand gestures, body gestures and facial gestures.

20. The method of claim 18, in which the prosodic features include at least one of:

facial expressions, gesture velocity, and gesture force.

7. A method of determining a predictive interaction model comprising the steps of:

determining a training corpus of turn annotated speech information;

determining discourse functions and prosodic features associated with the turn information; and

determining a predictive interaction model based on the discourse functions, the prosodic features and the turn information.

8. The method of claim 7, in which the predictive interaction model is determined based on at least one of machine learning, decision tree, Naï

ve Bayes, rules and statistics.

9. The method of claim 7, in which the discourse function determined based on a theory of discourse analysis.

19. The method of claim 7, in which the speech information is at least one of:

verbal information natural language and non-verbal natural language information.

10. A system for determining interactions comprising:

an input/output circuit for retrieving recognized speech and prosodic features;

a processor that determines speech information; and

discourse functions and prosodic features in the speech information;

determines a predictive interaction model; and

determines an interaction turn based on the predictive interaction model and the discourse functions and prosodic features.

11. The system of claim 10, in which the discourse functions are determined from automatically recognized speech information.

12. The system of claim 10, in which the discourse functions are determined based on a theory of discourse analysis.

13. The system of claim 12, in which the theory of discourse analysis is at least one of:

the Linguistic Discourse Model, the Unified Linguistic Discourse Model, Rhetorical Structures Theory, Discourse Structure Theory and Structured Discourse Representation Theory.

14. The system of claim 10, in which the processor also schedules an interaction event based on the turn prediction.

15. The system of claim 10, in which the prosodic features include at least one of:

a silence preceding a discourse functions;

a silence following a discourse function;

rate of speech;

pitch frequency;

changes in pitch frequency and volume.

16. Computer readable storage medium comprising:

computer readable program code embodied on the computer readable storage medium, the computer readable program code usable to program a computer to determine interactions comprising the steps of;

determining speech information;

determining discourse functions and prosodic features in the speech information;

determining a predictive interaction model; and

determining an interaction turn based on the predictive interaction model and the determined discourse functions and prosodic features.