Affect-sensitive intelligent tutoring system

• US 10,388,178 B2
• Filed: 10/05/2010
• Issued: 08/20/2019
• Est. Priority Date: 08/27/2010
• Status: Active Grant

• Current Assignee: Arthur Carl Graesser, D'Mello, Sidney Keith
• Original Assignee: Arthur Carl Graesser, D'Mello, Sidney Keith
• Inventors: Graesser, Arthur Carl, D'Mello, Sidney Keith
• Primary Examiner(s): Bullington, Robert P
• Application Number:
  US12/898,132
• Publication Number:
  US 20120052476A1
• Time in Patent Office: 3,241 Days
• Field of Search: 434362
• CPC Class Codes: G09B 7/04 characterised by modifying the teaching programme in response to a wrong answer, e.g. repeating the question, supplying a further explanation

Abstract

An Intelligent Tutoring System (ITS) system is provided that is able to identify and respond adaptively to the learner'"'"'s or student'"'"'s affective states (i.e., emotional states such as confusion. frustration, boredom, and flow/engagement) during a typical learning experience, in addition to adapting to the learner'"'"'s cognitive states. The system comprises a new signal processing model and algorithm, as well as several non-intrusive sensing devices, and identifies and assesses affective states through dialog assessment techniques, video capture and analysis of the student'"'"'s face, determination of the body posture of the student, pressure on a pressure sensitive mouse, and pressure on a pressure sensitive keyboard. By synthesizing the output from these measures, the system responds with appropriate conversational and pedagogical dialog that helps the learner regulate negative emotions in order to promote learning and engagement.

Claims

1. An automated affect-sensitive tutoring system, comprising:

at least one pressure measurement device on a chair or seat;

a computing device, said computing device comprising a microprocessor, means for receiving input from a user, and a non-transitory computer-readable storage medium with an executable program stored thereon, wherein the program engages in an interactive tutoring session with the user;

a dialog pattern analysis module, wherein the dialog pattern analysis module detects patterns of dialog within the tutoring session that indicate affective states of the user by analyzing

the correctness of the user'"'"'s answer,

the verbosity of the user'"'"'s answer,

the length of the user'"'"'s answer,

the number of attempts by the user to answer a single question, and

the number of questions that have been answered in the interactive tutoring session; and

at least one camera for determining the posture and facial expressions of the user during the interactive tutoring session.

2. The system of claim 1, wherein the means for receiving input comprises a keyboard, a computer mouse, a microphone, or a combination thereof.

3. The system of claim 1, wherein the at least one pressure measurement device comprises at least one pressure-sensitive pad placed on the sitting surface of the seat or chair, and at least one pressure-sensitive pad placed on the back of the seat or chair.

4. The system of claim 1, further comprising a facial expression analysis program that receives images of the facial expression of the user from the at least one camera.

5. The system of claim 1, wherein the system determines the affective state of the user.

6. The system of claim 1, wherein the system performs an assessment of the user'"'"'s affective state based upon the classification of the current affective state detected, the confidence level of that affective classification, the previous affective state detected, a measure of the user'"'"'s ability, and the conceptual quality of the user'"'"'s immediate prior response.

7. The system of claim 5, wherein the system determines its next interaction with the user based upon the system'"'"'s determination of the user'"'"'s affective state.

8. The system of claim 7, wherein the next interaction with the user comprises feedback for the user'"'"'s immediate prior response, an affective statement, the next dialogue move, an emotional display on the face of a system tutor agent on a computer monitor connected to the computer device, and emotional modulation of the voice produced by a text-to-speech engine on the computing device.

9. The system of claim 7, wherein the next interaction with the user comprises displaying gestures by an animated tutor agent on a computer monitor connected to the computer device.

10. The system of claim 1, further comprising a text-to-speech engine.

11. The system of claim 7, further wherein the next interaction with the user comprises mixed-initiative dialogue.

12. The system of claim 2, wherein the means for receiving input comprises a keyboard, wherein the keyboard is further configured to measure the amount of pressure placed on the keyboard by the user.

13. The system of claim 2, wherein the means for receiving input comprises a mouse, wherein the mouse is further configured to measure the amount of pressure placed on the mouse by the user.

14. The system of claim 7, wherein the system determines its next interaction with the user based upon the application of a set of dynamic, context-sensitive production rules to the following parameters:

the classification of the current affective state detected;

the confidence level of that affect classification;

the previous affective state detected;

a dynamically updated global measure of user ability; and

the conceptual quality of the user'"'"'s immediate response.