SITUATED SIMULATION FOR TRAINING, EDUCATION, AND THERAPY

• US 20150072322A1
• Filed: 11/13/2014
• Published: 03/12/2015
• Est. Priority Date: 12/27/2006
• Status: Active Grant

• Current Assignee: Case Western Reserve University
• Original Assignee: Case Western Reserve University
• Inventors: WILLIAMS, Stacy L., BUCHNER, Marc
• Granted Patent: US 10,089,895 B2
• Time in Patent Office: Days
• US Class Current: 434/185
• CPC Class Codes: G09B 19/04 Speaking with audible presentation of the material to be studied G09B5/04
• CPC Class Codes: G09B 5/065 Combinations of audio and video presentations, e.g. videotapes, videodiscs, television systems
• CPC Class Codes: G09B 7/02 of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student
• CPC Class Codes: G10L 25/48 specially adapted for particular use

Abstract

Systems, methods, and other embodiments associated with producing an immersive training content module (ITCM) are described. One example system includes a capture logic to acquire information from which the ITCM may be produced. An ITCM may include a set of nodes, a set of measures, a logic to control transitions between nodes during a training session, and a logic to establish values for measures during the training sessions. Therefore, the example system may also include an assessment definition logic to define a set of measures to be included in the ITCM and an interaction logic to define a set of interactions to be included in the ITCM. The ITCM may be written to a computer-readable medium.

Claims

1. A system, comprising:

a capture logic to acquire a set of information related to a virtual environment of a training session to be employed in producing an immersive training content module (ITCM);

an assessment definition logic to define a set of measures to be included in the ITCM, where the set of measures facilitates at least one of;

assessing the progress of the training session,

evaluating how the training session may be adapted, and

assessing a user'"'"'s ability demonstrated in the training session;

an interaction logic to define a set of interactions that define, at least in part, interaction data with possible responses to a stimulus occurring in the training session to be included in the ITCM;

a module logic to produce the ITCM based, at least in part, on the set of information, the set of measures, and the set of interactions; and

a write logic to store the ITCM in a computer readable medium, wherein the ITCM includes;

a set of nodes, where a node includes a content comprising content data to be presented during the training session;

the set of measures;

an assessment logic to populate members of the set of measures with values based, at least in part, on an assessment data and the interaction data acquired during the training session, where the assessment data is based, at least in part, on responses to the stimulus occurring in the training session; and

a transition logic to control transitions between nodes based, at least in part, on the values associated with the set of measures.

2. The system of claim 1, where the set of nodes are arranged in a graph, where an ITCM includes an artificial intelligence (AI) logic to evaluate training session data acquired during the training session.

3. The system of claim 2, where the assessment data is acquired from one or more of, an operator, and the AI logic, where the assessment logic is to determine the value for a member of the set of measures based on one or more of:

an operator signal,

an AI signal provided by the AI logic,

the operator signal as modified by the AI logic, and

the AI signal as modified by the operator.

4. The system of claim 1, where the content in a node includes content data describing one or more of:

a character to be presented during the training session,

a setting in which the character is to be presented,

video to be presented during the training session,

audio to be presented during the training session,

a digital image,

an interaction involving the character, and

a set of node-level measures.

5. The system of claim 4, where the logic in a node is to perform one or more of:

acquiring the interaction data, and

populating members of the set of node-level measures based, at least in part, on the assessment data.

6. The system of claim 1, where the set of information includes one or more of:

a real-time three-dimensional animation data,

a video data,

a digital image,

a sound encoded in a file,

a text,

a tactile data, and

an olfactory data,

where the set of measures includes one or more of;

an articulatory movement measure,

an articulatory precision measure,

a voice modulation measure,

a voice resonance measure,

a pacing measure,

a stuttering measure,

a slurring measure,

a performance measure, and

a comprehension measure, and

where the interaction data includes one or more of;

a possible response, and

a response time.

7. The system of claim 2, where the transition logic selects a member of the set of nodes to visit based on one or more of, an operator signal, an AI signal provided by the AI logic, the operator signal as modified by the AI logic, and the AI signal as modified by the operator.

8. The system of claim 2, where an ITCM includes a feedback logic to acquire a set of feedback data during a training session associated with the ITCM.

9. The system of claim 8, the set of feedback data including one or more of, heart rate data, skin temperature data, galvanic skin response (GSR) data, electro-cardiogram”

(EKG) data, electro-encephalogram (EEG) data, pupil dilation data, eye contact data, facial expression data, posture data, gestural data, intonation data, therapist impression data, and speech performance data,

where an ITCM includes an adaptation logic to automatically adapt, in real-time, a training session based, at least in part, on the set of feedback data.

10. The system of claim 9, where the adaptation logic determines an adaptation to be made to the training session based, at least in part, on one or more of, an operator signal, an AI signal provided by the AI logic, the operator signal as modified by the AI logic, and the AI signal as modified by the operator

11. The system of claim 10, where the adaptation logic determines the adaptation to be made based, at least in part, on physiological bio-feedback, behavioral non-verbal bio-feedback, and performance evaluation.

12. The system of claim 11, where the adaptation logic determines to adapt one or more of, a number of distractions in the training session, a type of distraction in the training session, a number of stressors in the training session, a type of stressor in the training session, a complexity associated with the training session, a volume associated with the training session, a pace associated with the training session, a proximity of a character associated with the training session, a degree of friendliness of a character associated with the training session, and a degree of animation of a character associated with the training session.

13. The system of claim 1, where the ITCM is associated with a real-time adaptive interactive speech therapy session.

14. The system of claim 2, where the AI logic is a back propagation neural network.

15. A computer-readable medium storing computer-executable instructions that when executed by a computer cause the computer to perform a method, the method comprising:

acquiring information related to an environment of a training session to be employed in producing an ITCM associated with real-time adaptive interactive speech therapy;

defining a set of measures to be included in the ITCM, where the set of measures facilitates at least one of;

assessing the progress of the training session,

evaluating how the training session may be adapted, and

assessing a user'"'"'s ability demonstrated in the training session;

defining a set of interactions that define, at least in part, interaction data with possible responses to a stimulus occurring in the training session to be included in the ITCM;

creating the ITCM; and

storing the ITCM on a computer-readable medium, where the ITCM includes;

a set of nodes having content, comprising content data to be presented during the training session;

the set of measures; and

an assessment logic to populate a member of the set of measures with values based, at least in part, on assessment data and the interaction data acquired during the training session, where the assessment data is based, at least in part, on responses to the stimulus occurring in the training session; and

a transition logic to control transitions between members of the set of nodes during the training session based, at least in part, on an interaction experienced at a node.

16. The computer-readable medium of claim 15, where acquiring the information includes acquiring one or more of, a real-time three-dimensional animation data, a video data, a digital image, a sound encoded in a file, a text, a tactile data, and an olfactory data, and where defining the set of measures includes defining one or more of, an articulatory movement measure, an articulatory precision measure, a voice modulation measure, a voice resonance measure, a pacing measure, a stuttering measure, a slurring measure, a performance measure, and a comprehension measure.

17. The computer-readable medium of claim 16, the method including:

controlling a computing component to initiate a training session associated with the ITCM;

acquiring training session data during the training session; and

determining a value for a member of the set of measures based, at least in part, on one or more of, a response, a response time, heart rate data, skin temperature data, galvanic skin response (GSR) data, electro-cardiogram (EKG) data, electro-encephalogram (EEG) data, pupil dilation data, eye contact data, facial expression data, posture data, gestural data, intonation data, therapist impression data, speech performance data, physiological bio-feedback, behavioral non-verbal bio-feedback, and performance evaluation.

18. The computer-readable medium of claim 17, the method including selectively adapting a training session parameter based, at least in part, on the training session data.

19. The computer-readable medium of claim 15, where selectively adapting the training session parameter includes adapting one or more of, a number of distractions in the training session, a type of distraction in the training session, a number of stressors in the training session, a type of stressor in the training session, a complexity associated with the training session, a volume associated with the training session, a pace associated with the training session, a proximity of a character associated with the training session, a degree of friendliness of a character associated with the training session, and a degree of animation of a character associated with the training session based, at least in part, on one or more of, an operator signal, an automated signal provided by an AI logic, the operator signal as modified by the AI logic, and the automated signal as modified by the operator.

20. A computing system, comprising:

means for acquiring information into an integrated content creation environment, where the information is to be employed in producing an ITCM associated with an automated, real-time, adaptive speech therapy session;

means for generating a set of measures to be included in the ITCM, where the set of measures facilitates at least one of;

assessing the progress of the automated, real-time, adaptive speech therapy session,

evaluating how the automated, real-time, adaptive speech therapy session may be adapted, and

assessing a user'"'"'s ability demonstrated in the automated, real-time, adaptive speech therapy session;

means for defining a set of interactions that define, at least in part, interaction data with possible responses to a stimulus occurring in the automated, real-time, adaptive speech therapy session to be included in the ITCM;

means for producing the ITCM; and

means for storing the ITCM on a computer-readable medium.