Communication And Skills Training Using Interactive Virtual Humans

• US 20120139828A1
• Filed: 02/11/2010
• Published: 06/07/2012
• Est. Priority Date: 02/13/2009
• Status: Active Grant

• Current Assignee: University of Florida Research Foundation, Incorporated (State University System of Florida), Augusta University Research Institute, Inc. (University System of Georgia)
• Original Assignee: University of Florida Research Foundation, Incorporated (State University System of Florida), Georgia Health Sciences University (University System of Georgia)
• Inventors: Lok, Benjamin Chak Lum, Lind, David Scott, Cendan, Juan Carlos, Raij, Andrew Brian, Rossen, Brent H., Kotranza, Aaron Andrew, Johnsen, Kyle John
• Granted Patent: US 9,978,288 B2
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G09B 23/28 for medicine
• CPC Class Codes: G09B 7/00 Electrically-operated teaching apparatus or devices working with questions and answers mechanically operated G09B3/00; computing arrangements G06F
• CPC Class Codes: G16Z 99/00 Subject matter not provided for in other main groups of this subclass

Abstract

A system for providing interaction between a virtual human and a user, the system comprising: a tangible interface providing a physical interface between the user and the virtual human, an imaging system directed towards the physical interface to provide images of the user interacting with the tangible interface; a tracking system tracking at least one position or the user; a microphone capturing speech from the user; a simulation system receiving inputs from the tangible interface, the imaging system, the tracking system and the microphone, the simulation system generating output signals corresponding to the virtual human; and a display presenting the output signals to the user.

Claims

1. A system for providing interaction between a virtual human and a user, the system comprising:

a tangible interface providing a physical interface between the user and the virtual human,

an imaging system directed towards the physical interface to provide images of the user interacting with the tangible interface;

a tracking system tracking at least one position or the user;

a microphone capturing speech from the user;

a simulation system receiving inputs from the tangible interface, the imaging system, the tracking system and the microphone, the simulation system generating output signals corresponding to the virtual human; and

a display presenting the output signals to the user.

2. The system of claim 1 wherein:

the output signals include video and audio signals.

3. The system of claim 1 wherein:

the display is at least one of a head mounted display, data projector and large screen television.

4. The system of claim 1 wherein:

the tangible interface includes a plurality of force sensors providing force signals to the simulation system.

5. The system of claim 1 wherein:

wherein the tracking system includes cameras for tracking a location of markers worn by the user.

6. The system of claim 5 wherein:

the camera is an infrared camera and the markers are infrared markers.

7. The system of claim 5 wherein:

the tracking system tracks the user'"'"'s head.

8. The system of claim 1 wherein:

the virtual human is developed using a virtual people factory (VPF) system in communication with an educator system and user system over a network.

9. The system of claim 8 wherein:

the educator system interacts with the VPF system to generate an initial script for the virtual human.

10. The system of claim 9 wherein:

the VPF system gathers inputs from the user system interacting with the virtual human, the inputs indicating when the virtual human responds incorrectly to a user response.

11. The system of claim 10 wherein:

the educator system updates the VPF system in response to the inputs indicating when the virtual human responds incorrectly to the user response.

12. The system of claim 8 wherein:

the VPF system generates a response to a user input stimulus based on the stimulus matching a trigger stored in the VPF system, the trigger corresponding to a prior user stimulus.

13. The system of claim 12 wherein:

wherein the VPF system ranks triggers based on a keyword relevance-weighting scheme to determine which trigger of a plurality of stored triggers best matches the stimulus, the VPF system outputting the response linked to the trigger having the highest relevance.

14. The system of claim 1 further comprising:

an interpersonal scenario visualizer (IPSViz) implementing an after action review (AAR) process for reviewing user interaction with the virtual human.

15. The system of claim 14 wherein:

the IPSViz generates a visualization screen depicting the interaction between the user and the virtual person in a virtual environment.

16. The system of claim 15 wherein:

the IPSViz generates a transcript screen depicting a text transcript of the interaction between the user and the virtual human, the text transcript including a timeline.

17. The system of claim 16 wherein:

the IPSViz generates a raw video screen depicting the interaction between the user and the virtual person in a real world environment.

18. The system of claim 16 wherein:

the IPSViz generates a graph of a factor from the interaction between the user and the virtual plotted against time.

19. The system of claim 18 wherein:

the factor plotted versus time is the user'"'"'s body lean.

20. The system of claim 18 wherein:

the factor plotted versus time is topic of discussion between the user and the virtual human.