Method and system for simulated interactive conversation

• US 20040230410A1
• Filed: 05/13/2003
• Published: 11/18/2004
• Est. Priority Date: 05/13/2003
• Status: Active Grant

• Current Assignee: Interactive Drama Incorporated
• Original Assignee: Interactive Drama Incorporated
• Inventors: Harless, Michael G., Zier, Marcia A., Harless, William G.
• Granted Patent: US 7,797,146 B2
• Time in Patent Office: Days
• US Class Current: 703/6
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

A method of simulating interactive communication between a user and a human subject. The method comprises: assigning at least one phrase to a stored content sequence, wherein the content sequence comprises a content clip of the subject; parsing the at least one phrase to produce at least one phonetic clone; associating the at least one phonetic clone with the stored content sequence; receiving an utterance from the user; matching the utterance to the at least one phonetic clone; and displaying the stored content sequence associated with the at least one phonetic clone

Claims

1. A method of simulating interactive communication between a user and a human subject, comprising:

assigning at least one phrase to a stored content sequence, wherein the content sequence comprises a content clip of the subject;

parsing the at least one phrase to produce at least one phonetic clone;

associating the at least one phonetic clone with the stored content sequence

receiving an utterance from the user;

matching the utterance to the at least one phonetic clone; and

displaying the stored content sequence associated with the at least one phonetic clone.

2. The method of claim 1, wherein parsing the phrase to produce the at least one phonetic clone further comprises:

performing a first partial parsing of the phrase to produce at least one first partially parsed phrase;

sub-parsing the at least one first partially parsed phrase to produce at least one first sub-parsed phrase; and

generating at least one phonetic clone from the at least one first sub-parsed phrase.

3. The method of claim 1, wherein parsing the phrase to produce the at least one phonetic clone further comprises:

performing a second partial parsing of the phrase to produce at least one second partially parsed phrase;

sub-parsing the at least one second partially parsed phrase to produce at least one second sub-parsed phrase; and

generating at least one phonetic clone from the at least one second partially parsed phrase.

4. The method of claim 1, wherein parsing the phrase to produce the at least one phonetic clone further comprises:

selecting a keyword from the phrase;

selecting at least one synonym of the keyword; and

generating at least one phonetic clone of the at least one synonym of the keyword.

5. The method of claim 1, wherein parsing the phrase to produce the at least one phonetic clone further comprises:

selecting a qualifier from the phrase;

selecting at least one synonym of the qualifier; and

generating at least one phonetic clone of the at least one synonym of the qualifier

6. The method of claim 1, wherein matching the utterance to the at least one phonetic clone further comprises:

processing the utterance to generate a perceived sound match;

comparing the perceived sound match to the at least one phonetic clone;

performing an arithmetic operation on the at least one phonetic clone and the perceived sound match to generate a result;

comparing the result to a threshold amount; and

if the result is greater than the threshold amount, determining that a match has been found.

7. The method of claim 6, wherein performing the arithmetic operation further comprises:

counting the number of letters matched between the perceived sound match and the at least one phonetic clone; and

multiplying the number of letters matched by a priority number to produce a product as the result.

8. The method of claim 1, further comprising:

storing a video clip as a content clip; and

following display of the content sequence, displaying a neutral image of the subject.

9. The method of claim 8, further comprising:

selecting a begin clip based on frame matching the last frame of the begin clip with the first frame of the content clip and the first frame of the begin clip with the neutral image of the subject; and

storing the begin clip sequentially prior to the stored content clip within the stored content sequence.

10. The method of claim 8, further comprising:

selecting an end clip based on frame matching the first frame of the end clip with the last frame of the content clip and the last frame of the end clip with the neutral image of the subject; and

storing the end clip sequentially after the stored content clip within the stored content sequence.

11. The method of claim 1, further comprising:

displaying a neutral image of the subject when not displaying the content sequence.

12. The method of claim 1, further comprising:

storing a listening clip of the subject; and

displaying the listening clip of the subject when not displaying the content sequence.

13. The method of claim 1, further comprising:

storing a multimedia object; and

displaying the multimedia object to the user upon request of the user.

14. The method of claim 13, wherein displaying the multimedia object to the user is performed upon matching the utterance of the user to a request to display the multimedia object.

15. The method of claim 13, wherein displaying the multimedia object to the user is performed based on the stored multimedia object being associated with the matched phonetic clone.

16. The method of claim 1, further comprising:

receiving a halt command from the user; and

displaying a neutral image of the subject following the halt command.

17. The method of claim 10, comprising:

receiving a halt command from the user

transitioning from the content clip to a neutral image of the subject; and

displaying the neutral image of the subject following the halt command.

18. The method of claim 1, comprising a second human subject and further comprising:

assigning at least one second phrase to a stored video clip of the second subject;

parsing the at least one second phrase to produce at least one phonetic clone of the at least one second phrase;

associating the at least one phonetic clone of the at least one second phrase with the stored video clip of the second subject;

receiving an utterance from the user;

matching the utterance to the at least one phonetic clone of the second subject;

comparing the match of the at least one phonetic clone of the second subject with the match of the at least one phonetic clone of the subject; and

if the match of the at least one phonetic clone of the second subject is superior to the match of the at least one phonetic clone of the subject, displaying the stored video clip of the second subject associated with the at least one phonetic clone and not displaying the stored video clip of the subject.

19. The method of claim 18, further comprising displaying the stored content sequence of the subject following the displaying of the stored content sequence of the second subject.

20. A system for simulating interactive communication between a user and a human subject, the system comprising:

a display for displaying the subject;

a memory; and

a processor, coupled to the memory and the display, the processor operable to;

assign at least one phrase to a stored content sequence of the subject, wherein the content sequence comprises a content clip of the subject;

parse the at least one phrase to produce at least one phonetic clone of the at least one phrase;

associate the at least one phonetic clone with the stored content sequence;

receive an utterance from the user;

match the utterance to the at least one phonetic clone; and

display the stored content sequence associated with the at least one phonetic clone.

21. A method of simulating interactive communication between a user and a human subject, comprising:

storing a content clip of the subject;

frame matching the content clip of the subject to a neutral image of the subject to select a begin clip;

matching an utterance of the user to the video clip; and

displaying the begin clip to the user followed by displaying the content clip to the user.

22. The method of claim 21, wherein frame matching the content clip further comprises:

examining a beginning frame of the content clip;

examining an ending frame of the begin clip; and

selecting the begin clip based on the similarity of the beginning frame of the content clip to the ending frame of the begin clip.

23. The method of claim 21, wherein the beginning frame of the content clip is the first frame.

24. The method of claim 21, wherein the ending frame of the begin clip is the last frame.

25. A method of simulating interactive communication between a user and a human subject, comprising:

receiving a voice input from the user;

matching the voice input to one of a plurality of a stored phonetic clones, the phonetic clones each corresponding to a target speech phrase associated with a stored content sequence file depicting the subject, the number of stored phonetic clones being greater than the number of stored content sequence files; and

displaying the stored content sequence file matched to the phonetic clone.

26. The method of claim 25, wherein matching the voice input to the phonetic clone further comprises:

processing the voice input to generate a perceived sound match;

comparing the perceived sound match to the phonetic clone;

performing an arithmetic operation on the phonetic clone and the perceived sound match to generate a result;

comparing the result to a threshold amount; and

if the result is greater than the threshold amount, determining that a match has been found.

27. The method of claim 25, wherein the target speech phrase is associated with a stored multimedia object and further comprising:

displaying the stored multimedia object matched to the phonetic clone.

28. The method of claim 25, further comprising:

displaying a stored neutral image of the subject following the display of the stored content sequence file of the subject.

29. The method of claim 25, further comprising:

displaying a stored neutral image of the subject prior to the display of the stored content sequence file of the subject.

30. The method of claim 25, further comprising:

displaying a stored listening clip of the subject following the display of the stored content sequence file of the subject.

31. The method of claim 25, further comprising:

displaying a stored listening clip of the subject prior to the display of the stored content sequence file of the subject.

32. The method of claim 25, further comprising:

receiving a second voice input from the user during the display of the stored content sequence file;

matching the second voice input from the user to a stored phonetic clone of a halt utterance; and

terminating the display of the stored content sequence file.

33. The method of claim 32, further comprising:

following termination of the display of the stored content sequence file, displaying a stored neutral image of the subject.

34. The method of claim 32, further comprising:

following termination of the display of the stored content sequence file, transitioning to a stored neutral image of the subject.

35. A conversation system for simulating interactive communication between a user and a human subject, comprising:

a display for displaying the subject;

a memory; and

a processor, coupled to the memory and the display, the processor operable to;

receive a voice input from the user

match the voice input to one of a plurality of a stored phonetic clones, a first portion of the phonetic clones each corresponding to a target speech phrase associated with a stored content sequence file depicting the first subject and a second portion of the phonetic clones each corresponding to a target speech phrase associated with a stored content sequence file depicting the second subject, the number of stored phonetic clones being greater than the number of stored content sequence files; and

display the stored content sequence file matched to the phonetic clone.

36. The system of claim 35, wherein the processor is further operable to:

score the quality of the match between the voice input from the user and a stored phonetic clone corresponding to a target speech phrase associated with a stored content sequence file depicting the first subject, producing a first subject score;

score the quality of the match between the voice input from the user and a stored phonetic clone corresponding to a target speech phrase associated with a stored content sequence file depicting the second subject, producing a second subject score; and

if the first subject score is greater than the second subject score, display the stored content sequence associated with the first subject.

37. The system of claim 35, wherein the processor is further operable to, following the display of the stored content sequence associated with the first subject, display the stored content sequence associated with the second subject.

38. A method of authoring interactive communication between a user and a human subject, comprising:

assigning a target speech phrase to a stored content sequence file of the subject;

parsing the target speech phrase to produce a plurality of stored phonetic clones, the phonetic clones corresponding to the target speech phrase, the number of stored phonetic clones being greater than the number of stored content sequence files; and

associating the phonetic clones with the stored content sequence file.

39. An authoring system for authoring an interactive communication script between a user and a human subject, comprising

assigning at least one phrase to a stored content sequence, wherein the content sequence comprises a content clip of the subject;

parsing the at least one phrase to produce at least one phonetic clone; and

associating the at least one phonetic clone with the stored content sequence.