System and method for associating semantically parsed verbal communications with gestures

• US 20090307595A1
• Filed: 06/09/2008
• Published: 12/10/2009
• Est. Priority Date: 06/09/2008
• Status: Abandoned Application

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Dewar, Ami H., Leah, Robert C., Clark, Jason T., Yim, Peter C., Poore, Nicholas E.
• Application Number:
  US12/135,328
• Publication Number:
  US 20090307595A1
• Time in Patent Office: Days
• US Class Current: 715/728
• CPC Class Codes: A63F 2300/6607 for animating game characters, e.g. skeleton kinematics
• CPC Class Codes: G06N 3/006 based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

A metaverse system and method for dynamically enacting syntax-based gestures in association with a metaverse application. The metaverse system includes a metaverse server and a semantic gesturing engine. The metaverse server executes a metaverse application. The metaverse application allows metaverse application allows a user on the client computer to enter a metaverse virtual world as an avatar via a metaverse client viewer. The semantic gesturing engine is coupled to the metaverse server and identifies a verbal communication from the avatar within the metaverse application, dynamically selects a gesture associated with the verbal communication in response to a determination that an association exists between the verbal communication and the gesture, and dynamically executes the selected gesture to cause the avatar to enact the selected gesture in conjunction with conveying the verbal communication.

Claims

1. A computer program product comprising a computer useable storage medium to store a computer readable program that, when executed on a computer, causes the computer to perform operations comprising:

identify a verbal communication from an avatar within a metaverse application of a metaverse virtual world;

dynamically select a gesture associated with the verbal communication in response to a determination that an association exists between the verbal communication and the gesture; and

dynamically execute the selected gesture to cause the avatar to enact the selected gesture in conjunction with conveying the verbal communication.

2. The computer program product of claim 1, wherein the computer readable program, when executed on the computer, causes the computer to perform further operations comprising:

parse a text string; and

analyze the parsed text string to determine a meaning of the parsed text string using at least one of a plurality of semantic analysis techniques, wherein the plurality of analysis techniques comprise morphological analysis and syntactical analysis.

3. The computer program product of claim 2, wherein the computer readable program, when executed on the computer, causes the computer to perform further operations comprising:

compare the parsed text string to entries in a syntax database, wherein the entries in the syntax database define associations between elements of common text strings and a semantic meaning of the elements of the common text strings;

determine whether a match exists between the parsed text string and the entries in the syntax database;

compare a derived meaning of the text string to entries in a gesture database, wherein the derived meaning of the text string is based on the match between the parsed text string and the entries in the syntax database, wherein the entries in the gesture database define associations between the semantic meanings of the elements of the common text strings and a plurality of gestures; and

determine whether a match exists between the derived meaning of the text string and the entries in the gesture database.

4. The computer program product of claim 1, wherein the computer readable program, when executed on the computer, causes the computer to perform operations comprising:

define the association between the verbal communication and the gesture in the metaverse application; and

allow a user to modify existing associations between user generated text and at least one of a plurality of gestures in the metaverse application.

5. The computer program product of claim 1, wherein the computer readable program, when executed on the computer, causes the computer to perform operations comprising:

store associations between common text strings and semantic meanings of the common text strings, wherein the common text strings and the semantic meanings are stored in an arrangement to define associations between the common text strings and semantic meanings of the common text strings; and

store associations between the semantic meanings of common text strings and a plurality of gestures, wherein the common text strings and the semantic meanings are stored in an arrangement to define associations between the semantic meanings of common text strings and the plurality of gestures.

6. The computer program product of claim 1, wherein the computer readable program, when executed on the computer, causes the computer to perform operations comprising:

receive voice input in conjunction with the metaverse application;

convert the voice input to a text string form; and

select the gesture associated with the text string form.

7. A system comprising:

a metaverse server coupled to the client computer, the metaverse server to execute a metaverse application, wherein the metaverse application allows a user on the client computer to enter a metaverse virtual world as an avatar via a metaverse client viewer; and

a semantic gesturing engine coupled to the metaverse server, the semantic gesturing engine to identify a verbal communication from the avatar within the metaverse application, to dynamically select a gesture associated with the verbal communication in response to a determination that an association exists between the verbal communication and the gesture, and to dynamically execute the selected gesture to cause the avatar to enact the selected gesture in conjunction with conveying the verbal communication.

8. The system of claim 7, wherein the semantic gesturing engine comprises a semantic parser, the semantic parser to parse a text string and to determine a meaning of the parsed text string through semantic analysis of semantic elements in the parsed text string.

9. The system of claim 8, wherein the semantic gesturing engine further comprises a comparator, the comparator to compare the parsed text string to entries in a syntax database and to determine whether a match exists between the parsed text string and the entries in the syntax database, wherein the entries in the syntax database define associations between elements of common text strings and semantic meanings of the elements of the common text strings.

10. The system of claim 8, wherein the comparator is further configured to compare a derived meaning of the text string to entries in a gesture database, wherein the derived meaning of the text string is based on the match between the parsed text string and the entries in the syntax database, and to determine whether a match exists between the derived meaning of the text string and the entries in the gesture database, wherein the entries in the gesture database define associations between the semantic meanings of the elements of the common text strings and a plurality of gestures.

11. The system of claim 7, wherein the semantic gesturing engine further comprises a configurator, the configurator to define the association between the verbal communication and the gesture in the metaverse application and to allow the user to modify existing associations between user generated text and at least one of the plurality of gestures in the metaverse application.

12. The system of claim 7, further comprising a memory device coupled to the metaverse server wherein the memory device comprises a syntax database to store associations between common text strings and semantic meanings of common text strings.

13. The system of claim 7, further comprising a memory device coupled to the metaverse server wherein the memory device comprises a gesture database to store associations between the semantic meanings of common text strings and a plurality of gestures in the metaverse application.

14. The system of claim 7, wherein the semantic gesturing engine further comprises a voice-to-text converter to convert a voice input audio from the user to a text string form.

15. A method comprising:

identifying a verbal communication from an avatar within a metaverse application of a metaverse virtual world;

dynamically selecting a gesture associated with the verbal communication in response to a determination that an association exists between the verbal communication and the gesture; and

dynamically executing the selected gesture to cause the avatar to enact the selected gesture in conjunction with conveying the verbal communication.

16. The method of claim 15, further comprising:

parsing a text string; and

determining a meaning of the parsed text string using at least one of a plurality of semantic analysis techniques, wherein the plurality of analysis techniques comprise morphological analysis and syntactical analysis.

17. The method of claim 16, further comprising:

comparing the parsed text string to entries in a syntax database, wherein the entries in the syntax database define associations between elements of common text strings and a semantic meaning of the elements of the common text strings; and

determining whether a match exists between the parsed text string and the entries in the syntax database.

18. The method of claim 17, further comprising:

comparing a derived meaning of the text string to entries in a gesture database, wherein the derived meaning of the text string is based on the match between the parsed text string and the entries in the syntax database, wherein the entries in the gesture database define associations between the semantic meanings of the elements of the common text strings and a plurality of gestures; and

determining whether a match exists between the derived meaning of the text string and the entries in the gesture database.

19. The method of claim 15, further comprising:

defining the association between the verbal communication and the gesture in the metaverse application; and

allowing a user to modify existing associations between user generated text and at least one of a plurality of gestures in the metaverse application.

20. The method of claim 15, further comprising:

storing associations between common text strings and semantic meanings of the common text strings, wherein the common text strings and the semantic meanings are stored in an arrangement to define associations between the common text strings and semantic meanings of the common text strings; and

storing associations between the semantic meanings of common text strings and a plurality of gestures, wherein the common text strings and the semantic meanings are stored in an arrangement to define associations between the semantic meanings of common text strings and the plurality of gestures.