Client server animation system for managing interactive user interface characters

• US 5,983,190 A
• Filed: 05/19/1997
• Issued: 11/09/1999
• Est. Priority Date: 05/19/1997
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Merrill, John Wickens Lamb, Weinberg, Mark Jeffrey, Trower, Tandy W. II
• Primary Examiner(s): Dorvil, Richemond
• Application Number:
  US08/858,648
• Time in Patent Office: 904 Days
• Field of Search: 704/220, 704/276, 704/275, 704/231, 704/251, 704/258, 704/270, 704/272, 704/200
• US Class Current: 704/276
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 8/00 Arrangements for software engineering testing or debugging G06F11/36; administrative, planning or organisation aspects of software project management G06Q10/06
• CPC Class Codes: Y10S 345/949 Animation processing method

Abstract

A client server animation system provides services to enable clients to play animation and lip-synched speech output for an interactive user interface character. Through the programming interface of the server, clients can specify both speech and cursor device input that an instance of an interactive user interface character will respond to when the clients are active. Clients can also request playback of animation and lip-synched speech output through this interface. Services can be invoked from application programs as well as web scripts embedded in web pages downloaded from the Internet.

Claims

1. A method for generating an interactive, animated character in the user interface of a computer using a client-server architecture, the method comprising:

in response to a request from a client, creating an instance of a character and displaying the character in the user interface;

in the server, receiving from a client a set of client-specified user input commands that the character will respond to, the set comprising cursor input from a cursor control device;

in the server, monitoring for the specified user input commands;

in the server, when one of the user input commands is detected, sending a notification to the client;

in the server, receiving from the client a request that is conditioned upon the notification from the server; and

in response to the request from the client, playing back a client-specified sequence of animation output to animate the character in the user interface.

2. The method of claim 1 wherein the set of client-specified user input commands further comprises speech input received through a speech recognition engine, the method further comprising:

in response to a request from the client, playing back a client-specified sequence of animation to animate the character in the user interface and generating speech output lip-synched to animation representing a mouth of the character.

3. The method of claim 1 further including:

queuing requests to animate the character from the client when the character is currently playing back an animation;

immediately returning control to the client making the request to animate the character after determining that the character is busy; and

deferring processing of the request until the current animation is complete.

4. The method of claim 1 further including:

arbitrating requests to control the character from more than one client.

5. The method of claim 1 wherein the step of creating an instance of a character further comprises registering the client with the server, the method further comprising:

registering a second client with the server;

in the server, receiving from the second client a second set of client-specified user input commands that the character will respond to, the second set comprising cursor input from the cursor control device;

keeping track of clients that have registered with the server;

arbitrating requests to control the character from more than one client; and

terminating the character when no clients are currently registered with the server.

6. The method of claim 1 wherein the step of creating an instance of the character comprises:

starting execution of the server in response to a request from the client;

in the server, registering a notification interface for the client in response to a request from the client; and

in the server, receiving from the client a request telling the server which character to create.

7. The method of claim 1 including:

synchronizing execution of the client with the execution of the character by allowing the client to post notification requests with the server in a first in first out queue used to store animation requests while the character is currently being animated, and sending a notification from the server to the client when the first notification request is at the top of the queue.

8. The method of claim 7 wherein the notification requests are embedded with text that is synthesized into speech output by the server so that the client can synchronize itself to individual words in the speech output.

9. A computer readable medium on which is stored software for performing the method of claim 1.

10. A client-server animation system for generating interactive animated characters, the system comprising:

an animation server for receiving requests from clients to create a character on the user interface, for controlling playback of a sequence of frames of animation and lip synched speech output from the character on the user interface in response to requests from the clients, for receiving an identification of cursor device and speech input commands, and for notifying the clients when the server determines that the cursor device input and the speech input commands have been provided by a user;

a speech recognition engine in communication with an audio input device for receiving speech input from the user and for analyzing the speech input to identify the speech input commands; and

in communication with the server for sending notification messages to the server when the speech input commands are detected; and

a speech synthesis engine in communication with an audio output device for generating speech output, and in communication with the server for receiving requests to generate audio output corresponding to a text string provided by the clients via the server, and for notifying the server when a tag is detected in the text string so that the server can synchronize display of text in the text string with the speech output.

11. The animation system of claim 10 wherein the server includes a queue for queuing requests from clients to play specified sequences of animation of the character;

and wherein the server keeps track of which of the clients is currently active and processes the requests in the queue corresponding to an active client.

12. The animation system of claim 10 wherein the server includes a mouth animation module for receiving notifications from the speech synthesis engine synchronized with speech output of phonemes, and wherein the mouth animation module is operable to play a frame of animation of a mouth of the character that corresponds to a current phoneme such that animation of the mouth is synchronized with the speech output.

13. The animation system of claim 10 wherein the animation server includes a parser for parsing speech input commands provided by the clients and passing parsed speech input commands to the speech recognition engine.

14. The animation system of claim 10 wherein the animation server includes a regionizer for scanning an animation frame and computing a non-rectangular bounding region for a non-transparent portion of the animation frame in real time as the sequence of constructed animation frames is played in the user interface on the display monitor;

and wherein the animation system includes a region window controller for receiving the non-rectangular bounding region from the regionizer, for creating a region window on a display screen independent of any other window on the display screen and having a screen boundary in the user interface defined by the non-rectangular bounding region, and for clipping the constructed animation frame to the non-rectangular bounding region.

15. The animation system of claim 10 including a web browser for retrieving a web page from secondary storage of a local computer or from a remote computer, for parsing the web page to identify an embedded agent object tag, and for starting the server in response to detecting the embedded agent object tag;

wherein the server is responsive to a first script command embedded in the web page to play a first sequence of frames of animation and lip synched speech output from the character on the user interface, and wherein the server is responsive to a second script command for receiving an identification of a speech input command and for sending notification to a local client representing the web script when the server detects the speech input command.

16. The system of claim 15 wherein the animation system includes a runtime compiler in communication with the web browser for compiling and executing a script program including the first and second script commands.

17. A method for generating an interactive, animated character in the user interface of a computer using a client-server architecture, the method comprising:

in response to a request from a client, creating an instance of a character and displaying the character in the user interface;

in the server, receiving from a client a set of client-specified user input commands that the character will respond to, the set comprising a speech input command;

in the server, monitoring for the specified user input commands;

in the server, sending a notification to the client when one of the user input commands is detected;

in the server, receiving from the client a request that is conditioned upon the notification from the server; and

in response to the request from the client, playing back a client-specified sequence of animation and speech output to animate the character in the user interface.

18. The method of claim 17 wherein the client is a script embedded in a web page, wherein the script includes a first script command specifying text of the speech input command, wherein the server sends a notification to the client when the server detects that an end user has spoken the speech input command;

and wherein the script includes a second script command requesting lip synched speech output from the server.

19. The method of claim 17 further including:

parsing a web page to identify an embedded script; and

compiling the script to create the client.

20. The method of claim 19 further including:

in the server, processing requests to animate the character and play lip-synched output from the web script client.

21. A computer readable medium on which is stored software for performing the method of claim 17.