Speech animation

• US 7,529,674 B2
• Filed: 03/08/2004
• Issued: 05/05/2009
• Est. Priority Date: 08/18/2003
• Status: Active Grant

• Current Assignee: SAP SE
• Original Assignee: SAP AG (SAP SE)
• Inventors: Gong, Li, Duong, Townsend, Yinger, Andrew
• Primary Examiner(s): Hudspeth; David R
• Assistant Examiner(s): ALBERTALLI, BRIAN LOUIS
• Application Number:
  US10/796,242
• Publication Number:
  US 20050043955A1
• Time in Patent Office: 1,884 Days
• Field of Search: None
• US Class Current: 704/270
• CPC Class Codes: G06T 13/205 driven by audio data
• CPC Class Codes: G10L 2021/105 Synthesis of the lips movements from speech, e.g. for talking heads

Abstract

Methods and systems, including computer program products, for speech animation. The system includes a speech animation engine and a client application in communication with the speech animation engine. The client application sends a request for speech animation to the speech animation engine. The request identifies data to be used to generate the speech animation, where speech animation is speech synchronized with facial expressions. The client application receives a response from the speech animation engine. The response identifies the generated speech animation. The client application uses the generated speech animation to animate a talking agent displayed on a user interface of the client application. The speech animation engine receives the request for speech animation from the client application, retrieves the data identified in the request without user intervention, generates the speech animation using the retrieved data and sends the response identifying the generated speech animation to the client application.

Claims

1. A computer-implemented method comprising:

accessing a request to provide speech animation content to a user using a graphically represented, animated talking agent, the request comprising;

emotion, expression, look direction, and voice parameters for the animated talking agent;

speech elements including textual content data to be spoken by the animated talking agent, at least a portion of the textual content data being dynamically derived from an item selected by the user in a current application session;

a speech attribute defining whether the speech elements are to be sequentially or randomly provided; and

context information identifying a characteristic of the user, the characteristic being used to customize the generated speech animation;

identifying raw data to be used to generate speech animation content based on the request;

generating the speech animation content adapted for the user, using the identified raw data; and

providing the generated speech animation content for display to the user.

2. The computer-implemented method of claim 1, wherein the item selected by the user is a product for sale via a website viewed by the user.

3. The computer-implemented method of claim 1, wherein the characteristic of the user is one of a name, an age, a gender, an ethnicity, a national origin, a language, an occupation, a position of employment, or an organization affiliated with the user.

4. The computer-implemented method of claim 1, wherein identifying raw data to be used to generate speech animation content comprises comparing the request to provide speech animation content against a cache of previous requests, and wherein generating the speech animation content comprises retrieving previously generated speech animation if the request is identical to a previous request.

5. The computer-implemented method of claim 1, comprising accessing another request to provide speech animation content to the user, the other request comprising a reference to content specified in a different request from the other request.

6. The computer-implemented method of claim 1, comprising accessing another request to provide speech animation content to the user, the other request comprising a static value specifying static text to be spoken by the talking agent.

7. The computer-implemented method of claim 1, comprising accessing another request to provide speech animation content to the user, the other request comprising a reference to pre-generated speech animation.

8. The computer-implemented method of claim 1, comprising:

maintaining an open connection to the user after providing the generated speech animation; and

pushing additional speech animation through the open connection to the user after providing the generated speech animation content for display to the user.

9. The computer-implemented method of claim 1, comprising:

polling a data structure associated with the user after accessing the request to provide speech animation content to the user; and

generating additional speech animation content for display to the user based on the polled data structure.

10. A computer-readable medium encoded with a computer program, the computer program comprising instructions that, when executed, operate to cause a computer to perform operations comprising:

accessing a request to provide speech animation content to a user using a graphically represented, animated talking agent, the request comprising;

emotion, expression, look direction, and voice parameters for the animated talking agent;

speech elements including textual content data to be spoken by the animated talking agent, at least a portion of the textual content data being dynamically derived from an item selected by the user in a current application session;

a speech attribute defining whether the speech elements are to be sequentially or randomly provided; and

context information identifying a characteristic of the user, the characteristic being used to customize the generated speech animation;

identifying raw data to be used to generate speech animation content based on the request;

generating the speech animation content adapted for the user, using the identified raw data; and

providing the generated speech animation content for display to the user.

11. The system of claim 10, wherein accessing the request includes accessing the request in real time.

12. The system of claim 10, wherein:

the textual content data specifies a pool of synonyms; and

generating the speech animation includes selecting a synonym from the pool of synonyms.

13. The system of claim 10, wherein generating the speech animation content includes incorporating the context information into spoken text of the generated speech animation.

14. The system of claim 10, wherein:

the request is accessed via a web application; and

the request is an HTTP request.

15. A system comprising:

one or more computers; and

a computer-readable medium coupled to the one or more computers having instructions stored thereon which, when executed by the one or more computers, causes the one or more computers to perform operations comprising;

accessing a request to provide speech animation content to a user using a graphically represented, animated talking agent, the request comprising;

emotion, expression, look direction, and voice parameters for the animated talking agent;

speech elements including textual content data to be spoken by the animated talking agent, at least a portion of the textual content data being dynamically derived from an item selected by the user in a current application session;

a speech attribute defining whether the speech elements are to be sequentially or randomly provided; and

context information identifying a characteristic of the user, the characteristic being used to customize the generated speech animation;

identifying raw data to be used to generate speech animation content based on the request;

generating the speech animation content adapted for the user, using the identified raw data; and

providing the generated speech animation content for display to the user.

16. The system of claim 15, wherein the textual content data includes markup tags that define prosodic, pronunciation, intonation, or inflection settings for speech.

17. The system of claim 15, wherein the textual content data includes markup tags that are used to insert dynamic elements.

18. The system of claim 15, wherein the textual content data includes static text.

19. The system of claim 15, wherein the context information includes information about how long an application session has been active.

20. The system of claim 15, wherein generating the speech animation content comprises retrieving part of the speech animation content from a cache and combining the retrieved part of the speech animation content with a part of the speech animation content dynamically generated and referencing the item selected by the user.