Automatic animation production system and method

• US 20050273331A1
• Filed: 06/03/2005
• Published: 12/08/2005
• Est. Priority Date: 06/04/2004
• Status: Abandoned Application

• Current Assignee: Reallusion, Inc.
• Original Assignee: Reallusion, Inc.
• Inventors: Lu, Tse-Jen
• Application Number:
  US11/143,661
• Publication Number:
  US 20050273331A1
• Time in Patent Office: Days
• US Class Current: 704/246
• CPC Class Codes: G06T 13/205 driven by audio data
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/176 Dynamic expression
• CPC Class Codes: G10L 2021/105 Synthesis of the lips movements from speech, e.g. for talking heads

Abstract

The present invention provides an automatic animation production system and method. The automatic animation production system creates animations by synthesizing various facial features according to audio analysis. By expanding animation parameters in a scenario template database according to audio analysis data, facial features in an image are varied with time, thereby creating animation. The scenario template database comprises a plurality of animation parameters. Combinations of various animation parameters can create various facial expressions in an image, in accompany with variations of audio, enriched effects are available.

Claims

1. An automatic animation production system driven by audio or events triggered by audio signals to produce animations according to selected scenarios, comprising:

a scenario selection interface for selecting at least one scenario template;

a scenario template database storing a plurality of scenario template data;

a scenario template processing module processing the selected scenario template data to generate a dynamic sequence of animation parameters synchronized to the audio or events; and

an animation production module loading the dynamic sequence of animation parameters to complete animation frames from which animations are produced.

2. The automatic animation production system as claimed in claim 1 further comprising:

a feature detection module identifying the features of an input image;

a geometry construction module constructing the identified input image which is a facial image into geometry data; and

an audio analysis module analyzing the audio to generate a mouth shape transition data and the synchronized events triggered/driven by the audio signals.

3. The automatic animation production system as claimed in claim 2, wherein the animation production module is provided for adjusting the geometry data according to the dynamic sequence of animation parameters in accompany with the audio and the mouth shape transition data to produce animations.

4. The automatic animation production system as claimed in claim 2, wherein the geometry construction module utilizes a progressive construction method which comprises the following steps:

(a) building a finest feature point set for the facial image and dividing the features of the facial image into various groups according to different facial portions;

(b) defining a plurality of levels of detail and establishing mapping correlation between the levels according to the finest feature point set;

(c) loading the identified feature of the facial image as a current level;

(d) adjusting features of a next finer level with the features of the current level;

(e) repeating step (d) until a finest feature is available; and

(f) constructing the geometry data with the finest feature.

5. The automatic animation production system as claimed in claim 1, wherein the scenario template data further comprising:

(a) a plurality of groups of animation part data for presenting sequential animations;

(b) each animation part data comprising a plurality of groups of animation state data for indexing or randomly expanding to sections of the animation part data;

(c) animation parameters data corresponding to each group of animation state data; and

(d) a hierarchical data structure comprising the animation part data;

animation state data and animation parameters data.

6. The automatic animation production system as claimed in claim 2, wherein the scenario template data further comprising:

(a) a plurality of groups of animation part data for presenting sequential animations;

(b) each animation part data comprising a plurality of groups of animation state data for indexing or randomly expanding to sections of the animation part data;

(c) animation parameters data corresponding to each group of animation state data; and

(d) a hierarchical data structure comprising the animation part data;

animation state data and animation parameters data.

7. The automatic animation production system as claimed in claim 1 wherein a expansion process in the scenario template processing module comprises the following steps:

(a) dividing the audio or events into groups as many as the animation parts data in the scenario template data;

(b) distributing animation parts of the scenario template on the divided group of the audio or events and maintaining a sequence of the animation parts data;

(c) distributing the animation states data of the scenario template data to constitute animation parts data according to the index or a probability model matching; and

(d) distributing the animation parameters data of the scenario template data for outputting the dynamic sequence of the animation parameters corresponding to the animation state data.

8. The automatic animation production system as claimed in claim 2, wherein a expansion process in the scenario template processing module comprises the following steps:

(a) dividing the events into groups as many as the animation parts data in the scenario template data;

(b) distributing animation parts of the scenario template on the divided group of the events and maintaining a sequence of the animation parts data;

(c) distributing the animation states data of the scenario template data to constitute animation parts data according to the index or a probability model matching; and

(d) distributing the animation parameters data of the scenario template data for outputting the dynamic sequence of the animation parameters corresponding to the animation state data.

9. The automatic animation production system as claimed in claim 1, wherein the scenario template comprises a dynamic sequence of animation parameters with variations of facial feature, texture of facial image or cartoon symbols.

10. The automatic animation production system as claimed in claim 2 wherein the scenario template comprises a dynamic sequence of animation parameters with variations of facial feature, texture of facial image or cartoon symbols.

11. A automatic animation production method, comprising the following steps:

(a) preparing geometry data for a animation production module

(b) loading scenario templates data selected manually or automatically from a scenario template database using a scenario selection interface;

(c) expanding the selected scenario template data to generate a dynamic sequence of animation parameters based on audio or events triggered by audio signals using a scenario template processing module to create animations; and

(d) receiving the dynamic sequence of animation parameters to generate animation frames using the animation production module.

12. The automatic animation production method as claimed in claim 11, wherein the step (a) further comprises the following steps:

(a1) loading a facial image;

(a2) recognizing and positioning features of the facial image using a feature detection module; and

(a3) constructing geometry data according to the recognized features using a geometry constructing module.

13. The automatic animation production method as claimed in claim 11, wherein the step (c) further comprises the following steps:

(c1) loading audio data

(c2) analyzing the audio data to generate the events using a audio analysis module; and

(c3) expanding the selected scenario template data to generate a dynamic sequence of animation parameters based on the events using a scenario template processing module to create animations.

14. The automatic animation production system as claimed in claim 13, wherein the step (c3) further comprising the following steps:

(c3-1) dividing the events into groups as many as the animation parts data in the scenario template data;

(c3-2) distributing the animation parts data of the scenario template data on the divided groups of the events and maintaining a sequence of the animation parts data;

(c3-3) distributing animation states data of the scenario template to constitute the animation parts data according to a index or a probability model matching; and

(c3-4) distributing the animation parameters data of the scenario template data for outputting the dynamic sequence of the animation parameters corresponding to the animation state data.

15. The automatic animation production method as claimed in claim 11, wherein the order of steps (a)(b)(c)(d) can be (b)(c)(a)(d).

16. The automatic animation production system as claimed in claim 11, wherein the scenario template data comprises a dynamic sequence of animation parameters with variations of facial feature, texture of facial image or cartoon symbols.

17. The automatic animation production system as claimed in claim 12, wherein the scenario template data comprises a dynamic sequence of animation parameters with variations of facial feature, texture of facial image or cartoon symbols.

18. The automatic animation production system as claimed in claim 13, wherein the scenario template data comprises a dynamic sequence of animation parameters with variations of facial feature, texture of facial image or cartoon symbols.

19. The automatic animation production system as claimed in claim 14, wherein the scenario template data comprises a dynamic sequence of animation parameters with variations of facial feature, texture of facial image or cartoon symbols.