Method and Apparatus for Character Animation

• US 20120026174A1
• Filed: 04/27/2010
• Published: 02/02/2012
• Est. Priority Date: 04/27/2009
• Status: Abandoned Application

• Current Assignee: Sonoma Data Solutions, LLC
• Original Assignee: Sonoma Data Solutions, LLC
• Inventors: McKeon, Thomas F., Molinari, John
• Application Number:
  US13/263,909
• Publication Number:
  US 20120026174A1
• Time in Patent Office: Days
• US Class Current: 345/473
• CPC Class Codes: G06T 13/205 driven by audio data
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G10L 2021/105 Synthesis of the lips movements from speech, e.g. for talking heads

Abstract

The present invention provides various means for the animation of character expression in coordination with an audio sound track. The animator selects or creates characters and expressive characteristic from a menu, and then enters the characteristics, including lip and mouth morphology, in coordination with a running sound track.

Claims

1. A method of character animation, the method comprising:

a) providing a general purpose computer having an electronic display and at least one user input means,

b) providing a data structure having at least a first and second data field, in which;

i) the first data field has at least one digital image that is a general facial portrait of a character to be animated on the electronic display, and

ii) the second data field has a first series of images that correspond to at least a portion of the facial morphology of the character to be animated that changes when the character to be animated appears to speaks, wherein each image of said first series is associated with a specific phoneme and is selectable via the user input means,

c) at least one of playing an audio sound track and reading a script to determine the sequence and duration of the phonemes intended to be spoken by the character to be animated,

d) selecting the appropriate phoneme via the user input means,

e) wherein the step of selecting the appropriate phoneme via the user input means causes the image associated with a specific phoneme to be overlaid on the general facial portrait image in temporal coordination with the sound track or script on the electronic display.

2. A method of character animation according to claim 1 further comprising providing a third data field having a second series of images that correspond to at least a portion of the facial morphology related to the emotional state of the character to be animated, wherein each image of the second series is associated with a specific emotional state and is selectable via the computer user input device.

3. A method of character animation according to claim 2 further wherein said step of:

a) at least one of playing an audio sound track and reading a script to determine the sequence and duration of the phonemes intended to be spoken by the character to be animated comprising comprises listening to a digital sound track to determine the emotional state of the animated character, and the additional step of;

b) causing the image that is associated with the appropriate emotional state to be overlaid on the general facial portrait image time in temporal coordination to the digital sound track on the electronic display by selecting the appropriate emotional state via the user input device.

4. A method of character animation according to claim 3 wherein;

a) said step of at least one of playing an audio sound track and reading a script to determine the sequence and duration of the phonemes intended to be spoken by the character to be animated comprising comprises listening to a digital sound track to determine the emotional state of the animated character, and;

b) wherein said step of causing the image that is associated with the appropriate emotional state to be overlaid on the general facial portrait image time in temporal coordination to the digital sound track on the electronic display by selecting the appropriate emotional state via the user input device causes a different image for at least one of the specific phoneme to be overlaid on the general facial portrait image on the electronic display in temporal coordination with the audio sound track than if another emotional state where selected.

5. A method of character animation according to claim 1 further comprising the step of changing at least one image from the first series of images after said step of selecting the appropriate phoneme associated with the changed image, said step of changing the at least one image being operative to change the appearance of all the further appearances of the at least one images that is overlaid on the general facial portrait image in temporal coordination with the digital sound track electronic display.

6. A method of character animation according to claim 2 further comprising the step of changing at least one image from the second series of images after said step of selecting the appropriate emotional state associated with the changed image, said step of changing the at least one image being operative to change the appearance of all the further appearance of the at least one images that is overlaid on the general facial portrait image in temporal coordination with the digital sound track.

7. A method of character animation according to claim 1 wherein the user input means is a keyboard.

8. A method of character animation according to claim 7 wherein the phoneme is selectable by a first key on the keyboard corresponding to the letter representing the sound of the phoneme and a second key on the keyboard to modify the phoneme selection by the length of the sound.

9. A method of character animation according to claim 8 wherein the second key on the keyboard does not represent a specific letter.

10. A computer readable media having a data structure for creating animated video frame sequences of characters, the data structure comprising:

a) a first data field containing data representing a phoneme that correlates with a selection mode of a computer user input device,

b) a second data field containing data that is at least one of representing or being associated with an image of the pronunciation of the phoneme contained in the first data field.

20. A method of configuring a general purpose computer for creating animated video frame sequences of characters according to claim 19 wherein the computer readable media further comprises the data structure of claim 10.

11. A computer readable media having a data structure for creating animated video frame sequences of characters, the data structure comprising:

a) a first data field containing data representing an emotional state that correlates with a selection mode of a computer user input device,

b) a second data field containing data that is at least one of representing or being associated with at least a portion of a facial image associated with a particular emotional state contained in the first data field.

12. A computer readable media having a data structure for creating animated video frame sequences of characters according to claim 11 further comprising,

a) a third data field containing data representing a phoneme,

b) a fourth data field containing data that is at least one of representing or being associated with an image of the pronunciation of the phoneme contained in the third data field.

13. A computer readable media having a data structure for creating animated video frame sequences of characters according to claim 12 further comprising,

a) a fifth data field containing data representing a phoneme,

b) a sixth data field containing data that is at least one of representing or being associated with an image of the pronunciation of the phoneme contained in the sixth data field.

c) wherein one of the emotional states in the first and second data fields is associated with the third and fourth data fields, and another of the emotional states in the first and second data fields is associated with the fifth and sixth data fields.

14. A GUI for character animation, the GUI comprising:

a) a first frame for displaying a graphical representation of the time elapsed in the play of a digital sound file,

b) a second frame for displaying at least parts of an image of an animated character for a video frame sequence in synchronization with the digital sound file that is graphically represented in the first frame,

c) at least one of an additional frame or a portion of the first and second frame for displaying a symbolic representation of the facial morphology for the animated character to be displayed in the second frame for at least a portion of the graphical representation of the time track in the first frame.

15. A GUI for character animation according to claim 14 wherein the facial morphology display in the at least one additional frame corresponds to different emotional states of the character to be animated with the GUI.

16. A GUI for character animation according to claim 14 wherein the facial morphology display in the at least one additional frame corresponds to the appearance of different phoneme as if the character to be animated were speaking.

17. A GUI for character animation according to claim 14 further comprising sub-frames of variable widths of elapsed playtime corresponding with the digital sound file to indicate the alternative parametric representation of the facial morphology.

19. A method of configuring a general purpose computer for creating animated video frame sequences of characters, the method comprising the steps of:

a) providing a computer readable media having thereon a set of computer instructions that is operative to create the GUI of claim 14.

18. A method of character animation, the method comprising:

a) providing a general purpose computer having an electronic display and at least one user input means,

b) providing a data structure having at least a first and second data field, in which;

i) the first data field has at least one digital image that is a general facial portrait of a character to be animated on the electronic display, and

ii) the second data field has a first series of images that correspond to at least a portion of the facial morphology of the character to be animated that changes when the character to be animated speaks, wherein each image of said first series is associated with a specific phoneme and is selectable via the user input device,

c) providing a means to select in sequence a plurality of phoneme from the second data field,

d) displaying the general facial portrait of the character to be animated on the electronic display,

e) wherein upon detection of a selected phoneme the general purpose computer is operative to overlay a corresponding image from the first series of image of the second data field on the general facial portrait image of the character to be animated on the electronic display.