Authoring and use systems for sound synchronized animation

• US 5,111,409 A
• Filed: 07/21/1989
• Issued: 05/05/1992
• Est. Priority Date: 07/21/1989
• Status: Expired due to Term

• Current Assignee: Sierra Entertainment, Inc. (Vivendi SE)
• Original Assignee: Brightmail, Inc. (Gen Digital Inc.)
• Inventors: Matthews, Joseph H. III, Gasper, Elon
• Primary Examiner(s): Herndon, Heather R.
• Application Number:
  US07/384,243
• Time in Patent Office: 1,019 Days
• Field of Search: 364/518, 364/521, 340/729, 434/169, 434/185, 434/167, 434/172
• US Class Current: 715/203
• CPC Class Codes: A63F 13/63 by the player, e.g. authoring using a level editor
• CPC Class Codes: A63F 2300/6018 where the game content is authored by the player, e.g. level editor or by game device at runtime, e.g. level is created from music data on CD
• CPC Class Codes: A63F 2300/6063 for sound processing
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: G06T 13/205 driven by audio data
• CPC Class Codes: G09B 19/04 Speaking with audible presentation of the material to be studied G09B5/04
• CPC Class Codes: G09B 19/06 Foreign languages with audible presentation of material to be studied G09B5/04
• CPC Class Codes: G09B 5/065 Combinations of audio and video presentations, e.g. videotapes, videodiscs, television systems
• CPC Class Codes: G10L 2021/105 Synthesis of the lips movements from speech, e.g. for talking heads
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/10 Indexing; Addressing; Timing or synchronising; Measuring tape travel

Abstract

A general purpose computer, such as a personal computer, is programmed for sound-synchronized random access and display of synthesized actors ("synactors") on a frame-by-frame basis. The interface between a user and the animation system is defined as a stage or acting metaphor. The user interface provides the capability to create files defining individually accessible synactors representing real or imaginary persons, animated characters and objects or scenes which can be programmed to perform speech synchronized action. Synactor speech is provided by well-known speech synthesis techniques or, alternatively, by inputting speech samples and communication characteristics to define a digital model of the speech and related animation for a particular synactor. A synactor is defined as combination of sixteen predefined images; eight images to be synchronized with speech and eight images to provide additional animated expression. Once created, a synactor may be manipulated similarly to a file or document in any application. Once created, a synactor is controlled with scripts defined and edited by a user via the user interface.

Claims

1. Apparatus for generating and displaying user created animated objects having synchronized visual and audio characteristics, said apparatus comprising:

a program-controlled microprocessor;

first means coupled to said microprocessor and responsive to user input signals for generation a first set of signals defining visual characteristics of a desired animated object;

second means coupled to said microprocessor and to said first means and responsive to user input signals for generating a second set of signals defining audio characteristics of said desired animated object; and

controller means coupled to said first and second means and to said microprocessor for generating a set of instructions collating and synchronizing said visual characteristics with said audio characteristics thereby defining said animated object having synchronized visual and audio characteristics.

2. The apparatus as in claim 1 further comprising:

integrator means coupled to said microprocessor and responsive to command signals generated by said microprocessor for producing signals representing encoded elements of sound and encoded elements of constituent object parts, said constituent object parts associated with said visual characteristics, said microprocessor responsive to user input signals and to said set of instructions for generating said command signals;

audio means coupled to said microprocessor and to said integrator means responsive to said signals representing encoded elements of sound for producing sounds associated with said signals representing encoded elements of sound; and

display means coupled to said microprocessor, to said integrator means and to said sound emitting means responsive to said signals representing encoded elements of constituent object parts for displaying visual images of said desired animated object, said visual images having said visual characteristics synchronized with said audio characteristics.

3. Apparatus as in claim 2 wherein said first means is further coupled to said display means, said display means responsive to said user input signals for displaying images of said visual characteristics as said first set of signals is being generated.

4. Apparatus as in claim 3 wherein said second means is further coupled to said display means and includes testing and editing means responsive to user input for displaying said desired animated object and testing and editing the synchronization of said audio characteristics with said visual characteristics as said second set of signals is being generated.

5. Apparatus as in claim 4 further comprising storage means coupled to said microprocessor for storing a plurality of data sets, at least one of said data sets defining the visual characteristics of a predetermined prototype animated object.

6. Apparatus as in claim 5 wherein said plurality of data sets include at least one data set defining the audio characteristics of selectable predetermined text.

7. Apparatus as in claim 5 wherein said plurality of data sets include at least one data set defining the audio characteristics of selectable prerecorded sounds.

8. Apparatus as in claim 2 wherein said audio means includes speech synthesizer means for digitally synthesizing signals representing sounds associated with said signals representing encoded elements of sound.

9. A method for generating user created animated objects having synchronized visual and audio characteristics, said method comprising the steps of:

generating a first set of signals defining visual characteristics of a desired animated object in response to user input signals;

generating a second set of signals defining audio characteristics of said desired animated object in response to user input signals; and

generating a set of instructions collating and synchronizing said visual characteristics with said audio characteristics thereby defining said desired animated object having synchronized visual and audio characteristics.

10. The method of claim 9 including the step of displaying visual images of said desired animated object during the generation of said first set of signals.

11. A method of synchronizing sound with visual images of animated objects pronouncing the sound, said method comprising the steps of:

defining a text string representing a desired sound to be synchronized with visual images of a speaking animated object;

translating said text string into a phonetic text string representative of said text string; and

translating said phonetic text string into a recite command, said recite command including phonetic/timing pairs, each of said phonetic/timing pairs comprising a phonetic code corresponding to an associated phonetic code of said phonetic text string and a number defining a predetermined time value, said phonetic code representative of a sound element to be pronounced and an associated image to be displayed while said sound element is being pronounced and said predetermined time value defines the amount of time said associated image is to be displayed.

12. A method as in claim 11 including the step of displaying said associated images during the pronounciation of said desired sound for testing the accuracy of the synchronization between said animated object and said pronounced desired sound.

13. A method as in claim 12 wherein said time value is adjustable, the further step of adjusting the value of said time value to edit and tune the accuracy of the synchronization between said animated object and said pronounced desired sound.