Method and means for computer sychronization of actions and sounds

• US 5,430,835 A
• Filed: 05/26/1994
• Issued: 07/04/1995
• Est. Priority Date: 02/15/1991
• Status: Expired due to Term

• Current Assignee: Sierra Entertainment, Inc. (Vivendi SE)
• Original Assignee: Sierra On-Line, Incorporated (Vivendi SE)
• Inventors: Heitman, Robert E., Williams, Kenneth A., Smith, Christopher F.
• Primary Examiner(s): Zimmerman, Mark K.
• Assistant Examiner(s): BURRASTON, NEIL
• Application Number:
  US08/250,098
• Time in Patent Office: 404 Days
• Field of Search: 395/152, 395/153, 395/154, 395/161, 345/122, 360/79, 40/457
• US Class Current: 715/203
• CPC Class Codes: H04N 21/4143 embedded in a Personal Computer [PC]
• CPC Class Codes: H04N 21/43074 of additional data with content streams on the same device, e.g. of EPG data or interactive icon with a TV program
• CPC Class Codes: H04N 21/4781 Games

Abstract

The invention includes a method of synchronizing actions and sounds for display on the visual display in a computer-system. The invention also includes a simulation including synchronized actions and sounds. Specifically, the method of synchronizing actions and sounds for display on a computer system having a visual display includes determining locations in a sound recording where predetermined actions are to

be displayed, measuring the time that elapses when the sound recording is played from reference point to the locations in the sound recording where the predetermined actions are to be displayed, associating the predetermined actions with the locations in the sound recording where the predetermined actions are to be displayed, and saving the locations and their associated actions. The invented method of synchronizing actions and sounds on a visual display in a computer system includes playing the sound, determining when a predetermined amount of time of the sound has elapsed, and displaying the action on the visual display when the predetermined amount of time of the sound has elapsed.

The simulation includes a sound recording, a mechanism for playing it, and a mechanism for measuring its elapsed time when it played; data defining predetermined actions; a mechanism to compare such elapsed time with time positions; and a mechanism to display the actions upon a predetermined relationship between the elapsed time and the time positions.

Claims

1. In a computer video game system having a sound generator and a visual display showing an image, a method of synchronizing actions of the image with a sound recording that has a given time duration, where each action is defined by data that is associated with a time position that signals when the action is to be displayed, the method comprising:

reading a first time position associated with data describing a first action;

playing the sound recording on the sound generator;

determining the elapsed time of the sound recording;

comparing the first time position with the elapsed time; and

if the first time position is greater than the elapsed time, then waiting a predetermined period of time and repeating the method beginning with the step of determining;

but

if the first time position is less than or equal to the elapsed time, then performing the steps of

recognizing a second time position associated with data, describing a second action;

contrasting the second time position with the elapsed time; and

if the second time position is greater than the elapsed time, then displaying the first action and repeating the method for subsequent time positions associated with data describing subsequent actions beginning with the step of determining;

but

if the second time position is less than or equal to the elapsed time, then repeating the method for subsequent time positions associated with data describing subsequent actions beginning with the step of recognizing.

2. The method of claim 1 where each time position indicates the amount of time of the sound recording that must elapse before the respective action is displayed.

3. The method of claim 1 where the step of displaying is also accomplished when there is no further data describing subsequent actions.

4. The method of claim 1 where the steps of waiting and repeating beginning with the step of determining are accomplished when there is further data describing subsequent actions and when there is further time remaining for the playing of the sound recording.

5. The method of claim 1 where the image is a character, the sound recording is a speech sample and the actions include mouth positions.

6. The method of claim 1 where the image is a character and the actions include character gestures.

7. In a computer video game having a sound generator for playing sound samples from a sound recording composed of a plurality of sound samples and a visual display for displaying an image, a method of synchronizing incorporation of an action in the image with the sound recording, where the action is selected from a predefined set of available actions, the method comprising:

selecting one or more of the sound samples to synchronize;

identifying a set of predetermined time locations in each of the selected sound samples where an action is to be incorporated in the image;

associating a specific action with each predetermined time location;

playing one of the sound samples on the sound generator;

determining a current time location in the playing of the sound sample;

if the sound sample is one of the selected sound samples, then;

comparing the current time location with the predetermined time locations for the sound sample; and

if the current time location is one of the predetermined time locations, then incorporating the associated specific action in the image; and

if the sound sample is not one of the selected sound samples, then

for at least one of the current time locations randomly selecting an action from the set of available actions and incorporating it in the image.

8. The method of claim 7 wherein, the actions are a character'"'"'s mouth movements and the mouth movements are defined by mouth shapes stored in a memory device accessible to the computer video game.

9. The method of claim 7 wherein the image is an animated image having a characteristic animation cycle and the step of determining occurs once each animation cycle.

10. The method of claim 9 wherein the step of randomly selecting is carried out only if a specified number of animation cycles have passed since the last action was incorporated in the image.

11. The method of claim 9 wherein the step of randomly selecting is carried out only if a specified number of animation cycles have passed since the last action was incorporated in the image and the specified number of cycles is a randomly selected number between one and the number of animation cycles occurring during the sound recording.