Post-synchronizing an information stream including lip objects replacement

• US 7,145,606 B2
• Filed: 01/06/2004
• Issued: 12/05/2006
• Est. Priority Date: 06/24/1999
• Status: Expired due to Fees

• Current Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Original Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Inventors: Haisma, Nicoline, Van Gestel, Henricus Antonius Wilhelmus, Bergevoet, Bas Arnold Jan, Sinke, Johannes Govert
• Primary Examiner(s): Kostak, Victor R.
• Application Number:
  US10/752,366
• Publication Number:
  US 20040141093A1
• Time in Patent Office: 1,064 Days
• Field of Search: 348/515, 348/578, 348/473, 348/553, 348/480, 348/484, 345/473, 434/185, 725/32, 704/235, 704/270, 704/278, 704/276
• US Class Current: 348/515
• CPC Class Codes: G06T 13/205 driven by audio data
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G06T 9/001 Model-based coding, e.g. wire frame

Abstract

A method for post-synchronizing information stream includes obtaining lip-objects from a video signal. The original lip-objects are replaced with new lip-objects which correspond to a translated audio signal. The new lip-objects may be obtained by tracking a further video signal or by using a database with visemes or lip-parameters. For a multi-language information stream, a desired language may be selected at the receiver.

Claims

1. A method of transmitting an information stream comprising a video signal and an original audio signal, the method comprising the acts of:

obtaining from said video signal original lip-objects

obtaining at least one translated audio signal relating to a different language than the original audio signal, and

adding new lip-objects to the information stream, which new lip-objects are each linked to the least one translated audio signal,

wherein the least one translated audio signal is obtained by performing a translation process comprising the acts of;

converting the original audio signal into translated text; and

deriving the least one translated audio signal and said new lip-objects from said translated text.

2. The method according to claim 1, wherein said new lip-objects are obtained by tracking at least one further video signal, said further video signal comprising lip-movements corresponding to said translated audio signal.

3. The method according to claim 1, wherein said converting acts comprises:

dividing the original audio signal into original phonemes;

converting said original phonemes into text; and

translating said text into said translated text.

4. A transmitter for transmitting an information stream comprising a video signal an original audio signal, the transmitter comprising:

means for obtaining from said video signal original lip-objects,

means for obtaining at least one translated audio signal relating to a different language than the original audio signal, and

means for adding new lip-objects to the information stream, which new lip-objects are each linked to the least one translated audio signal, and

means for converting the original audio signal into translated text by dividing the original audio signal into original phonemes, converting said original phonemes into text, and translating said text into said translated text.

5. A receiver for receiving an information stream comprising a video signal, a plurality of audio signals related to different languages and a plurality of lip-objects which lip-objects are each linked to at least one of said plurality of audio signals, which receiver comprises:

a selector for obtaining a selected audio signal from said plurality of audio signals; and

outputting means for outputting said selected audio signal and said video signal, said video signal comprising selected lip-objects, which selected lip-objects are linked to said selected audio signal,

wherein said selected audio signal is converted into translated text by dividing an on audio signal into original phonemes, converting said original phonemes into text, and translating said text into said translated text.

6. A communication system comprising:

a plurality of stations comprising means for transmitting and means for receiving an information stream, which information stream comprises a video signal and an original audio signal,

a communication network for linking said stations;

wherein the communication system comprising;

means for performing a translation process to obtain at least one translated audio signal;

means for tracking said video signal to obtain original lip-objects;

means for adding to the information stream new lip-objects corresponding to said translated audio signal, in addition to said original lip-objects; and

means for converting the original audio signal into translated text by dividing the original audio signal into original phonemes, converting said original phonemes into text, and translating said text into said translated text.

7. An information stream embedded in a carrier wave comprising a video signal and a plurality of audio signals relating to different languages;

and

said information stream being configured to cause linking a plurality of lip-objects to at least one of said plurality of audio signals;

at least a portion of said information stream being derived from converting an original audio signal into translated text by dividing the original audio signal into original phonemes, converting said original phonemes into text, and translating said text into said translated text.

8. A storage medium on which an information stream as claimed in claim 7 has been recorded.

9. A method of post-synchronizing an information stream, which information stream comprises an audio signal and a video signal, the method comprising the acts of:

performing a translation process to obtain at least one translated audio signal by converting the audio signal into translated text, and deriving said at least one translated audio signal and new lip-objects from said translated text;

tracking said video signal to obtain original lip-objects;

replacing said original lip-objects with said new lip-objects, said new lip-objects corresponding to said at least one translated audio signal, wherein said new lip-objects are obtained by tracking at least one further video signal, said further video signal comprising lip-movements corresponding to said at least one translated audio signal.

10. A method of post-synchronizing an information stream, which information stream comprises an original audio signal and a video signal, the method comprising the acts of:

performing a translation process to obtain at least one translated audio signal,

tracking said video signal to obtain original lip-objects;

replacing said original lip-objects with new lip-objects, said new lip-objects corresponding to said at least one translated audio signal,

wherein said translation process comprises the acts of;

converting the original audio signal into translated text; and

driving said at least one translated audio signal and said new lip-objects from said translated text,

wherein said converting act comprises;

dividing the original audio signal into original phonemes;

converting said original phonemes into text; and

translating said text into said translated text.

11. A device for post-synchronizing an information stream, which information stream comprises an audio signal and a video signal, the device comprising:

means for performing a translation process to obtain at least one translated audio signal,

means for tracking said video signal to obtain original lip-objects;

means for replacing said original lip-objects with new lip-objects, said new lip-objects corresponding to said at least one translated audio signal, wherein said new lip-objects are obtained by tracking at least one further video signal, said further video signal comprising lip-movements corresponding to said at least one translated audio signal;

wherein said performing means comprises means for converting the audio signal into translated text by dividing the audio signal into phonemes;

converting said phonemes into text; and

translating said text into said translated text.

12. A device for post-synchronizing an information stream, which information stream comprises an original audio signal and a video signal, the device comprising:

translation means for performing a translation process to obtain at least one translated audio signal,

means for tracking said video signal to obtain original lip-objects;

means for replacing said original lip-objects with new lip-objects, said new lip-objects corresponding to said at least one translated audio signal,

wherein said translation means comprises;

means for converting the original audio signal into translated text; and

means for deriving said translated audio signal and said new lip-objects from said translated text,

wherein said converting means comprises;

means for dividing the original audio signal into original phonemes;

means for converting said original phonemes into text; and

means for translating said text into said translated text.