Method, apparatus and computer program product for synchronizing separate compressed video and text streams to provide closed captioning and instant messaging integration with video conferencing

• US 20070064095A1
• Filed: 09/13/2005
• Published: 03/22/2007
• Est. Priority Date: 09/13/2005
• Status: Active Grant

• Current Assignee: AirBNB Incorporated
• Original Assignee: International Business Machines Corporation
• Inventors: Stecher, John, Baartman, Randall, Branda, Steven
• Granted Patent: US 7,561,178 B2
• Time in Patent Office: Days
• US Class Current: 348/14.80
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/23614 Multiplexing of additional data and video streams arrangements for simultaneous broadcast of plural pieces of information H04H20/28
• CPC Class Codes: H04N 21/43074 of additional data with content streams on the same device, e.g. of EPG data or interactive icon with a TV program
• CPC Class Codes: H04N 21/4348 Demultiplexing of additional data and video streams
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/4884 for displaying subtitles
• CPC Class Codes: H04N 21/8126 involving additional data, e.g. news, sports, stocks, weather forecasts
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

A method, apparatus and computer program product are provided for synchronizing separate compressed video and text streams to provide lightweight closed captioning and instant messaging integration with video conferencing. A video encoder encodes a video stream and periodically generates a synchronization frame event. Each generated synchronization frame event has a unique ID. A text recording agent receives the periodically generated synchronization frame events, and generates text packets associating stored text with the synchronization frame event. A video decoder decodes the video stream, periodically generating the synchronization frame event having the unique ID. A text display agent receives the periodically generated synchronization frame events and associates stored text packets with the synchronization frame events.

Claims

1. Apparatus for synchronizing separate compressed video and text streams to provide closed captioning and instant messaging integration with video conferencing comprising:

a video encoder for encoding a video stream and periodically generating a synchronization frame event, each generated synchronization frame event having a unique ID;

a text recording agent coupled to said video encoder receiving said periodically generated synchronization frame event and for storing text and associating said stored text with said synchronization frame event;

a video decoder for decoding said video stream and periodically generating said synchronization frame event having said unique ID; and

a text display agent coupled to said video decoder receiving said periodically generated synchronization frame event and for storing text packets and associating said stored text packet with said synchronization frame event.

2. Apparatus for synchronizing separate compressed video and text streams as recited in claim 1 wherein said video encoder includes an instrumented streaming video encoder.

3. Apparatus for synchronizing separate compressed video and text streams as recited in claim 1 wherein said video encoder identifies a key frame, and responsive to said identified key frame generates said synchronization frame event.

4. Apparatus for synchronizing separate compressed video and text streams as recited in claim 1 wherein said video decoder generates said synchronization frame event responsive to decoding said key frame.

5. Apparatus for synchronizing separate compressed video and text streams as recited in claim 1 wherein said video encoder inserts a synchronization frame into said video stream together with generating said synchronization frame event.

6. Apparatus for synchronizing separate compressed video and text streams as recited in claim 5 wherein said video decoder generates said synchronization frame event responsive to decoding said synchronization frame.

7. Apparatus for synchronizing separate compressed video and text streams as recited in claim 1 wherein said video encoder and said video decoder generating said synchronization frame event having said unique ID includes using a hashing algorithm.

8. Apparatus for synchronizing separate compressed video and text streams as recited in claim 1 wherein said video encoder and said video decoder generating said synchronization frame event having said unique ID includes receiving encoded frame data as an input.

9. Apparatus for synchronizing separate compressed video and text streams as recited in claim 1 wherein said video decoder includes an instrumented streaming video decoder.

10. Apparatus for synchronizing separate compressed video and text streams as recited in claim 1 wherein said text recording agent generates text packets, each text packet including said unique ID and said associated text and adds each said text packet to a text stream.

11. Apparatus for synchronizing separate compressed video and text streams as recited in claim 10 wherein said text display agent processes stored text packets and transfers text data for display with said decoded video.

12. Apparatus for synchronizing separate compressed video and text streams as recited in claim 1 includes a plurality of said text recording agents, each for sending a text stream in a selected language to viewers over a network.

13. A computer program product for synchronizing separate compressed video and text streams to provide closed captioning and instant messaging integration with video conferencing in an encoder and decoder system, said computer program product including instructions executed by the encoder and decoder system to cause the encoder and decoder system to perform the steps of:

encoding a video stream and periodically generating a synchronization frame event, each generated synchronization frame event having a unique ID;

receiving said periodically generated synchronization frame event and generating text packets including associating stored text with said synchronization frame event;

decoding said video stream and periodically generating said synchronization frame event having said unique ID;

receiving said periodically generated synchronization frame event; and

associating said stored text packet with said synchronization frame event

14. A computer program product for synchronizing separate compressed video and text streams as recited in claim 13 wherein generating said synchronization frame event having said unique ID includes receiving encoded frame data as an input.

15. A method for synchronizing separate compressed video and text streams to provide closed captioning and instant messaging integration with video conferencing comprising the steps of:

encoding a video stream and periodically generating synchronization frame events, each generated synchronization frame event having a unique ID;

providing a text recording agent for receiving said periodically generated synchronization frame event and generating text packets associating stored text with said synchronization frame events;

decoding said video stream and periodically generating said synchronization frame event having said unique ID; and

providing a text display agent for receiving and storing text packets, receiving said periodically generated synchronization frame event and associating stored text packets with said synchronization frame events.

16. A method for synchronizing separate compressed video and text streams as recited in claim 15 wherein generating said synchronization frame event having said unique ID includes receiving encoded frame data as an input.

17. A method for synchronizing separate compressed video and text streams as recited in claim 16 further includes using a hashing algorithm for generating said synchronization frame event having said unique ID.

18. A method for synchronizing separate compressed video and text streams as recited in claim 15 includes providing an instrumented streaming video encoder for encoding said video stream and periodically generating synchronization frame events.

19. A method for synchronizing separate compressed video and text streams as recited in claim 15 includes providing an instrumented streaming video decoder for decoding said video stream and periodically generating synchronization frame events.

20. A method for synchronizing separate compressed video and text streams as recited in claim 15 includes providing a plurality of said text recording agents, each for sending a text stream in a selected language to viewers over a network.