Method and apparatus for multiplexing separately-authored metadata for insertion into a video data stream

• US 6,357,042 B2
• Filed: 01/22/1999
• Issued: 03/12/2002
• Est. Priority Date: 09/16/1998
• Status: Expired due to Fees

• Current Assignee: Tata America International Corporation (Tata Sons Pvt Ltd.)
• Original Assignee: HOTV Incorporated
• Inventors: Srinivasan, Anand, Bhadada, Kamal, Shah, Mehul Y, Rangan, P Venkat, Chakraborty, Indranil, Mardikar, Mohan
• Primary Examiner(s): Faile, Andrew
• Assistant Examiner(s): Srivastava, Vivek
• Application Number:
  US09/235,781
• Publication Number:
  US 20010023436A1
• Time in Patent Office: 1,145 Days
• Field of Search: G06/F 1.5167, H04/N.100, H04/N.117, H04/N.7025, H04/N.711, 709/217, 345/327, 345/328, 348/12, 348/13, 348/10, 348/6, 348/7, 455/6.1, 455/6.2, 455/6.3, 455/3.1, 455/3.2, 370/503-519, 725/119, 725/143, 725/32, 725/36
• US Class Current: 725/32
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: H04N 21/23424 involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
• CPC Class Codes: H04N 21/234318 by decomposing into objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/23614 Multiplexing of additional data and video streams arrangements for simultaneous broadcast of plural pieces of information H04H20/28
• CPC Class Codes: H04N 21/2368 Multiplexing of audio and video streams
• CPC Class Codes: H04N 21/242 Synchronization processes, e.g. processing of PCR [Program Clock References] arrangements for synchronising broadcast or distribution via plural systems in broadcast distribution systems H04H20/18
• CPC Class Codes: H04N 21/25891 being end-user preferences retrieval of video data in a video database based on user preferences G06F16/739; arrangements for recognizing users' preferences H04H60/46; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/2668 Creating a channel for a dedicated end-user group, e.g. insertion of targeted commercials based on end-user profiles information retrieval from the Internet by querying with filtering and personalisation G06F16/9535; arrangements for replacing or switching information during the broadcast H04H20/10; push services over packet-switching network H04L12/1859; adaptation of message content in packet-switching networks H04L51/063
• CPC Class Codes: H04N 21/42204 User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205
• CPC Class Codes: H04N 21/426 Internal components of the client ; Characteristics thereof H04N5/44 takes precedence
• CPC Class Codes: H04N 21/4316 for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/4348 Demultiplexing of additional data and video streams
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/4622 Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet web site content organization and management for information retrieval from the Internet G06F16/958; transmission by internet of broadcast information H04H60/82; stock exchange data over packet-switching network H04L12/1804; push services including data channel over packet-switching network H04L12/1859
• CPC Class Codes: H04N 21/47 End-user applications
• CPC Class Codes: H04N 21/4725 using interactive regions of the image, e.g. hot spots details of information retrieval from the Internet by using URLs G06F16/955; processing chained hypermedia data for information retrieval G06F16/94
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 21/8455 involving pointers to the content, e.g. pointers to the I-frames of the video stream
• CPC Class Codes: H04N 21/854 : Content authoring
• CPC Class Codes: H04N 21/8547 : involving timestamps for synchronizing content
• CPC Class Codes: H04N 21/858 : Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot
• CPC Class Codes: H04N 21/8583 : by creating hot-spots
• CPC Class Codes: H04N 5/04 : Synchronising for television systems using pulse code modulation H04N7/56
• CPC Class Codes: H04N 5/073 : for mutually locking plural sources of synchronising signals, e.g. studios or relay stations
• CPC Class Codes: H04N 7/083 : with signal insertion during the vertical and the horizontal blanking interval , e.g. MAC data signals
• CPC Class Codes: H04N 7/088 : the inserted signal being digital
• CPC Class Codes: H04N 7/165 : Centralised control of user terminal subsequent to an upstream request signal H04N7/17345; Registering at central by two-way working H04N7/17309

Abstract

An authoring system for interactive video has two or more authoring stations for providing authored metadata to be related to a main video data stream and a multiplexer for relating authored metadata from the authoring sources to the main video data stream. The authoring stations annotate created metadata with presentation time stamps (PTS) from the main video stream, and the multiplexer relates the metadata to the main video stream by the PTS signatures. In analog streams PTS may be created and integrated. In some embodiments there may be multiple and cascaded systems, and some sources may be stored sources. Various methods are disclosed for monitoring and compensating time differences among sources to ensure time coordination in end product. In different embodiments transport of metadata to an end user station is provided by Internet streaming, VBI insertion or by Internet downloading. User equipment is enhanced with hardware and software to coordinate and present authored material with the main data stream.

Claims

1. An authoring system for interactive video, comprising:

video feeds providing video streams;

two or more authoring stations coupled to the video feeds providing authoring functions creating intermittent metadata for enhancing the video streams; and

a multiplexer for coordinating authored metadata with the video streams;

wherein, in the creation of metadata, the authoring stations note a presentation time stamp (PTS) of video frames or a synchronized system time from the video streams at the point in the video streams the authored metadata is to be later inserted, and incorporate the PTS in the authored metadata for matching the authored metadata with the video streams, and wherein time clocks are monitored for separate stream sources and clocks are adjusted to compensate for real-time differences in sources.

2. The system of claim 1 wherein one or more of the stream sources is from a stored source.

3. The system of claim 1 wherein PTS values are rewritten in one or more video streams to compensate for perceived time differences.

4. The system of claim 1 wherein the PTS-enhanced metadata is streamed over the Internet to an end user.

5. The system of claim 1 wherein the PTS-enhanced metadata is inserted into video blanking intervals (VBI) of an analog stream according to the PTS.

6. The system of claim 1 wherein the PTS-enhanced metadata is stored to be downloaded as needed by a user.

7. The system of claim 1 wherein the authoring stations include one or more of scene authoring, hyper-video authoring, and ad authoring stations.

8. The system of claim 1 wherein the video streams are analog streams, and the analog streams are converted to a digital format before authoring and multiplexing.

9. The system of claim 1 wherein the video streams are analog streams, and a presentation time stamp is generated and integrated with analog steams.

10. The system of claim 1 further comprising a user system enhanced with software for rendering the video streams and the authored metadata according to the PTS.

11. A method for coordinating authored video metadata with video data streams, comprising steps of:

(a) ensuring the video data streams have presentation time stamps (PTS);

(b) feeding the video data streams to authoring stations;

(c) authoring intermittent metadata at the authoring stations to be later used to enhance the video data streams;

(d) marking the metadata with presentation time stamps (PTS) taken from the video data streams at the time of authoring the metadata; and

(e) multiplexing the authored metadata with the video data streams, wherein the multiplexer places the metadata in relation to video data streams according to the PTS, and multiple sources of video is fed to the multiplexer as well as metadata, and the multiplexer compensates for real-time differences between the multiple sources.

12. The method of claim 11 wherein, in the compensating step, presentation time stamps (PTS) are amended according to source time differences.

13. The method of claim 11 wherein the PTS-enhanced metadata is streamed over the Internet to an end user.

14. The method of claim 11 wherein the PTS-enhanced metadata is inserted into video blanking intervals (VBI) of an analog stream according to the PTS.

15. The method of claim 11 wherein the PTS-enhanced metadata is stored to be downloaded as needed by a user.

16. The method of claim 11 wherein the authoring stations include one or more of scene authoring, hyper-video authoring, and ad authoring stations.

17. The method of claim 11 wherein the video data streams are analog steams, and the analog streams are converted to a digital format before authoring and multiplexing.

18. The method of claim 14 further comprising a step for displaying the video data streams and authored metadata according to PTS at a user'"'"'s station.

19. A video multiplexing system comprising:

inputs for digital video data streams; and

inputs from video authoring stations, comprising intermittent metadata associated with the video data streams by presentation time stamps (PTS) copied from the video data streams at the time of authoring; and

an output to a video transport interface;

wherein one or more of the inputs for digital video data streams are from stored sources, and the multiplexing system notes the presentation time stamps associated with the authored metadata, monitors real time clocks of the sources and uses the information to compensate one or both of the multiple streams, and places the authored metadata relative to the video data streams for transport to end users.

20. The multiplexing system of claim 19 wherein the multiplexer compensates incoming streams by buffering one or more of the streams.

21. The system of claim 19 wherein the multiplexer compensates incoming streams by amending the presentation time stamps of one or more of the digital video data streams.