Method and apparatus for fast metadata generation, delivery and access for live broadcast program

• US 7,548,565 B2
• Filed: 02/19/2003
• Issued: 06/16/2009
• Est. Priority Date: 07/24/2000
• Status: Expired due to Term

• Current Assignee: Scenera, Inc.
• Original Assignee: VMark, Inc. (VALID Soluciones Tecnológicas SA)
• Inventors: Chung, Min Gyo, Yoon, Ja-Cheon, Kim, Hyeokman, Sull, Sanghoon
• Primary Examiner(s): NGUYEN BA, HOANG VU A
• Application Number:
  US10/369,333
• Publication Number:
  US 20030177503A1
• Time in Patent Office: 2,309 Days
• Field of Search: 370/503, 370/535, 725/39, 725/100, 725/105, 725/110, 725/111, 725/131, 725/151, 375/354, 715/721
• US Class Current: 370/503
• CPC Class Codes: G06F 16/71 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/7844 using original textual content or text extracted from visual content or transcript of audio data
• CPC Class Codes: G06F 16/7847 using low-level visual features of the video content
• CPC Class Codes: G06T 3/4092 Image resolution transcoding, e.g. by using client-server architectures
• CPC Class Codes: G11B 2220/20 Disc-shaped record carriers
• CPC Class Codes: G11B 2220/41 Flat as opposed to hierarchical combination, e.g. library of tapes or discs, CD changer, or groups of record carriers that together store one title
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/105 of operating discs
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D

Abstract

Techniques for fast indexing of live video broadcasts are provided which incorporate both efficient manual processing and automatic indexing steps to generate semantically meaningful and practically usable highlight hierarchy of broadcast television programs in real-time. In one technique, a list of predefined keywords is provided, describing the highlights, and the manual marking process can be implemented by just a few mouse clicks. A technique is provided for grouping highlights into a semantic hierarchy in real-time. A technique is provided for efficiently generating highlight metadata on live broadcast programs, using a coarse-to-fine indexing methodology in order for a operator to quickly generate highlight summaries of live broadcast programs.

Claims

1. A method of providing metadata service, comprising:

supplying a broadcast stream from a broadcaster to a metadata service provider and a user'"'"'s receiving device, said broadcast stream comprising TV and audio programs and system times in a System Time Table (STT) for Advanced Television System Committee broadcasts or a Time Data Table (TDT) for Digital Video Broadcasting broadcasts;

at the metadata service provider, extracting said system time from either said STT or said TDT;

transcoding said broadcast stream while associating each frame of said transcoded stream with said system times; and

analyzing said transcoded stream to generate metadata associated with said stream, said metadata having one or more timestamps; and

delivering said metadata having said timestamps to the user'"'"'s receiving device, whereby said timestamps are represented in said system times in STT or TDT to identify one or more particular locations corresponding to said timestamps in said received broadcast stream to which said metadata is applicable.

2. Method, according to claim 1, wherein:

the step of analyzing the broadcast stream and generating associated metadata is performed at a metadata service provider location that receives the broadcast stream from the broadcaster positioned at another location.

3. Method, according to claim 1, wherein:

the step of delivering the metadata to the user'"'"'s receiving device is performed through a back channel comprising at least one of Internet, Intranet, PSTN, cable modem, and satellite.

4. Method, according to claim 1, wherein:

the step of delivering the metadata to the user'"'"'s receiving device is performed by delivering the metadata to a multiplexer of the broadcaster through a feedback channel; and

broadcasting the metadata together with the broadcast stream.

5. Method, according to claim 1, wherein:

the step of analyzing the broadcast stream and generating associated metadata is performed before broadcasting.

6. Method according to claim 1, wherein:

the step of delivering the metadata to the user'"'"'s receiving device is performed by multiplexing the metadata with the broadcast stream.

7. Method, according to claim 1, wherein:

the metadata of a broadcast program is inserted into an electronic programming guide (EPG) data field and sent to a user.

8. Method, according to claim 1, wherein:

said timestamps are represented in media time defined by subtracting the start time that is described in an EPG for the broadcast program from said timestamps.

9. Method, according to claim 1, wherein the user'"'"'s receiving device comprises a digital set-top box (STB) with digital video recorder (DVR) capability.

10. Method, according to claim 1, wherein said associating each frame of said transcoded stream with said system times comprises stamping artificial visual patterns corresponding to system times onto a buffered version of said broadcast stream.

11. Method, according to claim 1, additionally comprising storing the broadcast stream in the user'"'"'s receiving device.

12. Method, according to claim 1, wherein said analyzing comprises a coarse indexing step and a fine indexing step.

13. Method, according to claim 1, wherein said analyzing is performed according to a template.

14. Method, according to claim 13, wherein said template includes a highlight template having a list of highlight categories.

15. Method of indexing an audio/video program, in real time, comprising:

coarse indexing by a processor-implemented highlight marker according to a plurality of highlight templates providing lists of categorized highlight themes, said coarse indexing comprising capturing desirable highlights by marking a moment of said highlights, and attaching relevant highlight themes selected from said highlight templates to said captured highlight in real time while viewing the video program, said highlight templates being stored in a computer-readable memory;

associating each frame of the audio/video program with system times extracted from a System Time Table (STT) or Time Data Table (TDT);

delivering said captured highlights with said attached themes to a processor-implemented main indexer; and

fine indexing by said main indexer comprising refining said coarse indexing by obtaining an exact start timestamp from the system times and duration for said captured highlight and associating the exact start timestamp and duration with the captured highlight.

16. Method, according to claim 15, wherein said plurality of highlight templates are provided before the start of said coarse indexing process.

17. Method of random accessing to one or more frames of a broadcast program, comprising:

receiving, at a receiving unit, an MPEG transport stream from a broadcasting network;

recording the MPEG transport stream;

reading information required for randomly accessing locations by analyzing the MPEG transport stream; and

randomly accessing locations in one of said recorded or received MPEG transport stream according to system times extracted from either a System Time Table (STT) for Advanced Television System Committee broadcasts or a Time Data Table (TDT) for Digital Video Broadcasts that are associated with each frame of the recorded or received transport streams.

18. Method, according to claim 17, wherein said reading information comprises:

extracting a system time and its byte position from one of said recorded or received transport streams;

generating a byte-offset table that contains system times and corresponding byte positions; and

storing the byte-offset table.

19. Method, according to claim 18, wherein said randomly accessing further comprises:

reading the stored byte-offset table; and

seeking the position of the recorded MPEG transport stream using the byte-offset table that contains the byte position and its corresponding system time.

20. Method, according to claim 17, wherein said randomly accessing further comprises:

extracting a system time and its byte position from one of said recorded or received transport streams; and

seeking a byte position according to a system time from the MPEG transport stream.

21. Method, according to claim 20, wherein seeking a byte position according to a system time further comprises:

predicting the required position by multiplying a relative time by a constant bit rate and adding the result to the start position of the recorded MPEG transport stream, if a broadcast signal is broadcast with a constant bit rate; and

refining said byte position by adding or subtracting according to a difference in time between the required system time and the system time contained in a transport packet located at that position.