Time ordered indexing of an information stream

• US 20030086409A1
• Filed: 01/29/2002
• Published: 05/08/2003
• Est. Priority Date: 11/03/2001
• Status: Active Grant

• Current Assignee: Micro Focus IP Development Ltd. (Open Text Corporation)
• Original Assignee: Automony Systems, Limited
• Inventors: Karas, D. Matthew, Muldrew, William J.
• Granted Patent: US 7,206,303 B2
• Time in Patent Office: Days
• US Class Current: 370/350
• CPC Class Codes: G06F 16/748 Hypervideo linking data to content, e.g. by linking an URL to a video object in the context of video distribution systems H04N21/858
• CPC Class Codes: G10L 15/05 Word boundary detection
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04H 60/37 for identifying segments of broadcast information, e.g. scenes or extracting programme ID
• CPC Class Codes: H04H 60/56 Arrangements characterised by components specially adapted for monitoring, identification or recognition covered by groups H04H60/29-H04H60/54
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/2353 specifically adapted to content descriptors, e.g. coding, compressing or processing of metadata
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 21/8586 by using a URL processing chained hypermedia data for information retrieval G06F16/94; information retrieval from the Internet by using URLs G06F16/955; URL in broadcast information H04H20/93; Web-based protocols H04L67/02

Abstract

Methods and apparatuses in which two or more types of attributes from an information stream are identified. Each of the identified attributes from the information stream is encoded. A time ordered indication is assigned with each of the identified attributes. Each of the identified attributes shares a common time reference measurement. A time ordered index of the identified attributes is generated.

Claims

1. A method, comprising:

identifying two or more types of attributes of an information stream;

encoding each identified attribute from the information stream;

assigning a time ordered indication with each of the identified attributes, each of the identified attributes sharing a common time reference; and

generating a time ordered index of the identified attributes.

2. The method of claim 1, wherein the information stream includes audio-visual data.

3. The method of claim 1, wherein the information stream includes speech data.

4. The method of claim 1, wherein at least one of the identified attributes comprises a scene change.

5. The method of claim 1, wherein at least one of the identified attributes comprises a fade.

6. The method of claim 1, wherein at least one of the identified attributes comprises a pan.

7. The method of claim 1, wherein at least one of the identified attributes comprises a human language type.

8. The method of claim 1, wherein at least one of the identified attributes comprises a human accent.

9. The method of claim 1, wherein at least one of the identified attributes comprises image characteristics.

10. The method of claim 1, wherein at least one of the identified attributes comprises a spoken word.

11. The method of claim 1, wherein the identified attributes are encoded via extensible markup language.

12. The method of claim 1, wherein the time ordered indication comprises a start time and a time frame in which the identified attribute was conveyed.

13. The method of claim 1, wherein the common time reference comprises a frame count.

14. The method of claim 1, wherein the common time reference comprises a time indication.

15. The method of claim 1, further comprising generating a query on the one or more of identified attributes in the time ordered indexed.

16. The method of claim 1, further comprising correlating a first identified attribute of the information stream with a second identified attribute having a similar time code.

17. The method of claim 16, wherein the similar time code comprises the first identified attribute possessing a start time approximately the same as the second identified attribute or an overlapping of time frames associated with the first identified and the second identified attribute.

18. The method of claim 1, further comprising:

individually synchronizing each word in a transcript from the information stream to be frame accurate to corresponding video data based upon both sharing the common time reference.

19. A machine-readable medium that stores instructions, which when executed by a machine, cause the machine to perform operations comprising:

identifying two or more types of attributes of an information stream;

encoding each of the identified attributes from the information stream;

assigning a time ordered indication with each of the identified attributes, each of the identified attributes share a common time reference; and

generating a time ordered index of the identified attributes.

20. The article of manufacture of claim 19, which causes the machine to perform further operations comprising:

generating a query on the one or more of the time ordered indexed identified attributes.

21. The article of manufacture of claim 19, which causes the machine to perform further operations comprising:

correlating a first identified attribute of the information stream with a second identified attribute having a similar time code.

22. The method of claim 19, further comprising:

individually synchronizing each word in a transcript from the information stream to be frame accurate to corresponding video data based upon both sharing the common time reference.

23. An apparatus, comprising:

means for identifying two or more types of attributes of an information stream;

means for encoding each identified attribute from the information stream;

means for assigning a time ordered indication with each of the identified attributes, each of the identified attributes share a common time reference; and

means for generating a time ordered index of the identified attributes.

24. The apparatus of claim 23, further comprising:

means for generating a query on the one or more of the time ordered indexed identified attributes.

25. A method, comprising:

converting spoken words in an information stream to written text, the information stream containing audio-visual information; and

generating a separate encoded file for every word, each encoded file contains a time ordered indication reference to a relevant video frame.

26. The method of claim 25, further comprising:

detecting shot change information; and

creating thumbnail images for each shot change; and

generating an encoded file for every shot change, each encoded file containing a time ordered indication referencing to one or more corresponding spoken words.

27. The method of claim 25, further comprising:

generating a link to the corresponding relevant video frame based upon a user selecting one or more of the spoken words.

28. The method of claim 25, further comprising:

generating a link to relevant material based upon the spoken words and synchronizing a display of the link in less than five seconds from analyzing the information stream.

29. A machine-readable medium that stores instructions, which when executed by a machine, cause the machine to perform operations comprising:

converting spoken words in an information stream to written text, the information stream containing audio-visual information; and

generating a separate encoded file for every word, each encoded file contains a time ordered indication reference to a relevant video frame.

30. The article of manufacture of claim 29, which causes the machine to perform the further operations comprising:

detecting shot change information; and

creating thumbnail images for each shot change;

generating an encoded file for every shot change, each encoded file containing a time ordered indication referencing to one or more corresponding spoken words.

31. The article of manufacture of claim 29, which causes the machine to perform the further operations comprising:

generating a link to relevant material based upon the spoken words and synchronizing a display of the link in less than five seconds from analyzing the information stream.

32. An apparatus, comprising:

means for converting spoken words in an information stream to written text, the information stream containing audio-visual information; and

means for generating a separate encoded file for every word, each encoded file contains a time ordered indication reference to a relevant video frame.

33. The apparatus of claim 32, further comprising:

means for detecting shot change information; and

means for creating thumbnail images for each shot change;

means for generating an encoded file for every shot change, each encoded file containing a time ordered indication referencing to one or more corresponding spoken words.

34. An apparatus comprising:

a software engine operable to apply one or more attribute filters to detect attributes from an information stream, identify the attributes, and assign a time ordered indication with each of the identified attributes, the software engine having an index control module to facilitate time order indexing of the identified attributes in order to be frame accurate; and

a computer readable medium to store the software engine.

35. The apparatus of claim 34, wherein the time ordered indication comprises a beginning frame count referenced to universal frame count referenced for all of the attributes from the information stream and an amount of frames in which the attribute was conveyed.

36. The apparatus of claim 34, wherein the time ordered indication comprises a start time and a time frame in which the identified attribute was conveyed.

37. The apparatus of claim 34, further comprising:

a manipulation-module to perform operations on a first set of attributes in order to manipulate a second set of attributes.

38. The apparatus of claim 37, wherein the first set of attributes compromises a section of transcripted text and the second set of attributes comprises video images having approximately the same time ordered indications as the transcripted text.

39. The apparatus of claim 34, further comprising:

a triggering and synchronization module to dynamically trigger a link and synchronize the appearance of the link based upon a transcripted text from the information stream.

40. A machine-readable medium that stores instructions, which when executed by a machine, cause the machine to perform operations comprising:

identifying one or more attributes of an information stream from a live broadcast;

encoding each of the identified attributes from the information stream;

assigning a time ordered indication with each of the identified attributes, each of the identified attributes share a common time reference; and

generating a synchronized link to relevant material based on the content of the live broadcast, the synchronized link to be displayed with the live broadcast.

41. The article of manufacture of claim 40, which causes the machine to perform further operations comprising:

synchronizing the synchronized link to appear at approximately an utterance of the most relevant word related to the content of the live broadcast.

42. An apparatus, comprising:

means for identifying one or more attributes of an information stream from a live broadcast;

means for encoding each of the identified attributes from the information stream;

means for assigning a time ordered indication with each identified attribute, each identified attribute shares a common time reference measurement; and

means for generating a synchronized link to relevant material based on the content of the live broadcast, the synchronized link to be displayed with the live broadcast.

43. The apparatus of claim 42, further comprising:

means for synchronizing the link to appear at approximately an utterance of the most relevant word related to the content of the live broadcast.