Hyper video: information retrieval using multimedia

• US 7,039,633 B1
• Filed: 10/03/2002
• Issued: 05/02/2006
• Est. Priority Date: 10/29/1999
• Status: Expired due to Term

• Current Assignee: Level 3 Communications LLC (Lumen Technologies, Inc.)
• Original Assignee: Verizon Laboratories Incorporated (Verizon Communications Inc.)
• Inventors: Dey, Jayanta Kumar, Xu, Haiping
• Primary Examiner(s): Corrielus, Jean M.
• Assistant Examiner(s): To, Baoquoc N
• Application Number:
  US10/264,112
• Time in Patent Office: 1,307 Days
• Field of Search: 707/3, 707/4, 707/5, 707/10, 715/501, 715/501.01
• US Class Current: 1/1
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9532 Query formulation
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access
• CPC Class Codes: Y10S 707/99936 Pattern matching access

Abstract

Disclosed is a method and device for selecting documents, such as Web pages or sites, for presentation to a user, in response to a user expression of interest, during the course of presentation to the user of a document, such as a video or audio selection, whose content varies with time. The method takes advantage of information retrieval techniques to select documents related to the portion of the temporal document in which the user has expressed interest. The method generates the search query to use to select documents by reference to text associated with the portion of the temporal document in which the user has expressed interest, as by using the closed caption test associated with the video, or by using speech recognition techniques. The method further uses a weighting function to weigh the terms used in the search query, depending on their temporal relationship to the user expression of interest. To conserve resources, the method does not transmit the closed caption or other synchronized multimedia information to the user, and obtains the necessary information about the temporal occurrence of terms in the temporal document from a database.

Claims

1. A method for finding documents which relate to a portion of a temporal document, comprising:

(a) in response to a signal of interest at a particular time during the temporal document, identifying a portion of the temporal document occurring during a time interval before said signal of interest;

(b) selecting text associated with the portion of the temporal document identified, by reference to a database wherein is stored each term in text associated with the temporal document, together with a time in the temporal document at which the term occurs;

(c) for a first term in the selected text, determining a first weight based on;

a function W(t) associated with said time interval, and the time in the temporal document at which said first term occurs, wherein W(t) is equal for all times between t₁ before the signal of interest is given and t₂ before the signal is given, and decreases from t₁ until the time of the signal, and increase from a time t₃ before the signal is given to the time t₂, and is zero for all other times;

(d) identifying the related documents by use of information retrieval techniques as applied to said first term and said first weight.

2. The method of claim 1, wherein the temporal document is video or audio material.

3. The method of claim 2, wherein the video material is stored on a video server.

4. The method of claim 3, wherein closed-captioned text associated with the temporal document is not made available when the temporal document is accessed from the video server.

5. The method of claim 3, wherein the text stored in the database is determined by application of speech recognition techniques to the audio component of the portion of the temporal document identified.

6. The method of claim 3, wherein the text stored in the database is the closed-captioned text associated with the portion of the temporal document identified.

7. The method of claim 1, wherein the temporal document includes text.

8. The method of claim 7, wherein the document text appearing to the user varies with time and the text selected is that portion of the temporal document identified.

9. The method of claim 8, wherein the text includes news bulletins, weather, sports scores or stock transaction or pricing information.

10. The method of claim 1, wherein W(t) is equal for all times between t₁ before the signal of interest is given and t₂ before the signal is given, and is zero for all other times.

11. The method of claim 10, wherein t₁ is 2 seconds and t₂ is 30 seconds.

12. The method of claim 1, wherein t₁ is 2 seconds, t₂ is 15 seconds, and t₃ is 30 seconds.

13. The method of claim 12, wherein W(t) decreases linearly from t₁ until the time of the signal, and increases linearly from t₃ before the signal is given to t₂.

14. The method of claim 13, wherein t₁ is 2 seconds, t₂ is 15 seconds, and t₃ is 30 seconds.

26. The device of claim 14, wherein t₁ is 2 seconds, t₂ is 15 seconds, and t₃ is 30 seconds.

27. The device of claim 26, wherein W(t) decreases linearly from t₁ until the time of the signal, and increases linearly from t₃ before the signal is given to t₂.

28. The device of claim 27, wherein t₁ is 2 seconds, t₂ is 15 seconds, and t₃ is 30 seconds.

15. A device for finding documents which relate to a portion of a temporal document, comprising:

(a) means for identifying a portion of the temporal document, in response to a signal of interest at a particular time during the temporal document, said portion of the temporal document occurring during a time interval before said signal of interest;

(b) mean for selecting text associated with the portion of the temporal document identified, by reference to a database wherein is stored each term in text associated with the temporal document, together with a time in the temporal document at which the term occur;

(c) for the first term in the selected text, means for determining a first weight based on;

a function W(t) associated with said time interval, and the time in the temporal document at which said first term occurs, wherein W(t) is equal for all times between t₁ before the signal of interest is given and t₂ before the signal is given, and decreases from t₁ until the time of the signal, and increases from a time t₃ before the signal is given to the t₂ and is zero for all other times;

(d) means for identifying the related document by use of information retrieval techniques as applied to said first term and said first weight.

16. The device of claim 15, wherein the temporal document is video or audio material.

17. The device of claim 16, wherein the video material is stored on a video server.

18. The device of claim 17, wherein closed-captioned text associated with the temporal document is not made available when the temporal document is accessed from the video server.

19. The device of claim 17, wherein the text stored in the database is determined by application of speech recognition techniques to the audio component of the portion of the temporal document identified.

20. The device of claim 17, wherein the text stored in the database is the closed-captioned text associated with the portion of the temporal document identified.

21. The device of claim 15, wherein the temporal document includes text.

22. The device of claim 21, wherein the document text appearing to the user varies with time and the text selected is that portion of the temporal document identified.

23. The device of claim 22, wherein the text includes news bulletins, weather, sports scores or stock transaction or pricing information.

24. The device of claim 15, wherein W(t) is equal for all times between t₁ before the signal of interest is given and t₂ before the signal is given, and is zero for all other times.

25. The device of claim 24, wherein t₁ is 2 seconds and t₂ is 30 seconds.