Hypervideo: information retrieval using time-related multimedia:

• US 6,996,775 B1
• Filed: 10/29/1999
• Issued: 02/07/2006
• Est. Priority Date: 10/29/1999
• Status: Expired due to Fees

• Current Assignee: Verizon Laboratories Incorporated (Verizon Communications Inc.)
• Original Assignee: Verizon Laboratories Incorporated (Verizon Communications Inc.)
• Inventors: Dey, Jayanta Kumar, Ponte, Jay Michael
• Primary Examiner(s): Shah, Sanjiv
• Application Number:
  US09/430,437
• Time in Patent Office: 2,293 Days
• Field of Search: 707/3, 707/6, 715/500.1, 715/521, 715/520, 709/219, 345/716, 345/717, 345/721
• US Class Current: 715/256
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9538 Presentation of query results

Abstract

Disclosed is a method and device for selecting documents, such as Web pages or sites, for presentation to a user, in response to a user expression of interest, during the course of presentation to the user of a document, such as a video or audio selection, whose content varies with time. The method takes advantage of information retrieval techniques to select documents related to the portion of the temporal document in which the user has expressed interest. The method generates the search query to use to select documents by reference to text associated with the portion of the temporal document in which the user has expressed interest, as by using the closed caption test associated with the video, or by using speech recognition techniques. The method further uses a weighting function to weigh the terms used in the search query, depending on their temporal relationship to the user expression of interest.

Claims

1. A computer-implemented method for finding documents which relate to a portion of a temporal document, the temporal document including at least one of audio material or video material, comprising:

(a) in response to receiving a signal of interest at a particular time during the temporal document, identifying a portion of the temporal document for which related documents are to be found;

(b) selecting text associated with the portion of the temporal document identified, wherein different portions of said text are associated with different times of said temporal document;

(c) weighting each term in the selected text by a function W(t) according to the time t at which the term occurs relative to the time at which the signal of interest occurs;

(d) finding the related documents by use of information retrieval techniques as applied to the weighted terms.

2. The method of claim 1, wherein the temporal document includes video material and said video material is stored on a video server.

3. The method of claim 1, wherein the selected text is determined by application of speech recognition techniques to the audio component of the portion of the temporal document identified.

4. The method of claim 1, wherein the selected text is the closed-captioned text associated with the portion of the temporal document identified.

5. The method of claim 1, wherein the temporal document includes text.

6. The method of claim 5, wherein the document text appearing to the user varies with time and the selected text is that portion of the temporal document identified.

7. The method of claim 6, wherein the document text includes news bulletins, weather, sports scores or stock transaction or pricing information.

8. The method of claim 1, wherein W(t) is equal for all times between t₁ before the signal of interest is given and t₂ before the signal of interest is given, and is zero for all other times.

9. The method of claim 8, wherein t₁ is 2 seconds and t₂ is 30 seconds.

10. The method of claim 1, wherein W(t) is equal for all times between t₁ before the signal of interest is given and t₂ before the signal of interest is given, and decreases from t₁ until the time of the signal of interest, and increases from a time t₃ before the signal of interest is given to the time t₂, and is zero for all other times.

11. The method of claim 10, wherein t₁ is 2 seconds, t₂ is 15 seconds, and t₃ is 30 seconds.

12. The method of claim 10, wherein W(t) decreases linearly from t₁ until the time of the signal of interest, and increases linearly from t₃ before the signal of interest is given to t₂.

13. The method of claim 12, wherein t₁ is 2 seconds, t₂ is 15 seconds, and t₃ is 30 seconds.

14. The method of claim 1, wherein the related documents are accessed through the Internet.

15. The method of claim 14, further including selecting the related documents from among a collection of documents which may be accessed through the Internet, by utilizing databases comprising information about the collection.

16. The method of claim 15, wherein the related documents are selected from the collection according to the scores achieved when evaluating documents in the collection according to a formula giving scores to documents depending upon the occurrence in the documents of terms which occur in text associated with the portion of the temporal document identified, where each term is weighted by a function W(t) according to the time t at which the term occurs relative to the time at which the signal of interest occurs.

17. The method of claim 16, wherein a predetermined number of documents are selected.

18. The method of claim 16, wherein terms which occur in portions of the temporal document other than the portion identified are utilized in calculating the scores achieved when evaluating documents in the collection.

19. The method of claim 16, wherein evaluating documents in the collection includes accessing compressed document surrogates.

20. The method of claim 16, wherein related documents are selected from the collection by a server which is distinct from the server which receives the signal of interest.

21. A computer-implemented device for finding documents which relate to a portion of a temporal document, the temporal document including at least one of audio material or video material, comprising:

(a) means for identifying a portion of the temporal document for which related documents are to be found, in response to receiving a signal of interest at a particular time during the temporal document;

(b) means for selecting text associated with the portion of the temporal document identified wherein different portions in said text are associated with different times of said temporal document;

(c) means for weighting each term in the selected text by a function W(t) according to the time t at which the term occurs relative to the time at which the signal of interest occurs;

(d) means for finding the related documents by use of information retrieval techniques as applied to the weighted terms.

22. The device of claim 21, wherein the temporal document includes video material and said video material is stored on a video server.

23. The device of claim 21, wherein the selected text is determined by application of speech recognition techniques to the audio component of the portion of the temporal document identified.

24. The device of claim 21, wherein the selected text is the closed-captioned text associated with the portion of the temporal document identified.

25. The device of claim 21, wherein the temporal document includes text.

26. The device of claim 25, wherein the document text appearing to the user varies with time and the selected text is that portion of the temporal document identified.

27. The device of claim 26, wherein the document text includes news bulletins, weather, sports scores or stock transaction or pricing information.

28. The device of claim 21, wherein W(t) is equal for all times between t₁ before the signal of interest is given and t₂ before the signal of interest is given, and is zero for all other times.

29. The device of claim 28, wherein t₁ is 2 seconds and t₂ is 30 seconds.

30. The device of claim 21, wherein W(t) is equal for all times between t₁ before the signal of interest is given and t₂ before the signal of interest is given, and decreases from t₁ until the time of the signal of interest, and increases from a time t₃ before the signal of interest is given to the time t₂, and is zero for all other times.

31. The device of claim 30, wherein t₁ is 2 seconds, t₂ is 15 seconds, and t₃ is 30 seconds.

32. The device of claim 30, wherein W(t) decreases linearly from t₁ until the time of the signal of interest, and increases linearly from t₃ before the signal of interest is given to t₂.

33. The device of claim 32, wherein t₁ is 2 seconds, t₂ is 15 seconds, and t₃ is 30 seconds.

34. The device of claim 21, wherein the related documents are accessed through the Internet.

35. The device of claim 34, further including means for selecting the related documents from among a collection of documents which may be accessed through the Internet, by utilizing databases comprising information about the collection.

36. The device of claim 35, wherein the related documents are selected from the collection according to the scores achieved when evaluating documents in the collection according to a formula giving scores to documents depending upon the occurrence in the documents of terms which occur in text associated with the portion of the temporal document identified, where each term is weighted by a function W(t) according to the time t at which the term occurs relative to the time at which the signal of interest occurs.

37. The device of claim 36, wherein a predetermined number of documents, are selected.

38. The device of claim 36, wherein terms which occur in portions of the temporal document other than the portion identified are utilized in calculating the scores achieved when evaluating documents in the collection.

39. The device of claim 36, wherein evaluating documents in the collection includes accessing compressed document surrogates.

40. The device of claim 36, wherein related documents are selected from the collection by a server which is distinct from the server which receives the signal of interest.