Method and system for identifying and displaying images in response to search queries

• US 20080082497A1
• Filed: 09/29/2006
• Published: 04/03/2008
• Est. Priority Date: 09/29/2006
• Status: Active Grant

• Current Assignee: A9.com Incorporated (Amazon.com, Inc.)
• Original Assignee: A9.com Incorporated (Amazon.com, Inc.)
• Inventors: Siegel, Hilliard B., Leblang, Jonathan A.
• Granted Patent: US 8,631,012 B2
• Time in Patent Office: Days
• US Class Current: 707/3
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9538 Presentation of query results

Abstract

A method and system are disclosed that associate search keywords with relevant images from within sources, such as books, magazines, journals, etc., thereby making the images available for search using the associated keyword. Such associations enable a search engine to identify, in response to a search query, a relevant image within the source that would have not been provided otherwise. For example, an image of the Titanic that appears in a historical or cultural book may be associated with the search keyword “ship.” Accordingly, if a user submits the search query “ship and Infamous Ocean Voyages” (i.e., the keyword “ship” and the title of a book), an image of the Titanic from the book entitled “Infamous Ocean Voyages” may be returned among the search results.

Claims

1. A system for associating keywords with images found on the pages of a source, the system comprising:

a first data store that stores text and related text metadata obtained from the source pages;

a second data store that stores images and related image metadata obtained from the source pages; and

a relevance engine that generates a relevance index based at least in part on the text and related text metadata stored in the first data store and the images and related image metadata stored in the second data store, wherein the relevance index records an association between a keyword from the text of the source and an image from the source, and wherein the association may be used to access the image associated with the keyword in response to a search query including the keyword.

2. The system of claim 1, wherein the first data store and second data store are combined.

3. The system of claim 1, wherein the relevance engine generates the relevance index using at least one relevance technique based on a physical arrangement of the source pages.

4. The system of claim 1, wherein the relevance engine generates the relevance index using at least one relevance technique based on typography of the text in the source pages.

5. The system of claim 1, wherein the relevance engine generates the relevance index using at least one relevance technique based on semantic analysis of the source pages.

6. The system of claim 1, wherein the relevance engine generates the relevance index using at least one relevance technique based on navigational elements of the source pages.

7. The system of claim 1, wherein the relevance engine generates the relevance index using at least one relevance technique based on statistical analysis of the source pages.

8. The system of claim 1, wherein the relevance engine generates the relevance index using at least one relevance technique based on manual assessment of the source pages.

9. The system of claim 1, wherein the association between the keyword and the image is recorded in the relevance index as a pointer to the image stored in the image data store.

10. A computer-implemented method for associating keywords with images within a source, the computer-implemented method comprising:

obtaining data from one or more pages of the source, wherein the data includes text data, image data and related metadata;

for a given term in the text, determining if the term is associated with an image in the one or more pages based on the obtained data; and

if the term is associated with an image from the one or more pages, storing the term as a keyword associated with the image, wherein the association may be used to access the image associated with the keyword in response to a search query including the keyword.

11. The computer-implemented method of claim 10, wherein determining if the term is associated with an image comprises determining if the term falls within a predetermined distance of the image.

12. The computer-implemented method of claim 11, wherein the predetermined distance is a physical distance between the term and the image.

13. The computer-implemented method of claim 11, wherein the predetermined distance is a predetermined number of terms between the term and the image.

14. The computer-implemented method of claim 10, wherein determining if the term is associated with an image comprises:

determining if the image is glossy; and

identifying the term from caption text for the image.

15. The computer-implemented method of claim 10, wherein determining if the term is associated with an image comprises determining if the term is found within the image.

16. The computer-implemented method of claim 10, wherein determining if the term is associated with an image comprises:

determining if the term is an image reference term; and

determining if the image is in a vicinity of the term.

17. The computer-implemented method of claim 10, wherein determining if the term is associated with an image comprises:

identifying a term appearing in a different font; and

determining if the image is in a vicinity of the term.

18. The computer-implemented method of claim 10, wherein determining if the term is associated with an image comprises:

identifying a term appearing in parenthesis; and

determining if the image is in a vicinity of the term.

19. The computer-implemented method of claim 10 further comprising:

determining the type of source from which the data is obtained; and

determining if the term is associated with an image in the one or more pages based on the obtained data and the type of source.

20. The computer-implemented method of claim 10, wherein determining if the term is associated with an image comprises:

determining a ratio of a number of terms to a number of images in the source; and

determining if the ratio is below a threshold.

21. The computer-implemented method of claim 10, wherein determining if the term is associated with an image comprises:

determining if the term appears in navigational metadata; and

determining if the image is referenced in the navigational metadata.

22. The computer-implemented method of claim 10, wherein determining if the term is associated with an image includes applying at least one of the following techniques to the text data image data, and related metadata:

a physical arrangement technique, a typography technique, a semantic analysis technique, statistical analysis technique, a navigational analysis technique and a manual technique.

23. A system for associating keywords with images found on the pages of a source, the system comprising:

a data store that stores text, images and related metadata obtained from the source pages; and

a relevance engine that generates a relevance index based at least in part on the text, images and metadata stored in the data store, wherein the relevance index records an association between a keyword from the text and an image from the source, and wherein the association may be used to access the image associated with the keyword in response to a search query including the keyword.

24. The system of claim 23, wherein the data store comprises a first data store that stores text and text related metadata and a second data store that stores images and image related metadata.

25. The system of claim 23, wherein the relevance engine generates the relevance index based upon at least one relevance technique.

26. The system of claim 25, wherein the at least one relevance technique includes one or more of a physical arrangement technique, a typography technique, a semantic analysis technique, statistical analysis technique, a navigational analysis technique and a manual technique.

27. The system of claim 25, wherein the at least one relevance technique associates the keyword from the text and the image of the source by:

determining if a term from the text falls within a predetermined distance of the image; and

recording the term as the keyword associated with the image in the relevance index.

28. The system of claim 27, wherein the predetermined distance is a physical distance between the term and the image.

29. The system of claim 27, wherein the predetermined distance is a predetermined number of terms between the term and the image.

30. The system of claim 25, wherein the at least one relevance technique associates the keyword from the text and the image of the source by:

determining if the image is glossy;

identifying a term from caption text for the image; and

recording the term as the keyword associated with the image in the relevance index.

31. The system of claim 25, wherein the at least one relevance technique associates the keyword from the text and the image of the source by:

determining if a term is found within the image; and

recording the term as the keyword associated with the image in the relevance index.

32. The system of claim 25, wherein the at least one relevance technique associates the keyword from the text and the image of the source by:

determining if a term is an image reference term;

determining if the image is in a vicinity of the term; and

recording the term as the keyword associated with the image in the relevance index.

33. The system of claim 25, wherein the at least one relevance technique associates the keyword from the text and the image of the source by:

identifying a term appearing in a different font;

determining if the image is in a vicinity of the term; and

recording the term as the keyword associated with the image in the relevance index.

34. The system of claim 25, wherein the at least one relevance technique associates the keyword from the text and the image of the source by:

identifying a term appearing in parenthesis;

determining if the image is in a vicinity of the term; and

recording the term as the keyword associated with the image in the relevance index.

35. The system of claim 25, wherein at least one relevance technique associates the keyword from the text and the image of the source by:

identifying a term appearing in a navigational element of the source;

identifying the image found at a page number corresponding to the term; and

recording the term as the keyword associated with the image in the relevance index.

36. The system of claim 25, wherein the at least one relevance technique associates the keyword from the text and the image of the source by:

manually identifying a term relevant to the image; and

recording the keyword associated with the image in the relevance index.

37. The system of claim 23, wherein the association between the keyword and the image is recorded in the relevance index as a pointer to the image stored in the image data store.

38. A computer-readable medium having instructions encoded thereon that, when executed by a computing device, cause the computing device to:

obtain data from one or more pages of the source, wherein the data includes text data, image data and related metadata;

for a given term in the text, determine if the term is associated with an image in the one or more pages based on the obtained data; and

if the term is associated with an image from the one or more pages, store the term as a keyword associated with the image, wherein the association may be used to access the image associated with the keyword in response to a search query including the keyword.

39. The computer-readable medium of claim 38, wherein determining if the term is associated with an image comprises determining if the term falls within a predetermined distance of the image.

40. The computer-readable medium of claim 38, wherein the predetermined distance is a physical distance between the term and the image.

41. The computer-readable medium of claim 39, wherein the predetermined distance is a predetermined number of terms between the term and the image.

42. The computer-readable medium of claim 38, wherein determining if the term is associated with an image comprises:

determining if the image is glossy; and

identifying the term from caption text for the image.

43. The computer-readable medium of claim 38, wherein determining if the term is associated with an image comprises determining if the term is found within the image.

44. The computer-readable medium of claim 38, wherein determining if the term is associated with an image comprises:

determining if the term is an image reference term; and

determining if the image is in a vicinity of the term.

45. The computer-readable medium of claim 38, wherein determining if the term is associated with an image comprises:

identifying a term appearing in a different font; and

determining if the image is in a vicinity of the term.

46. The computer-readable medium of claim 38, wherein determining if the term is associated with an image comprises:

identifying a term appearing in parenthesis; and

determining if the image is in a vicinity of the term.

47. The computer-readable medium of claim 38, wherein determining if the term is associated with the images comprises:

determining a ratio of a number of terms to a number of images in the source; and

determining if the ratio is below a threshold.

48. The computer-readable medium of claim 38, wherein determining if the term is associated with the image comprises:

manually identifying a term relevant to the image; and

recording the term as the keyword associated with the image in the relevance index.

49. The computer-readable medium of claim 38, wherein determining if the terms is associated with the image comprises:

determining if the term appears in navigational metadata; and

determining if the image is referenced in the navigational metadata.

50. The computer-readable medium of claim 38, wherein the instructions encoded thereon, when executed by the computing device, further cause the computing device to:

determine the type of source from with the data is obtained; and

determine if the term is associated with an image in the one or more pages based on the obtained data and the type of source.

51. The computer-readable medium of claim 38, wherein the instructions encoded thereon, when executed by the computing device, further cause the computing device to apply at least one of the following to the text data, image date and related metadata:

a physical arrangement technique, a typography technique, a semantic analysis technique, statistical analysis technique, a navigational analysis technique and a manual technique.

52. A computer-implemented method for searching content, the computer-implemented method comprising:

receiving a search query including a search term;

searching a relevance index for a keyword that corresponds to the search term, wherein the relevance index stores an association between the keyword and an image from within a source of content; and

if a keyword corresponding to the search term is found in the relevance index, returning search results corresponding to the search query that include the image associated with the keyword.