Information retrieval using enhanced document vectors

• US 20030018617A1
• Filed: 07/01/2002
• Published: 01/23/2003
• Est. Priority Date: 07/18/2001
• Status: Abandoned Application

• Current Assignee: SAP AG (SAP SE)
• Original Assignee: SAP AG (SAP SE)
• Inventors: Schwedes, Holger
• Application Number:
  US10/188,304
• Publication Number:
  US 20030018617A1
• Time in Patent Office: Days
• US Class Current: 707/2
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

An information retrieval system includes an enhanced document vector module to generate enhanced document vectors representative of documents in a collection. The enhanced document vectors include text- and non-text components. The non-text components may include the location, in-links, and/or out-links in hypertext documents and attributes of the documents, e.g., size, create-date, and response-time. A processor uses the enhanced document vectors to perform an information retrieval operation, such as a clustering or classification operation.

Claims

1. A method comprising:

generating a plurality of document vectors for a corresponding plurality of documents, said document vectors including text components and non-text components; and

performing an information retrieval operation using the generated document vectors.

2. The method of claim 1, wherein performing the information retrieval operation comprises determining a similarity between two of the document vectors.

3. The method of claim 2, wherein determining a similarity comprises determining at least one of a distance and an angle between the two document vectors.

4. The method of claim 1, wherein performing the information retrieval operation comprises performing a clustering operation.

5. The method of claim 1, wherein performing the information retrieval operation comprises performing a classification operation.

6. The method of claim 1, wherein performing the information retrieval operation comprises performing a feature extraction operation.

7. The method of claim 1, further comprising:

identifying text components and non-text components in the plurality of documents; and

generating an enhanced document vector space including a plurality of dimensions corresponding to the text components and the non-text components.

8. The method of claim 7, wherein identifying non-text components of the plurality of documents comprises identifying at least one of a location, a link, a size, a create-date, and a response-time of one or more of the plurality of documents.

9. The method of claim 1, further comprising:

weighting one or more of the text and non-text components.

10. The method of claim 9, wherein weighting comprises performing a TFDIF weighting operation on the one or more of the text and non-text components.

11. Apparatus comprising:

a processor operative to generate a plurality of enhanced document vectors representative of a plurality of documents, at least one of the enhanced document vectors in said plurality including text components and non-text components.

12. The apparatus of claim 11, wherein the enhanced document vectors are representative of hypertext documents.

13. The apparatus of claim 12, wherein the non-text components include a location of the hypertext document.

14. The apparatus of claim 13, wherein the location comprises a URL (Uniform Resource Locator).

15. The apparatus of claim 12, wherein the non-text components include in-links.

16. The apparatus of claim 12, wherein the non-text components include out-links.

17. The apparatus of claim 11, wherein the non-text components include at least one of a size, a create-date, and a response-time of one or more of the plurality of documents.

18. The apparatus of claim 11, wherein the processor is further operative to perform an information retrieval operation utilizing the enhanced document vectors.

19. The apparatus of claim 18, wherein the information retrieval operation comprises determining at least one of an angle and a distance between two of the enhanced document vectors.

20. The apparatus of claim 18, wherein the information retrieval operation comprises determining a similarity between a plurality of said enhanced document vectors.

21. The apparatus of claim 18, wherein the information retrieval operation comprises a clustering operation.

22. The apparatus of claim 18, wherein the information retrieval operation comprises a classification operation.

23. The apparatus of claim 18, wherein the information retrieval operation comprises a feature extraction operation.

25. The system of claim 24, wherein the source of documents comprises one or more databases.

26. The system of claim 24, wherein the source of documents comprises one or more servers.

27. The system of claim 24, wherein the source of documents comprises a networked computer system.

28. The system of claim 24, wherein the documents comprise hypertext documents.

29. The system of claim 28, wherein the non-text components locations of the hypertext documents.

30. The system of claim 28, wherein the non-text components comprise hyperlinks.

31. The system of claim 24, wherein the non-text components comprise attributes of the documents.

32. The system of claim 24, wherein the information retrieval operation comprises a clustering operation.

33. The apparatus of claim 24, wherein the information retrieval operation comprises a classification operation.

34. The apparatus of claim 24, wherein the information retrieval operation comprises a feature extraction operation.

36. The article of claim 35, wherein the instructions operative to cause the machine to perform the information retrieval operation comprises instructions operative to cause the machine to perform a clustering algorithm.

24. A system comprising:

a source of a first plurality of documents, documents in said first plurality including text components and non-text components;

an input device operative to receive a user query;

a search engine operative to retrieve a second plurality of documents from the first plurality of documents in response to the user query;

an enhanced document vector module operative to generate a plurality of enhanced document vectors representative of documents in the second plurality of documents, said enhanced document vectors including text components and non-text components; and

a processor operative to perform an information retrieval operation using said enhanced document vectors.

35. An article comprising a machine-readable medium including machine-executable instructions operative to cause a machine to:

generate a plurality of enhanced document vectors for a corresponding plurality of documents, said enhanced document vectors including text components and non-text components; and

perform an information retrieval operation using said enhanced document vectors.