System for creating a space-efficient document categorizer for training and testing of automatic categorization engines

• US 6,985,950 B1
• Filed: 03/06/2001
• Issued: 01/10/2006
• Est. Priority Date: 03/06/2001
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Hanson, Andrew, Lamkins, David B.
• Primary Examiner(s): Maung, Zarni
• Assistant Examiner(s): DINH, KHANH Q
• Application Number:
  US09/800,766
• Time in Patent Office: 1,771 Days
• Field of Search: 709/219, 709/224, 709/226, 709/227, 709/228, 709/245, 707/10, 707/3, 707/105, 707/4, 707/5, 713/171
• US Class Current: 709/226
• CPC Class Codes: G06F 16/287 Visualization; Browsing
• CPC Class Codes: G06F 16/957 Browsing optimisation, e.g. caching or content distillation
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation

Abstract

A method of receiving a network resource address and creating a copy of the network resource and storing the created copy in a database is provided. This creation and storage of a copy of the network resource is performed when there does not exist, in the database, information derived from the resource pointed to by the network address.

Claims

1. A method comprising:

receiving a locator of a network resource;

determining if a database already contains stored information derived from the network resource at a previous point in time, effectively freezing the network resource to the previous point in time; and

upon determination that the database does not contain stored information derived from the network resource at the previous point in time, storing information derived from the network resource pointed to by the locator of the network resource, the process of storing comprising;

creating a copy of at least a portion of the network resource pointed to by the locator, and

writing the copy to the database.

2. The method of claim 1 wherein the storing of information further comprises compressing the copy prior to writing the copy.

3. The method of claim 1 wherein the storing the information further comprises generating meta information from the copy.

4. The method of claim 3 wherein the generating meta information involves extracting information from the network resource.

5. The method of claim 3 wherein the generating meta information involves deriving information from the network resource.

6. The method of claim 3 wherein the meta information comprises one or more of a file name, a uniform resource locator, a file format, and a language.

7. The method of claim 3 wherein the storing the information further comprises writing the generated meta information to the database.

8. The method of claim 3 further comprising:

receiving instructions to modify the generated meta information; and

modifying the generated meta information in accordance with the received instructions to generate modified meta information.

9. The method of claim 8 wherein the storing the information further comprises writing the modified meta information to the database.

10. The method of claim 1 wherein the storing the information further comprises modifying references to objects within the information to reflect the new location of referenced objects as being stored in the database.

11. The method of claim 1 further comprising sending the copied information to a generic user agent.

12. The method of claim 1 wherein the network resource comprises one or more world wide web pages.

13. The method of claim 12 wherein the one or more world wide web pages comprises a main frame and one or more sub-frames.

14. The method of claim 13 wherein the main frame and one or more sub-frames are stored as a single file.

15. An apparatus comprising:

a storage medium having stored therein a plurality of programming instructions designed to;

receive a locator of a network resource,

determine if a database already contains stored information derived from the network resource at a previous point in time, effectively freezing the network resource to the previous point in time, and

upon determination that the database does not contain stored information derived from the network resource at the previous point in time, store information derived from the network resource pointed to by the locator of the network resource, the process of storing comprising;

creating a copy of at least a portion of the network resource pointed to by the locator, and

writing the copy to the database; and

a processor coupled to the storage medium to execute the programming instructions.

16. The apparatus of claim 15 wherein the database resides on a separate machine from the processor.

17. The apparatus of claim 15 wherein the storing of information further comprises compressing the copy prior to writing the copy.

18. The apparatus of claim 15 wherein the storing the information further comprises generating meta information from the copy.

19. The apparatus of claim 18 wherein the generating meta information involves extracting information from the network resource.

20. The apparatus of claim 18 wherein the generating meta information involves deriving information from the network resource.

21. The apparatus of claim 18 wherein the meta information comprises one or more of a file name, a uniform resource locator, a file format, and a language.

22. The apparatus of claim 18 wherein storing the information further comprises writing the generated meta information to the database.

23. The apparatus of claim 18 wherein the programming instructions are further designed to:

receive instructions to modify the generated meta information; and

modify the generated meta information in accordance with the received instructions to generate modified meta information.

24. The apparatus of claim 23 wherein storing the information further comprises writing the modified meta information to the database.

25. The apparatus of claim 15 wherein the storing the information further comprises modifying references to objects within the information to reflect the new location of referenced objects as being stored in the database.

26. The apparatus of claim 15 further comprising sending the copied information to a generic user agent.

27. The apparatus of claim 15 wherein the network resource comprises one or more world wide web pages.

28. The apparatus of claim 27 wherein the one or more world wide web pages comprises a main frame and one or more sub-frames.

29. The apparatus of claim 28 wherein the main frame and one or more sub-frames are stored as a single file.