Apparatus and method for cataloging data

• US 20020129062A1
• Filed: 03/08/2001
• Published: 09/12/2002
• Est. Priority Date: 03/08/2001
• Status: Abandoned Application

• Current Assignee: Wood River Technologies, Inc.
• Original Assignee: Wood River Technologies, Inc.
• Inventors: Luparello, F. Thomas
• Application Number:
  US09/802,069
• Publication Number:
  US 20020129062A1
• Time in Patent Office: Days
• US Class Current: 715/513
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9538 Presentation of query results

Abstract

A method of cataloging data is provided. The method includes identifying a data source having data representative of a web page, reading, from the data source, source code representative of the text displayed to a viewer of the web page, identifying, based on the source code, whether at least a portion of the data source corresponds to a predetermined search category, and cataloging the data source in accordance with the identifying. An automatic cataloging device is also provided. The automatic cataloging device includes a data storage device and a processor. The processor of the automatic cataloging device obtains an address of a web page having data from the data storage device, reads source code from the web page, identifies data from the source code that corresponds to a predetermined search category, and saves data related to the corresponding data in a predefined category within the data storage device.

Claims

1. A method of cataloging data on a network, comprising:

identifying a data source having data representative of a web page;

reading, from the data source, source code representative of the text displayed to a viewer of the web page;

identifying, based on the source code, whether at least a portion of the data source corresponds to a predetermined search category; and

cataloging the data source in accordance with said identifying.

2. The method of claim 1, wherein cataloging the data source includes cataloging a pointer indicating a location of the data source.

3. The method of claim 2, wherein the pointer is an address.

4. The method of claim 1, wherein cataloging the data source includes cataloging data copied from the source code.

5. The method of claim 4, wherein the data copied from the source code includes all data displayed as text on the web page.

6. The method of claim 4, wherein the data copied from the source code includes the portion of the displayed text that corresponds to a predetermined search category.

7. The method of claim 1, wherein identifying a data source includes retrieving a data source pointer that identifies a location of a data source from a database stored in a data storage device.

8. The method of claim 7, wherein the data source pointer includes an Internet address.

9. The method of claim 7, wherein the data source pointer includes an address of a data source stored in a database in a data storage device.

10. The method of claim 1, wherein the displayed text includes all text displayed by the data source.

11. The method of claim 1, wherein retrieving source code includes copying the displayed text from the data source into a memory device.

12. The method of claim 1, further comprising examining the source code.

13. The method of claim 1, wherein identifying, based on the source code, whether at least a portion of the data source corresponds to a predetermined search category includes cataloging data related to the data source, and further comprising comparing the identified data to data previously cataloged from that data source.

14. The method of claim 13, further comprising cataloging the identified data only if the identified data does not correspond to any previously identified data.

15. The method of claim 1, wherein identifying, based on the source code, whether at least a portion of the data source corresponds to a predetermined search category includes searching the displayed text for a predefined string of characters.

16. The method of claim 1, wherein identifying, based on the source code, whether at least a portion of the data source corresponds to a predetermined search category includes searching the source code for data corresponding to a Boolean query.

17. The method of claim 1, wherein identifying, based on the source code, whether at least a portion of the data source corresponds to a predetermined search category includes searching the displayed text for data corresponding to at least one of a plurality of Boolean queries.

18. The method of claim 1, wherein the search category includes a standard industry code category.

19. The method of claim 1, wherein said cataloging includes:

determining that the data source has previously been retrieved;

comparing data from the data source with data previously contained in the data source; and

establishing that data in the data source has been modified.

20. The method of claim 19, wherein comparing data from the data source with data previously contained in the data source includes:

calculating a number indicative of the number of characters contained in source code previously read from the data source;

calculating a number indicative of the number of characters contained in source code currently read from data source; and

wherein establishing that data in the data source has been modified includes establishing that the number indicative of the number of characters contained in the source code previously contained in the data source is not equal to the number indicative of the number of characters contained in the currently read source code.

21. The method of claim 20, wherein cataloging includes saving a number indicative of the number of characters contained in the source code.

22. The method of claim 20, wherein identifying data from the displayed text that corresponds to a predetermined search category is not performed if the number of characters contained in the source code currently read from data source is the same as the number of characters contained in the source code previously read from the data source.

23. The method of claim 19, wherein comparing data from the data source with data previously contained in the data source includes:

calculating a previous checksum indicative of at least one quality of data contained in source code previously read from data source;

calculating a current checksum indicative of at least one quality of data contained in the source code currently read from data source; and

wherein establishing that data in the data source has been modified includes establishing that the previous checksum is not equal to the current checksum.

24. The method of claim 23, wherein the quality includes a quantity of characters.

25. The method of claim 23, wherein the quality includes an identity of characters.

26. The method of claim 23, wherein the quality includes an arrangement of characters.

27. The method of claim 1, wherein cataloging includes formatting data related to the data source as a record.

28. The method of claim 1, further comprising accessing cataloged data.

29. The method of claim 1, further comprising accessing cataloged data from the Internet.

30. The method of claim 26, wherein access to cataloged data requires entry of a predetermined password.

31. The method of claim 1, further comprising searching the source code for a reference to a second data source.

32. The method of claim 1, further comprising searching the source code for a reference to a second data source only if at least a portion of the data source corresponds to the predetermined search category.

33. The method of claim 1, wherein the data source is identified by a pointer indicating a location of the data source.

34. The method of claim 33, further comprising storing the pointer of a data source.

35. The method of claim 33, further comprising re-searching a previously read data source.

36. The method of claim 33, further comprising storing the pointer of a cataloged data source.

37. The method of claim 36, further comprising re-searching a cataloged data source.

38. A method of cataloging data available through a network, comprising:

accessing a web page;

retrieving source code, including text displayed on the web page, from the web site;

querying the retrieved source code for data corresponding to a predefined category;

saving data related to the corresponding data to a data storage device; and

searching the source code for an address of another web page when corresponding data has been identified in the source code.

39. An automatic web page cataloging device, comprising:

a data storage device; and

a processor containing instructions that, when executed by said processor, cause said processor to;

obtain an address of a web page having data from said data storage device;

read source code, including text displayed on the web page, from the first data source;

identify data from the source code that corresponds to a predetermined search category; and

save data related to the corresponding data in a predefined category within said data storage device.

40. The device of claim 40, wherein said processor performs without human intervention.