Method and apparatus for searching multiple data element type files

• US 7,162,483 B2
• Filed: 07/15/2002
• Issued: 01/09/2007
• Est. Priority Date: 07/16/2001
• Status: Active Grant

• Current Assignee: Shlomo E. Friman
• Original Assignee: Shlomo E. Friman
• Inventors: Friman, Shlomo E.
• Primary Examiner(s): HWANG, JOON H
• Application Number:
  US10/195,147
• Publication Number:
  US 20030046299A1
• Time in Patent Office: 1,639 Days
• Field of Search: 707 1- 10, 707100-1041
• US Class Current: 1/1
• CPC Class Codes: G06F 16/972 Access to data in other repository systems, e.g. legacy data or dynamic Web page generation
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface

Abstract

Method and apparatus is disclosed for constructing, and updating, a searchable database from very large data collections including diverse types of data thus enabling the data to be cataloged, data mined, indexed, parsed, cross-referenced and published for rapid location and free text retrieval of data elements from terabytes of information or millions of elements. The method and apparatus enables full text searches with the aid of an Internet browser using very complex search criteria without special skills. The full text searches are performed on a searchable database developed from the data collections in a large data storage system and wherein data type elements are identified, hyperlinked when possible, and stored in a retrievable format. In addition to the searchable database, a database is constructed that shows graphically the relationships between a current module being viewed and any other modules the current module may reference or be referenced by.

Claims

1. A method for providing a searchable database derived from a collection of information from a mainframe database, a corporate database, or a large data storage system having a plurality of data elements of multiple data types, each data element comprising a plurality of strings of characters or words, the method comprising:

storing all data elements of a same data type in a corresponding data file;

reading all the data elements in each data file a first time to determine names of the data elements and a data type associated with the data element names;

storing each data element name and corresponding data type in a data element database;

reading all the stored data elements in each data file a second time to examine the character strings or words of each data element;

determining whether any of the examined character strings or words of each data element match a data element name stored in the data element database, regardless of the associated data type of each data element name;

providing a hyperlink for each data element having an examined character string or word matching a data element name stored in the data element database;

storing each data element and any hyperlink in a directory of the data element database corresponding to a data type of a data element; and

indexing data elements in each directory to provide the searchable database.

2. The method of claim 1 further comprising filtering all of the data elements in each data file to remove non-printable characters from the data elements in each data file.

3. The method of claim 1 wherein storing each data element in a directory of the data element database corresponding to the data type of the data element includes storing each read data element as a hyper text markup language file.

4. The method of claim 1 further comprising creating a record in a cross reference database every time a hyperlink is associated with a data element.

5. The method of claim 1 further comprising comparing data files for each data type to new data files for each data type to determine added data elements for each data type.

6. The method of claim 5 further comprising:

reading all added data elements in each new data file a first time to determine names of the added data elements and a data type associated with the added data element names;

storing each added data element name and corresponding data type in the data element database;

reading all the added data elements in each new data file a second time to examine character strings or words of each added data element;

determining whether any of the examined character strings or words of each added data element match a data element name stored in the data element database, regardless of the associated data type of each data element name;

providing a hyperlink for each added data element having an examined character string or word matching a data element name stored in the data element database;

storing each added data element and any corresponding hyperlink in a directory of the data element database corresponding to a data type of an added data element; and

indexing all data elements in each directory to provide a searchable database.

7. The method of claim 1 further comprising comparing data files for each data type to new data files for each data type to determine changed data elements for each data type.

8. The method of claim 7 further comprising:

reading all changed data elements in each new data file a first time to determine names of the changed data elements and a data type associated with the changed data element names;

storing each changed data element name and corresponding data type in the data element database;

reading all the changed data elements in each new data file a second time to examine character strings or words of each changed data element;

determining whether any of the examined character strings or words of each changed data element match a data element name stored in the data element database, regardless of the associated data type of each data element name;

providing a hyperlink for each changed data element having an examined character string or word matching a data element name stored in the element database;

storing each changed data element and any corresponding hyperlink in a directory of the data element database corresponding to a data type of a changed data element; and

indexing all data elements in each directory to provide a searchable database.

9. The method of claim 1 further comprising comparing data files for each data type to new data files for each data type to determine deleted data elements and providing an indication in saved data elements in the searchable database corresponding to the deleted data elements as being archived.

10. The method of claim 1 wherein the data types include one or more of jobs, controls, programs, copy books, source code, procedures, emails, documents, and natural language files.

11. The method of claim 1 further comprising identifying all data elements of the collection of information by sequentially scanning the collection to identify all data elements belonging to data type.

12. The method of claim 11 wherein identifying all data elements further comprises grouping all data elements of the same type into the corresponding data file.

13. The method of claim 12 wherein grouping all data elements further comprises separating data elements by a separator identifier.

14. A searchable database system derived from a collection of information from a mainframe database, a corporate database, or a large data storage system having a plurality of data elements of multiple data types, each data element comprising a plurality of strings of characters or words, the systems comprising:

an interface to receive a plurality of data files storing all data elements of a same data type;

a data element database including a plurality of directories for each data type;

a searchable database; and

a processor to;

read all the stored data elements in each data file a first time to determine names of the data elements and a data type associated with the data element names;

store each data element name and corresponding data type in the data element database;

read all the stored data elements in each data file a second time to examine the character strings or words of each data element;

determine whether any of the examined character strings or words of each data element match a data element name stored in the data element database, regardless of the associated data type of each element name;

provide a hyperlink for each data element having an examined character string or word matching a data element name stored in the data element database;

store each data element and any hyperlink in a directory of the data element database corresponding to a data type of a data element; and

index data elements in each directory to create the searchable database.

15. The system of claim 14 wherein all of the data elements in each data file are filtered to remove non-printable characters from the data elements in each data file.

16. The system of claim 14 wherein each stored data element is a hyper text markup language file.

17. The system of claim 14 further comprising a cross reference database wherein the processor is configured to create a record in the cross reference database every time a hyperlink is associated with a data element.

18. The system of claim 14 wherein the interface is configured to receive new data files and the processor is configured to compare the data files for each data type to the new data files for each data type to determine added data elements for each data type.

19. The system of claim 18 wherein the processor is configured to:

read all added data elements in each new data file a first time to determine names of the added data elements and a data type associated with the added data element names;

store each added data element name and corresponding data type in the data element database;

read all the added data elements in each new data file a second time to examine the character strings or words of each added data element;

determine whether any of the examined character strings or words of each added data element match a data element name stored in the data element database, regardless of the associated data type of each element name;

provide a hyperlink for each added data element having an examined character string or word matching a data element name stored in the data element database;

store each added data element and any corresponding hyperlink in a directory of the data element database corresponding to a data type of an added data element; and

index all data elements in each directory to provide the searchable database.

20. The system of claim 14 wherein the interface is configured to receive new data files and the processor is configured to compare the data files for each data type to the new data files for each data type to determine changed data elements for each data type.

21. The system of claim 20 wherein the processor is configured to:

read all changed data elements in each new data file a first time to determine names of the changed data elements and a data type associated with the changed data element names;

store each changed data element name and corresponding data type in the data element database;

read all the changed data elements in each new data file a second time to examine character strings or words of each changed data element;

determine whether any of the examined character strings or words of each changed data element match a data element name stored in the data element database, regardless of the associated data type of each element name;

provide a hyperlink for each changed data element having an examined character string or word matching a data element name stored in the data element database;

store each changed data element and any corresponding hyperlink in a directory of the data element database corresponding to a data type of a changed data element; and

index all data elements in each directory to provide a searchable database.

22. The system of claim 14 wherein the interface is configured to receive new data files and the processor is configured to compare the data files for each data type to the new data files for each data type to determine deleted data elements and to provide an indication in saved data elements in the searchable database corresponding to the deleted data elements as being archived.

23. The system of claim 14 wherein the data types include one or more of jobs, controls, programs, copy books, source code, procedures, emails, documents, and natural language files.