Automatic indexing of digital image archives for content-based, context-sensitive searching

• US 7,689,544 B2
• Filed: 07/23/2004
• Issued: 03/30/2010
• Est. Priority Date: 07/23/2003
• Status: Active Grant

• Current Assignee: Siemens Healthcare GMBH (Siemens AG)
• Original Assignee: Siemens AG
• Inventors: Koenig, Helmut
• Primary Examiner(s): Alam; Hosain T
• Assistant Examiner(s): Tang; Jieying
• Application Number:
  US10/897,102
• Publication Number:
  US 20050021512A1
• Time in Patent Office: 2,076 Days
• Field of Search: 707/3, 707/1, 707/104.1, 707/10, 382/170, 345/854, 353/25, 705/2, 705/3
• US Class Current: 707/741
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G16H 30/20 for handling medical images, e.g. DICOM, HL7 or PACS

Abstract

A method is for automatically indexing multimedia data archives and categorizing the files held therein and also to a client/server architecture in an image retrieval system for content-based searching for relevant files in a particular format and having a particular file structure. The parsed files stored in a data archive managed by the document management system are subjected to a feature extraction algorithm. The features obtained are then used for producing a binary-coded inverted index which includes elements of at least two attributes and holds context information which is held in these files and is needed for content-based image retrieval. If new files or files with an extended or modified content are stored in the data archive, then it involves a parsing algorithm and also an algorithm for automatically extracting features of these files being executed under event control during every storage process, in order to extend the inverted index by individual attributes or in order to update particular elements of already existing index attributes.

Claims

1. A method for allowing a content-based search of a data archive for files and for image files referenced by the files, the method comprising:

extracting information provided in structured documents from a number of files managed by a document management system in an image retrieval system, to produce an inverted index for content-based searching of image files in a digital image database, including elements of at least two attributes, including a location of files responsive to a search query and process steps to be taken based on the query results, and context information held in the files and used for content-based image retrieval;

executing a parsing algorithm and an algorithm for extracting features of at least one of new files and one of the files whose content has been at least one of extended and altered when the files are stored in the data archive managed by the document management system for the purpose of updating the inverted index;

providing a selection of process steps for further handling of information contained in files which have been found based on the inverted index;

sending a context-sensitive query to the document management system to search for documents using the inverted index by inputting context-sensitive search terms using a browser program running on a client computer;

transmitting a message from the document management system to a process control system in which the process steps for further handling of the information contained in files which have been found and axe relevant to the search terms entered are requested;

forwarding the process steps ascertained by the process control system to the document management system;

providing the files found and the process steps to the client computer;

viewing the files provided by the document management system and viewing the image files referenced by the files provided;

selecting at least one process step which is suitable in consideration of the information viewed; and

transmitting at least one of selected and planned process steps, including the elements of the two attributes from the client computer to the process control system on the basis of the content of the documents provided, wherein

the process steps are related to at least one of medical examinations, therapeutic treatments and medications for a diagnosis included in the inverted index.

2. The inverted index at least one of produced, extended and updated by the method as claimed in claim 1, for content-based searching for at least one of the files and image files, referenced by the files and stored in a digital image archive.

3. The method for allowing a content-based search of a data archive, as claimed in claim 1, further comprising extending the inverted index by executing a parsing algorithm and an algorithm for extracting features of at least one of new files and one of the files whose content has been at least one of extended and altered due to selection of the at least diagnostic and therapeutic process steps.

4. The method for allowing a content-based search of a data archive, as claimed in claim 1, further comprising producing the inverted index in at least one of a DICOM (Digital Imaging and Communication In Medicine) format and a HL7 (Health Level 7) format.

5. The method for allowing a content-based search of a data archive, as claimed in claim 1, further comprising associating the inverted index with one of an individual patient and a group of patients, wherein the inverted index includes patient data recorded during an examination period.

6. A document management system of an image retrieval system including a computer software product stored on a computer readable medium in which computer instructions are stored, which instructions, when executed by a computer, cause the computer to perform a content-based search of a data archive for relevant files and for image files referenced by the files, comprising:

a parsing module configured to perform a syntax analysis of contents of files stored in the data archive; and

a processing module configured to provide event-controlled conversion of parsed documents into a suitable intermediate representation containing features extracted from parsed files and from image files referenced by the parsed files, said features being usable to produce an inverted index for content-based searching of image files in a digital image database, including elements of at least two attributes including a location of files responsive to a search query of the data archive and process steps to be taken based on results of the search query, for context information held in the files and hence for content-based image retrieval during the search for image files in an image archive including a collected medical knowledge database and an inference system that stores collected medical knowledge and the stored analytical rules to weigh terms of the search against the collected medical knowledge and the stored analytical rules to derive inferences, the processing module being further configured to provide a selection of at least one of diagnostic and therapeutic process steps for further handling of information contained in the files based on the at least two attributes.

7. The document management system as claimed in claim 6, further comprising:

at least one of a data storing module and an application programming interface configured to provide information associated with codes and coding schemes and usable for coding the inverted index to be created; and

an indexing module configured to access information relating to at least one of stored and input codes and coding schemes stored in the data store and configured to perform automatic indexing of intermediate representations obtained for content-based image retrieval.

8. A client/server system in an image retrieval system for content-based searching for image files stored in an image archive including,

a client computer configured to send a search query for relevant information, and

a document management system, including a digital image database having elements of at least two attributes including a location of files responsive to a search query and process steps related to at least one of diagnosis and treatment of a patient to be taken based on the query results, an index memory and a digital data archive, configured to analyze the search query and search an inverted index of image files in a digital image database, stored in a memory unit and usable to access image files, and for providing a selection of process steps for further handling of information contained in files found based on the inverted index, the client/server system comprising:

a process control system including a computer software product stored on a computer readable medium in which computer instructions are stored, which instructions, when executed by a computer, cause the computer to perform event-controlled monitoring of heterogeneous processes initiated by the client computer and the document management system, stipulating dependencies and logic sequences for the processes, and storing this information in the document management system, the process control system including a collected medical knowledge database and an inference system that stores collected medical knowledge and stored analytical rules to weigh the search query against the collected medical knowledge and the stored analytical rules to derive inferences, wherein

the process controls system is configured to control transmitting at least one of selected and planned process steps, including the elements of the two attributes from the client computer to the process control system on the basis of the content of the documents provided.

9. The client/server system as claims in claim 8, the process steps are related to at least one of medical examinations, therapeutic treatments and medications for a diagnosis included in inverted index.

10. A document management system of an image retrieval system including a computer software product stored on a computer readable medium in which computer instructions are stored, which instructions, when executed by a computer, cause the computer to perform automatic indexing data archives and categorizing files stored therein for the purpose of allowing a content-based search for relevant files and also for image files referenced by the files and are held in an image archive, comprising:

instructions for performing a syntax analysis of the contents of files stored in the data archive;

instructions for event-controlled conversion of analyzed files into a suitable intermediate representation containing features extracted from the analyzed files and from the image files referenced by the analyzed files, said features being usable to produce an inverted index, including an image index attribute relating to an image recorded for an examined patient, for searching for context information held in the files and hence for content-based image retrieval during the search for image files in the image archive;

instructions for providing a selection of process steps for further handling of information contained in the files;

instructions for transmitting at least one of selected and planned process steps, including elements of two attributes including a location of files responsive to a search query and process steps related to at least one of diagnosis and treatment of a patient to be taken based on the query results; and

instructions for extending the inverted index by transmitting information contained in new documents from a process control system, that includes a collected medical knowledge database and an inference system that takes collected medical knowledge and stored analytical rules to weigh the search query against the collected medical knowledge and stored analytical rules to derive inferences, to a memory unit in the document management system that stores the inverted index.

11. The document management system as claimed in claim 10, further comprising:

instructions for providing information associated with codes and coding schemes and usable for coding the inverted index to be created; and

instructions for accessing information relating to provided information associated with at least one of codes and coding schemes and for performing automatic indexing of the intermediate representations obtained for content-based image retrieval.

12. The document management system as claimed in claim 10, wherein the process steps are related to at least one of medical examinations, therapeutic treatments and medications for a diagnosis included in inverted index.

13. A client/server system in an image retrieval system including a computer software product stored on a computer readable medium in which computer instructions are stored, which instructions, when executed by a computer, cause the computer to perform content-based searching for image files stored in an image archive including,

instructions for sending a search query for relevant information in a collected medical knowledge database and an inference system that takes the collected medical knowledge and stored analytical rules to weigh the search query against the collected medical knowledge and stored analytical rules to derive inferences, and

instructions for analyzing the search query, for searching an inverted index including an image index attribute relating to an image recorded for an examined patient, stored and usable to access image files, and for providing a selection of at least one of diagnostic and therapeutic process steps for further handling of information contained in files found based on the inverted index, the client/server system comprising:

instructions for event-controlled monitoring of heterogeneous processes initiated by a means for sending and a means for analyzing, stipulating dependencies and logic sequences for the processes, and storing this information in the means for analyzing; and

instructions for transmitting at least one of the selected diagnosis and therapeutic process steps to the means for sending.

14. A method for a data archive for files and for image files referenced by the files, the method comprising:

producing an inverted index including elements of at least two attributes and context information held in the files and used for content-based image retrieval;

providing a selection of at least one of diagnostic and therapeutic process steps for further handling of information contained in files which have been found based on the inverted index;

sending a query to a document management system to search for documents using the inverted index by inputting search terms using a browser program running on a client computer;

transmitting a message from the document management system to a process control system in which process steps for further handling of the information contained in files which have been found and are relevant to the search terms entered are requested;

forwarding the process steps ascertained by the process control system to the document management system;

providing the files found and the process steps to the client computer;

viewing the files provided by the document management system and viewing the image files referenced by the files provided;

selecting at least one process step which is suitable in consideration of the information viewed; and

transmitting at least one of selected and planned process steps from the client computer to the process control system on the basis of the content of the documents provided.

15. The method as claimed in claim 14, further comprising a step of extending the inverted index by transmitting selected process steps from the process control system to a memory unit in the document management system and entering these process steps into the memory cells in the elements of index attributes is provided.

16. The inverted index at least one of produced, extended and updated by the method as claimed in claim 14, for content-based searching for at least one of the files and image files, referenced by the files and stored in a digital image archive.

17. An apparatus for a data archive for files and for image files referenced by the files, the apparatus including a computer software product stored on a computer readable medium in which computer instructions are stored, which instructions, when executed by a computer, cause the computer to perform a method of automatic indexing, the method comprising:

executing instructions for producing an inverted index for content-based searching of image files in a digital image database, including elements of at least two attributes, including a location of files responsive to a search query and process steps to be taken based on the query results, and context information held in the files and used for content-based image retrieval;

executing instructions for providing a selection of at least one of diagnostic and therapeutic process steps for further handling of information contained in files which have been found based on the inverted index; and

executing instructions for updating the inverted index, including an image index attribute relating to an image recorded for an examined patient, by executing a parsing algorithm and an algorithm for extracting features of at least one of new files and one of the files whose content has been at least one of extended and altered due to selection of the at least diagnostic and therapeutic process steps.

18. An apparatus for a data archive for files and for image files referenced by the image files, the apparatus comprising a computer software product stored on a computer readable medium in which computer instructions are stored, which instructions, when executed by a computer, cause the computer to:

execute instructions for extracting features from a number of files managed by a document management system in an image retrieval system, to produce an inverted index for content-based searching of image files in a digital image database, including elements of at least two attributes and context information held in the files and used for content-based image retrieval;

executing a parsing algorithm and an algorithm for extracting features of at least one of new files and one of the files whose content has been at least one of extended and altered when the files are stored in the data archive managed by the document management system for the purpose of updating the inverted index;

executing instructions for providing a selection of at least one of diagnostic and therapeutic process steps for further handling of information contained in files which have been found based on the inverted index;

executing instructions for sending a query to the document management system to search for documents using the inverted index by inputting search terms using a browser program running on a client computer;

executing instructions for transmitting a message from the document management system to a process control system in which the process steps for further handling of the information contained in files which have been found and are relevant to the search terms entered are requested;

executing instructions for forwarding the process steps ascertained by the process control system to the document management system;

executing instructions for providing the files found and the process steps to the client computer;

executing instructions for viewing the files provided by the document management system and viewing the image files referenced by the files provided;

executing instructions for selecting at least one process step which is suitable in consideration of the information viewed;

executing instructions for transmitting at least one of selected and planned process steps from the client computer to the process control system on the basis of the content of the documents provided, wherein

the process steps are related to at least one of medical examinations, therapeutic treatments and medications for a diagnosis included in inverted index.