Optimizing map/reduce searches by using synthetic events

• US 8,799,269 B2
• Filed: 01/03/2012
• Issued: 08/05/2014
• Est. Priority Date: 01/03/2012
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Friedlander, Robert R., Kraemer, James R.
• Primary Examiner(s): HO, BINH VAN
• Application Number:
  US13/342,406
• Publication Number:
  US 20130173585A1
• Time in Patent Office: 945 Days
• Field of Search: 707/713
• US Class Current: 707/713
• CPC Class Codes: G06F 16/90335 Query processing

Abstract

A processor-implemented method, system, and/or computer program product optimizes a search for data from documents. A processor receives an instruction to perform an initial map/reduce search for a specific set of data in documents from a first database. A synthetic event, which is a non-executable descriptor of the specific set of data in documents from the first database, is generated, and a revised map/reduce search for the synthetic event in a second database is conducted. The processor then returns a solution for the revised map/reduce search.

Claims

1. A processor-implemented method of optimizing a search for data from documents, the processor-implemented method comprising:

a processor receiving an instruction from a requester to perform an initial map/reduce search for a specific combination of data located in each document from a first database;

the processor generating a synthetic event, wherein the synthetic event is a non-executable descriptor of the specific combination of data that is located in documents from the first database, and wherein the synthetic event is stored in a second database;

the processor transmitting a message to the requester, wherein the message identifies the synthetic event in the second database that describes the specific combination of data located in the documents from the first database;

the processor generating a copy of the synthetic event for each document, in the first database, that contains the specific combination of data;

the processor generating a synthetic event count of how many copies of the synthetic event are stored in the second database;

the processor receiving and performing a revised map/reduce search for the synthetic event in the second database, wherein the revised map/reduce search determines how many copies of the synthetic event are stored in the second database; and

the processor returning a solution for the revised map/reduce search, wherein the solution describes the synthetic event count of the copies of the synthetic event found in the second database.

2. The processor-implemented method of claim 1, wherein documents in the first database are image files, wherein processing the image files causes an image to be rendered on a display, and wherein the processor-implemented method further comprises:

the processor describing the synthetic event in a non-executable file, wherein the non-executable file describes visual attributes of the image rendered on the display.

3. The processor-implemented method of claim 1, wherein documents in the first database are image files, wherein processing the image files causes an image to be rendered on a display, and wherein the processor-implemented method further comprises:

the processor describing the synthetic event in a non-executable file, wherein the non-executable file describes physical attributes of objects depicted by the image rendered on the display.

4. The processor-implemented method of claim 1, wherein documents in the first database are medical image files, wherein processing the medical image files causes an image of an anatomical object to be rendered on a display, and wherein the processor-implemented method further comprises:

the processor retrieving, from medical records, clinical information related to the medical image files; and

the processor describing the synthetic event in a non-executable file, wherein the non-executable file describes, based on the clinical information, a medical cause of a condition of the anatomical object that is depicted on the display.

5. The processor-implemented method of claim 1, wherein documents in the first database are audio files, wherein the audio files are a record of an audio event, and wherein the processor-implemented method further comprises:

the processor describing the synthetic event in a non-executable file, wherein the non-executable file describes the audio event.

6. The processor-implemented method of claim 1, wherein documents in the first database are audio files, wherein the audio files are a record of an audio event, and wherein the processor-implemented method further comprises:

the processor describing the synthetic event in a non-executable file, wherein the non-executable file describes a cause of the audio event.

7. The processor-implemented method of claim 1, wherein documents in the first database are disparately formatted files that all describe a same physical event, and wherein the processor-implemented method further comprises:

the processor describing the synthetic event in a non-executable file, wherein the non-executable file describes the same physical event.

8. The processor-implemented method of claim 7, wherein the disparately formatted files comprise a text file, an image file, and a physical sensor output file.

9. The processor-implemented method of claim 1, wherein documents from the first database describe a same type of physical attribute using different scales, and wherein the processor-implemented method further comprises:

the processor describing the synthetic event in a non-executable file, wherein the non-executable file harmonizes the different scales into a same scale.

10. The processor-implemented method of claim 1, further comprising:

the processor receiving physical readings from at least one physical sensor, wherein said physical readings describe a physical event, and wherein said physical event and said synthetic event are described by a same set of attributes; and

the processor describing the synthetic event in a document.

11. The processor-implemented method of claim 1, further comprising:

the processor generating the synthetic event in response to a predefined secondary event occurring, wherein the predefined secondary event is a physical activity being performed by an observed subject of at least one physical sensor, and wherein the synthetic event is described in a non-executable file that describes the physical activity being performed by the observed subject.

12. The processor-implemented method of claim 1, further comprising:

the processor limiting the synthetic event to correspond to a predetermined period of time from when the instruction to perform the initial map/reduce search was received.

20. The processor-implemented method of claim 1, wherein the specific combination of data comprises a combination of words described as word A, word B, and word C, and wherein the processor-implemented method further comprises:

the processor determining a likelihood of the specific combination of data being located in the first database based on a Bayesian probability formula of;

13. A computer program product for optimizing a search for data from documents, the computer program product comprising:

a non-transitory computer readable storage media;

first program instructions to receive an instruction from a requester to perform an initial map/reduce search for a specific combination of data located in each document from a first database;

second program instructions to generate a synthetic event, wherein the synthetic event is a non-executable descriptor of the specific combination of data that is located in documents from the first database, and wherein the synthetic event is stored in a second database;

third program instructions to transmit a message to the requester, wherein the message identifies the synthetic event in the second database that describes the specific combination of data located in the documents from the first database;

fourth program instructions to generate a copy of the synthetic event for each document, in the first database, that contains the specific combination of data;

fifth program instructions to generate a synthetic event count of how many copies of the synthetic event are stored in the second database;

sixth program instructions to receive and perform a revised map/reduce search for the synthetic event in the second database, wherein the revised map/reduce search determines how many copies of the synthetic event are stored in the second database; and

seventh program instructions to return a solution for the revised map/reduce search, wherein the solution describes the synthetic event count of the copies of the synthetic event found in the second database; and

wherein the first, second, third, fourth, fifth, sixth, and seventh program instructions are stored on the non-transitory computer readable storage media.

14. The computer program product of claim 13, further comprising:

eighth program instructions to generate the synthetic event in response to a predefined secondary event occurring, wherein the predefined secondary event is a physical activity being performed by an observed subject of at least one physical sensor, and wherein the synthetic event is described in a non-executable file that describes the physical activity being performed by the observed subject; and

wherein the eighth program instructions are stored on the non-transitory computer readable storage media.

15. The computer program product of claim 13, wherein documents in the first database are image files, wherein processing the image files causes an image to be rendered on a display, and wherein the computer program product further comprises:

eighth program instructions to describe the synthetic event in a non-executable file, wherein the non-executable file describes physical attributes of the image rendered on the display; and

wherein the eighth program instructions are stored on the non-transitory computer readable storage media.

16. The computer program product of claim 13, further comprising:

eighth program instructions to receive physical readings from at least one physical sensor, wherein said physical readings describes a physical event, and wherein said physical event and said synthetic event are described by a same set of attributes; and

ninth program instructions to describe the synthetic event in a non-executable file; and

wherein the eighth and ninth program instructions are stored on the non-transitory computer readable storage media.

17. A computer system comprising:

a central processing unit (CPU), a computer readable memory, and a computer readable storage media;

first program instructions to receive an instruction from a requester to perform an initial map/reduce search for a specific combination of data located in each document from a first database;

second program instructions to generate a synthetic event, wherein the synthetic event is a non-executable descriptor of the specific combination of data that is located in documents from the first database, and wherein the synthetic event is stored in a second database;

third program instructions to transmit a message to the requester, wherein the message identifies the synthetic event in the second database that describes the specific combination of data located in the documents from the first database;

fourth program instructions to generate a copy of the synthetic event for each document, in the first database, that contains the specific combination of data;

fifth program instructions to generate a synthetic event count of how many copies of the synthetic event are stored in the second database;

sixth program instructions to receive and perform a revised map/reduce search for the synthetic event in the second database, wherein the revised map/reduce search determines how many copies of the synthetic event are stored in the second database; and

seventh program instructions to return a solution for the revised map/reduce search, wherein the solution describes the synthetic event count of the copies of the synthetic event found in the second database; and

wherein the first, second, third, fourth, fifth, sixth, and seventh program instructions are stored on the computer readable storage media for execution by the CPU via the computer readable memory.

18. The computer system of claim 17, further comprising:

eighth program instructions to generate the synthetic event in response to a predefined secondary event occurring, wherein the predefined secondary event is a physical activity being performed by an observed subject of at least one physical sensor, and wherein the synthetic event is described in a non-executable file that describes the physical activity being performed by the observed subject; and

wherein the eighth program instructions are stored on the computer readable storage media for execution by the CPU via the computer readable memory.

19. The computer system of claim 17, wherein documents in the first database are image files, wherein processing the image files causes an image to be rendered on a display, and wherein the computer system further comprises:

eighth program instructions to describe the synthetic event in a non-executable file, wherein the non-executable file describes physical attributes of the image rendered on the display; and

wherein the eighth program instructions are stored on the computer readable storage media for execution by the CPU via the computer readable memory.