Determining Related Data Points from Multi-Modal Inputs

• US 20140358843A1
• Filed: 05/29/2013
• Published: 12/04/2014
• Est. Priority Date: 05/29/2013
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Subramaniam, L. Venkata, Mehta, Sameep, Krishnapuram, Raghuram, Faruquie, Tanveer A.
• Granted Patent: US 9,396,433 B2
• Time in Patent Office: Days
• US Class Current: 706/58
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation

Abstract

Techniques, systems, and articles of manufacture for determining related data points from multi-modal inputs. A method includes collecting multiple items of multi-modal data comprising at least one dimension from multiple data sources, wherein said at least one dimension comprises a geographic dimension, a temporal dimension and/or an event-related dimension, determining a window of relevance for each of the multiple items of multi-modal data with respect to the at least one dimension, and identifying two or more of the multiple items of multi-modal data as related, by determining an overlap of the window of relevance corresponding to each of the two or more items of multi-modal data with respect to the at least one dimension.

Claims

1. A method comprising:

collecting multiple items of multi-modal data comprising at least one dimension from multiple data sources, wherein said at least one dimension comprises a geographic dimension, a temporal dimension and/or an event-related dimension;

determining a window of relevance for each of the multiple items of multi-modal data with respect to the at least one dimension; and

identifying two or more of the multiple items of multi-modal data as related, by determining an overlap of the window of relevance corresponding to each of the two or more items of multi-modal data with respect to the at least one dimension;

wherein at least one of the steps is carried out by a computer device.

2. The method of claim 1, wherein said collecting comprises collecting multiple items of multi-modal data from multiple data sources over two or more periods of time.

3. The method of claim 1, wherein said collecting comprises collecting multiple items of multi-modal data from multiple data sources corresponding to two or more spatial locations within the geographic region.

4. The method of claim 1, wherein said multi-modal data comprises text message data, speech data, global positioning system data, audio data, mobile call detail records, interne content, and/or mobile-specific content.

5. The method of claim 1, wherein said collecting comprises collecting multiple items of multi-modal data via one or more physical sensors.

6. The method of claim 5, wherein the one or more physical sensors comprise mobile network mobile call detail records, global positioning system sensors, audio sensors, and/or traffic sensors.

7. The method of claim 1, comprising:

annotating each of the multiple items of collected multi-modal data with information.

8. The method of claim 7, wherein said information comprises a time corresponding to said collection, a location corresponding to said collection, a semantic relevance to a given topic, a veracity score associated with the source of the item of data, a weight, and/or a relevance score associated with a given topic.

9. The method of claim 1, wherein said determining a window of relevance comprises manually defining the window of relevance with respect to the at least one dimension based on domain knowledge.

10. The method of claim 1, wherein said determining a window of relevance comprises automatically determining the window of relevance using labeled historical data associated with the at least one dimension.

11. The method of claim 1, wherein said determining an overlap of the window of relevance corresponding to each of the two or more items of multi-modal data with respect to the at least one dimension comprises determining a physical distance between each of the two or more items of multi-modal data.

12. The method of claim 1, wherein said determining an overlap of the window of relevance corresponding to each of the two or more items of multi-modal data with respect to the at least one dimension comprises determining a syntactic distance between each of the two or more items of multi-modal data.

13. The method of claim 12, wherein said determining the syntactic distance comprises measuring the syntactic distance via an edit distance measure, a phonetic distance measure, a pre-fix matching measure, and/or an abbreviation expansion measure.

14. The method of claim 1, wherein said determining an overlap of the window of relevance corresponding to each of the two or more items of multi-modal data with respect to the at least one dimension comprises determining a semantic distance between each of the two or more items of multi-modal data.

15. The method of claim 14, wherein said determining the semantic distance comprises measuring the semantic distance based a relatedness of words contained and/or derived from the two or more items of multi-modal data.

16. The method of claim 1, comprising:

storing the identification of the two or more of the multiple items of multi-modal data as related with respect to the at least one dimension.

17. An article of manufacture comprising a computer readable storage medium having computer readable instructions tangibly embodied thereon which, when implemented, cause a computer to carry out a plurality of method steps comprising:

collecting multiple items of multi-modal data comprising at least one dimension from multiple data sources, wherein said at least one dimension comprises a geographic dimension, a temporal dimension and/or an event-related dimension;

determining a window of relevance for each of the multiple items of multi-modal data with respect to the at least one dimension; and

identifying two or more of the multiple items of multi-modal data as related, by determining an overlap of the window of relevance corresponding to each of the two or more items of multi-modal data with respect to the at least one dimension.

18. A system comprising:

a memory; and

at least one processor coupled to the memory and configured for;

collecting multiple items of multi-modal data comprising at least one dimension from multiple data sources, wherein said at least one dimension comprises a geographic dimension, a temporal dimension and/or an event-related dimension;

determining a window of relevance for each of the multiple items of multi-modal data with respect to the at least one dimension; and

identifying two or more of the multiple items of multi-modal data as related, by determining an overlap of the window of relevance corresponding to each of the two or more items of multi-modal data with respect to the at least one dimension.

19. The system of claim 18, wherein said multi-modal data comprises text message data, speech data, global positioning system data, audio data, mobile call detail records, internet content, and/or mobile-specific content.

20. A method comprising:

collecting multiple items of multi-modal data comprising multiple dimensions from multiple data sources, wherein said multiple dimensions comprise a geographic dimension, a temporal dimension and an event-related dimension;

determining a window of relevance for each of the multiple items of multi-modal data with respect to each of the multiple dimensions;

identifying two or more of the multiple items of multi-modal data as related, by determining an overlap of the window of relevance corresponding to each of the two or more items of multi-modal data with respect to at least two of the multiple dimensions; and

generating an alert based on the content of the two or more related items of multi-modal data;

wherein at least one of the steps is carried out by a computer device.