Normalizing ingested signals

• US 10,324,948 B1
• Filed: 07/18/2018
• Issued: 06/18/2019
• Est. Priority Date: 04/27/2018
• Status: Expired due to Fees

• Current Assignee: Banjo, Inc.
• Original Assignee: Banjo, Inc.
• Inventors: Leung, KW Justin, Urbach, Michael Avner, Guereca-Pinuelas, Armando, Latko, Christopher
• Primary Examiner(s): Singh, Amresh
• Application Number:
  US16/038,537
• Time in Patent Office: 335 Days
• Field of Search: None
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
• CPC Class Codes: G06F 16/285 Clustering or classification

Abstract

The present invention extends to methods, systems, and computer program products for normalizing ingested signals. In general, signal ingestion modules ingest different types of raw structured and/or raw unstructured signals on an ongoing basis. The signal ingestion modules normalize raw signals into normalized signals having a Time, Location, Context (or “TLC”) dimensions. A Time (T) dimension can be a time of origin or alternatively “event time” of a signal. A Location (L) dimension can be anywhere across a geographic area, such as, a country (e.g., the United States), a State, a defined area, an impacted area, an area defined by a geo cell, an address, etc. A Context (C) dimension indicates circumstances surrounding formation/origination of a raw signal in terms that facilitate understanding and assessment of the raw signal. The Context (C) dimension of a raw signal can be derived from express as well as inferred signal features of the raw signal.

Claims

1. A method comprising:

ingesting a raw signal including a time stamp, an indication of a signal type, an indication of a signal source, and content;

normalizing the raw signal into a normalized signal by reducing the dimensionality of the raw signal, including;

determining a time dimension associated with the raw signal from the time stamp;

inferring location annotations and context annotations from characteristics of the raw signal;

determining a location dimension associated with the raw signal from one or more of;

location information included in the raw signal or the location annotations;

determining a context dimension associated with the raw signal from one or more of;

context information included in the raw signal or the context annotations, including;

calculating a probability of a real-world event type as a ratio of raw signals turning into real-world events; and

calculating probability details indicating a probabilistic model used to calculate the probability and features of the raw signal considered in calculating the probability; and

including the time dimension, the location dimension, the context dimension, the indication of the signal type, the indication of the signal source, and the content in the normalized signal, wherein the context dimension includes the probability and probability details; and

detecting an occurring real-world event of the real-world event type from the time dimension, location dimension, and context dimension included in the normalized signal; and

notifying one or more entities about the real-world event.

2. The method of claim 1, further comprising indicating the probability details in a hash field.

3. The method of claim 2, further comprising deriving the hash field.

4. The method of claim 1, wherein detecting an occurring real-world event comprises detecting one of:

a fire, police presence, an accident, a natural disaster, weather, a shooter, a concert, or a protest; and

wherein notifying one or more entities about the real-world event comprises notifying one of;

a person, a business entity, or a governmental agency.

5. The method of claim 1, wherein ingesting a raw signal comprises ingesting a raw signal that includes generic source data, the raw signal selected from among:

a social signal, a web signal, or a streaming signal; and

wherein normalizing the raw signal into a normalized signal comprises re-encoding the generic source data into normalized source data having lower dimensionality.

6. The method of claim 5, wherein re-encoding the generic source data into normalized source data having lower dimensionality comprises applying a transdimensionality transform defined in a Time, Location, Context (“

TLC”

) dimensional model to the generic source data.

7. The method of claim 1, wherein calculating a probability of a real-world event type as a ratio of raw signals turning into real-world events comprises calculating the probability as a ratio of raw signals turning into real-world events based on signal properties of the raw signal including:

a media type, the signal source, and a geographic type.

8. A computer-implemented method comprising:

ingesting a raw signal including a time stamp, location information, an indication of a signal type, an indication of a signal source, and content;

normalizing the raw signal into a normalized signal, including;

determining a time dimension from the time stamp;

determining a location dimension from the location information;

inserting the time dimension and location dimension into a TL signal;

storing the TL signal in TL message storage;

subsequent to storing the TL signal, accessing the TL signal from the TL message storage;

inferring context annotations based on characteristics of the TL signal, including one or more of;

the time dimension, the location dimension, the signal type, the signal source, and the content;

appending the context annotations to the TL signal;

determining a context dimension associated with the TL signal from the context annotations, including;

calculating a probability of a real-world event type as a ratio of raw signals turning into real-world events; and

calculating probability details indicating a probabilistic model used to calculate the probability and features of the raw signal considered in calculating the probability; and

including the time dimension, the location dimension, the context dimension, the indication of the signal type, the indication of the signal source, and the content in the normalized signal, wherein the context includes the probability and probability details; and

storing the normalized signal in TLC message storage; and

detecting an occurring real-world event of the real-world event type from the time dimension, location dimension, and context dimension included in the normalized signal

notifying one or more entities about the real-world event.

9. The computer-implemented method of claim 8, wherein ingesting a raw signal comprises ingesting one of a social signal, a web signal, or a streaming signal.

10. The computer-implemented method of claim 8, wherein determining a location dimension from the location information comprises identifying one or more of:

a geo cell and a designated market area (DMA).

11. The computer-implemented method of claim 8, wherein looking up classification tags comprises looking up classification tags corresponding to event types.

12. The computer-implemented method of claim 8, further comprising:

deriving a hash field; and

indicating the probabilistic model and features of the raw signal in the hash field; and

wherein including the probability and probability details in the normalized signal comprises including the hash field in the normalized signal.

13. The computer-implemented method of claim 8, wherein ingesting a raw signal comprising ingesting a raw signal that includes generic source data;

and

wherein normalizing the raw signal into a normalized signal comprising applying a transdimensionality transform defined in a Time, Location, Context (“

TLC”

) dimensional model to the generic source data re-encoding the generic source data into normalized source data having lower dimensionality.

14. The computer-implemented method of claim 8, wherein calculating a probability of a real-world event type comprises computing the probability as a ratio of raw signals turning into real-world events based on signal properties of the raw signal including:

a media type, the signal source, and a geographic type.

15. A computer system comprising:

a processor;

system memory coupled to the processor and storing instructions configured to cause the processor to;

ingest a raw signal including a time stamp, an indication of a signal type, an indication of a signal source, and content;

normalize the raw signal into a Time, Location, Context (“

TLC”

) normalized signal by reducing the dimensionality of the raw signal, including;

determine a time dimension associated with the raw signal from the time stamp;

infer location annotations and context annotations from characteristics of the raw signal;

determine a location dimension associated with the raw signal from one or more of;

location information included in the raw signal or the location annotations;

determine a context dimension associated with the raw signal from one or more of;

context information included in the raw signal or the context annotations, including;

calculate a probability of a real-world event type as a ratio of raw signals turning into real-world events; and

calculating probability details indicating a probabilistic model used to calculate the probability and features of the raw signal considered in calculating the probability; and

include the time dimension, the location dimension, the context dimension, the indication of the signal type, the indication of the signal source, and the content in the Time, Location, Context (“

TLC”

) normalized signal, wherein the context dimension includes the probability and probability details; and

detect an occurring real-world event of the real-world event type from the time dimension, location dimension, and context dimension included in the normalized signal; and

notify one or more entities about the real-world event.

16. The computer system of claim 15, wherein instructions configured to cause the processor to detect a real-world event comprise instructions configured to cause the processor to detect one of:

a fire, police presence, an accident, a natural disaster, weather, a shooter, a concert, or a protest; and

wherein instructions configured to notify one or more entities about the real-world event comprise instructions configured to cause the processor to notify one of;

a person, a business entity, or a governmental agency about the occurring real-world event.

17. The computer system of claim 15, further comprising instructions configured to cause the processor to:

derive a hash field; and

indicate the probabilistic model and features of the raw signal in the hash field; and

wherein instructions configured to cause the processor to include the probability and probability details in the Time, Location, Context (“

TLC”

) normalized signal comprise instructions configured to cause the processor to include the hash field in the Time, Location, Context (“

TLC”

) normalized signal.

18. The computer system of claim 15, wherein instructions configured to cause the processor to ingest a raw signal comprise instructions configured to cause the processor to ingest a raw signal that includes generic source data, the raw signal selected from among:

a social signal, a web signal, or a streaming signal; and

wherein instructions configured to cause the processor to normalize the raw signal into a Time, Location, Context (“

TLC”

) normalized signal comprise instructions configured to cause the processor to apply a transdimensionality transform defined in a Time, Location, Context (“

TLC”

) dimensional model to the generic source data to re-encode the generic source data into normalized source data having lower dimensionality.

19. The computer system of claim 15, wherein instructions configured to cause the processor to calculate a probability of a real-world event type as a ratio of raw signals turning into real-world events comprise instructions configured to cause the processor to compute the probability as a ratio of raw signals turning into real-world events based on signal properties of the raw signal including:

a media type, the signal source, and a geographic type.