Ingesting streaming signals

• US 10,404,840 B1
• Filed: 02/25/2019
• Issued: 09/03/2019
• Est. Priority Date: 04/27/2018
• Status: Expired due to Fees

• Current Assignee: Banjo, Inc.
• Original Assignee: Banjo, Inc.
• Inventors: Leung, KW Justin, Urbach, Michael Avner, Guereca-Pinuelas, Armando, Latko, Christopher E., Patton, Damien, Mehta, Rish
• Primary Examiner(s): Zong, Ruolei
• Application Number:
  US16/285,031
• Time in Patent Office: 190 Days
• Field of Search: 709224
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
• CPC Class Codes: G06F 16/353 into predefined classes
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04L 65/75 Media network packet handling
• CPC Class Codes: H04L 65/762 at the source reformatting of additional data in video distribution servers H04N21/2355
• CPC Class Codes: H04L 65/765 intermediate
• CPC Class Codes: H04L 69/22 Parsing or analysis of headers

Abstract

The present invention extends to methods, systems, and computer program products for ingesting streaming signals. Signal ingestion modules sample a frame from a raw streaming signal. A preliminary severity or possible event type is computed from the sampled frame. A deeper inspection request is triggered of the raw streaming signal or of another raw streaming signal. Segments of content from the raw streaming signal or the other raw streaming signal are inspected. An actual severity or actual event type is computed from the inspected content. The actual severity or actual event type can be included in a normalized signal.

Claims

1. A method comprising:

concurrently during ingestion of a raw streaming signal from a streaming signal source;

sampling a single frame contained in the raw streaming signal;

separating the raw streaming signal into a plurality of segments; and

storing the plurality of segments at a storage device;

computing a preliminarily severity of a possible real-world event of a real-world event type from a hint with respect to the more complete content of the raw streaming signal included in the sampled single frame;

triggering a deeper inspection of the raw streaming signal based on the preliminary severity;

performing the deeper inspection of the raw streaming signal, including;

requesting the deeper inspection;

playing a segment, from among the plurality of segments, from the storage device;

inspecting the content of the segment during play of the segment;

computing an actual severity of the real-world event of the real-world event type based on the preliminary severity and the inspected content; and

updating a normalized Time, Location, Context (TLC) signal corresponding to the raw streaming signal to include the actual severity.

2. The method of claim 1, further comprising ingesting the raw streaming signal that includes one or more of:

metadata automatic vehicle location (AVL) data, Internet of Things (IoT) data, drone telemetry, Federal Aviation Association (FAA) telemetry, satellite telemetry, radar data, or water level data.

3. The method of claim 1, wherein sampling the single frame comprises sampling the raw streaming signal in accordance with a sampling frequency.

4. The method of claim 1, wherein storing the plurality of segments at a storage device comprises storing the plurality of segments at a storage device configured to provide multi consumer access by time and range indexing.

5. The method of claim 1, wherein playing a segment comprises feeding the segment to a severity computation module.

6. The method of claim 1, further comprising normalizing the raw streaming signal into the Time, Location, Context (TLC) signal, including:

determining a time associated with the streaming signal;

determining a location associated with the streaming signal; and

including the time, the location, the actual severity, and the plurality of segments in the Time, Location, Context (TLC) signal; and

further comprising sending the Time, Location, Context (TLC) signal to an event detection infrastructure.

7. The method of claim 1, wherein computing a preliminarily severity comprises computing a preliminarily severity based on metadata associated with the raw streaming signal.

8. A method comprising:

sampling a single frame contained in a raw streaming signal from a streaming signal source;

preliminarily classifying the raw streaming signal as indicative of a real-world event type through reference to a content classifier and from a hint with respect to the more complete content of the raw streaming signal included in the sampled single frame;

triggering a deeper inspection of the raw streaming signal based on the preliminary classification;

performing the deeper inspection of the raw streaming signal, including;

requesting the deeper inspection;

playing a portion of the raw streaming signal from the streaming signal source;

inspecting the content of the portion during play of the portion; and

computing a probability that the raw streaming signal is actually indicative of the real-world event type; and

updating a context dimension of a normalized Time, Location, Context (TLC) signal corresponding to the raw streaming signal to include the probability of the real-world event type.

9. The method of claim 8, further comprising ingesting the raw streaming signal that includes one or more of:

metadata automatic vehicle location (AVL) data, Internet of Things (IoT) data, drone telemetry, Federal Aviation Association (FAA) telemetry, satellite telemetry, radar data, or water level data.

10. The method of claim 8, wherein playing the portion comprises feeding the to the content classifier.

11. The method of claim 8, further comprising:

receiving a context annotation from the content classifier; and

looking up a classification tag corresponding to the context annotation.

12. The method of claim 11, further comprising normalizing the raw streaming signal into the Time, Location, Context (TLC) signal, including:

determining a time associated with the streaming signal;

determining a location associated with the streaming signal; and

including the time, the location, the classification tag, and the plurality of segments in the Time, Location, Context (TLC) signal; and

further comprising sending the Time, Location, Context (TLC) signal to an event detection infrastructure.

13. The method of claim 8, wherein preliminarily classifying the raw streaming signal as indicative of a real-world event type comprises preliminarily classifying the raw streaming signal based on metadata associated with the raw streaming signal.

14. A method comprising:

sampling a single frame contained in a first raw streaming signal from a streaming signal source;

preliminarily classifying the first raw streaming signal as indicative of a real-world event type through reference to a content classifier and from a hint with respect to the more complete content of the first raw streaming signal included in the sampled single frame;

triggering a deeper inspection of a second raw signal based on the preliminary classification;

performing the deeper inspection of the second raw signal, including;

requesting the deeper inspection;

playing a segment from among a plurality of segments of the second raw streaming signal;

inspecting content of the segment during replay of the segment; and

computing a probability that the first raw streaming signal is actually indicative of a real-world event of the real-world event type based on the inspected content; and

updating a context dimension of a normalized Time, Location, Context (TLC) signal corresponding to the first raw streaming signal to include the probability of the real-world event type.

15. The method of claim 14, further comprising ingesting the first raw streaming signal that includes one or more of:

metadata, automatic vehicle location (AVL) data, Internet of Things (IoT) data, drone telemetry, Federal Aviation Association (FAA) telemetry, satellite telemetry, radar data, or water level data.

16. The method claim 15, wherein performing the deeper inspection of the second raw signal comprises performing a deeper inspection of the second raw streaming signal that includes a different one or more of:

metadata, automatic vehicle location (AVL) data, Internet of Things (IoT) data, drone telemetry, Federal Aviation Association (FAA) telemetry, satellite telemetry, radar data, or water level data.

17. The method of claim 15, wherein performing the deeper inspection of the second raw signal comprises performing a deeper inspection of the second raw streaming signal that includes one or more of:

audio data or video data.

18. The method of claim 14, wherein playing a segment comprises feeding the segment to the content classifier.

19. The method of claim 14, further comprising:

receiving a context annotation from the content classifier; and

looking up a classification tag corresponding to the context annotation.

20. The method of claim 14, wherein computing a probability that the first raw streaming signal is actually indicative of a real-world event of the real-world event type comprises computing the probability based on metadata associated with the second raw streaming signal.