Methods and Devices for Cognitive-based Image Data Analytics in Real Time

• US 20170372153A1
• Filed: 01/09/2015
• Published: 12/28/2017
• Est. Priority Date: 01/09/2014
• Status: Active Grant

• Current Assignee: Irvine Sensors Corp.
• Original Assignee: Irvine Sensors Corp.
• Inventors: Justice, James, Ludwig, David, Villacorta, Virgilio, Asadi, Omar, Knutson, Fredrik, Yam, Mannchouy, Weaver, Eric
• Granted Patent: US 10,078,791 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/217 Validation; Performance evaluation; Active pattern learning techniques
• CPC Class Codes: G06F 18/2415 based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 2200/28 involving image processing hardware
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/955 using specific electronic processors
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

An invention is disclosed for a real time video analytic processor that embodies algorithms and processing architectures that process a wide variety of sensor images in a fashion that emulates how the human visual path processes and interprets image content. spatial, temporal, and color content of images are analyzed and the salient features of the images determined. These salient features are then compared to the salient features of objects of user interest in order to detect, track, classify, and characterize the activities of the objects. Objects or activities of interest are annotated in the image streams and alerts of critical events are provided to users. Instantiation of the cognitive processing can be accomplished on multi-FPGA and multi-GPU processing hardwares.

Claims

1. A real time video analytic image processor which embodies algorithms, processing architectures, and instantiations on single or multi-FPGA assemblies and on single or multi-GPU assemblies which emulates the human visual path'"'"'s capability for object detection, tracking, classification, and activity interpretation.

2. The Image Processor of claim 1 is capable of processing data from single or multiple imaging sensors.

3. The Image Processor of claim 1 is capable of processing still imagery.

4. The Image Processor of claim 1 is capable of processing video imagery including High Definition (HD) Video and Full Motion Video (FMV).

5. The Image Processor of claim 1 is capable of processing thermal imagery.

6. The Image Processor of claim 1 is capable of processing multispectral imagery.

7. The Image Processor of claim 1 is capable of processing hyperspectral imagery.

8. The Image Processor of claim 1 is capable of processing LIDAR imagery.

9. The Image Processor of claim 1 is capable of processing radar imagery including Synthetic Aperture Radar (SAR) and Ground Moving Target Indicator (GMTI) imagery.