Sensor exploration and management through adaptive sensing framework

• US 20080243439A1
• Filed: 03/28/2007
• Published: 10/02/2008
• Est. Priority Date: 03/28/2007
• Status: Abandoned Application

• Current Assignee: Signal Innovations Group, Inc. (BAE Systems Plc)
• Original Assignee: Signal Innovations Group, Inc. (BAE Systems Plc)
• Inventors: Stern, Trampas, Eliazar, Austin I.D., Carin, Lawrence, Runkle, Paul R., Tank, Tushar
• Application Number:
  US11/727,668
• Publication Number:
  US 20080243439A1
• Time in Patent Office: Days
• US Class Current: 702/188
• CPC Class Codes: G06F 18/253 of extracted features
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 10/62 relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
• CPC Class Codes: G08B 21/0423 detecting deviation from an expected pattern of behaviour or schedule
• CPC Class Codes: G08B 21/043 detecting an emergency event, e.g. a fall
• CPC Class Codes: G08B 21/0476 Cameras to detect unsafe condition, e.g. video cameras

Abstract

The identification and tracking of objects from captured sensor data relies upon statistical modeling methods to sift through large data sets and identify items of interest to users of the system. Statistical modeling methods such as Hidden Markov Models in combination with particle analysis and Bayesian statistical analysis produce items of interest, identify them as objects, and present them to users of the system for identification feedback. The integration of a training component based upon the relative cost of sampling sensors for additional parameters, provides a system that can formulate and present policy decisions on what objects should be tracked, leading to an improvement in continuous data collection and tracking of identified objects within the sensor data set.

Claims

1. A system for collecting data from a deployed sensor network and providing predictive analysis for use in system operations comprising:

at least two sensors located in geospatially separate areas;

a communications means for transporting collected data from said sensors to a system server;

a memory storage unit within said server on which are stored software modules for tracking, activity evaluation, sensor management agent, sensor control, and issuing system alerts to users;

said software modules using statistical modeling means for predictive state management based upon a plurality of parameters to produce a probabilistic evaluation for an occurrence of event change in the modeled sensor data;

using said predicted probabilistic evaluation data to preferentially select portions of said collected sensor data for continued evaluation;

without human input, identify previously unknown events or objects within said collected sensor data and provide said information to a decision agent software process;

said software modules accepting feedback from said users to update a learning database for defining said preferentially selected sensor data within said system server;

issuing sensor control signals from said sensor management agent software module to said sensors located in geospatially separate areas to request additional sensor data collection, or to modify parameters for sensor data collection;

without human supervision, comparing said preferentially selected portions of collected sensor data to a predefined set of events and causing said decision agent process to issue said system alert to users when any of said predefined events is detected and a pre-set risk threshold is exceeded.

2. A system as shown in claim 1 for collecting data from a deployed sensor network and providing predictive analysis for use in system operations further comprising:

said sensors may be sensors that collect video, audio, radar, infrared, ultrasonic, or hyper-spectral data, or any combination of said sensor types.

3. A system as shown in claim 1 for collecting data from a deployed sensor network and providing predictive analysis for use in system operations further comprising:

Said tracking software module receives sensor input data from deployed sensor devices;

Said tracking software module is active to modify a sensor input data base;

Said tracking software transforms sensor input data into object data and stores said object data into an object and object state data base;

Said tracking software module operates upon received sensor data to reconcile data changes between predicted object change and observed object change in said received sensor data and update said sensor input data base;

Said tracking software module utilizes said data changes to produce state data for objects defined in said sensor input data;

Said tracking software module outputs said object state data to said sensor management agent software module.

4. A system as shown in claim 1 for collecting data from a deployed sensor network and providing predictive analysis for use in system operations further comprising:

Said sensor management agent software module accepts object state data from said tracking software module;

Said sensor management agent software module establishes an information value for each object state based upon a cost for acquiring new observed data for said object state, user feedback, state update data, state prediction data, and a risk assessment value as input from said activity evaluation software module;

Said sensor management agent software module statistical modeling algorithms to calculate an expected relative valuation for each object and sensor measurement action and provides this data to said decision agent process;

Said sensor management agent software module, without human intervention, develops a policy for decisions regarding escalation of object state data for further action by the system and outputs sensor control and system alert information to sensors and users of the system.

5. A system as shown in claim 1 for collecting data from a deployed sensor network and providing predictive analysis for use in system operations further comprising:

Said activity evaluation software module accepts evaluated object state data from said sensor management agent and training feedback data from a system user;

Said activity evaluation software module utilizes training feedback data to actively identify new objects and update the object model data base stored within said server;

Said activity evaluation software module evaluates object state data through the use of a Bayesian modeling means to identify a level of risk that each identified object is a normal object for the given data model and outputs said risk assessment to said sensor management agent software module.

6. A system as shown in claim 1 for collecting data from a deployed sensor network and providing predictive analysis for use in system operations further comprising:

Said statistical modeling means utilizes Hidden Markov Model statistical modeling.

7. A system as shown in claim 1 for collecting data from a deployed sensor network and providing predictive analysis for use in system operations further comprising:

Said statistical modeling means utilizes principal components analysis.

8. A system as shown in claim 1 for collecting data from a deployed sensor network and providing predictive analysis for use in system operations further comprising:

Said statistical modeling means utilizes nonlinear object ID tracking.

9. A system as shown in claim 3 for collecting data from a deployed sensor network and providing predictive analysis for use in system operations further comprising:

Said stored object data is created using a parametric representation of the distance between the object centroid and the external object boundary as a function of angle;

10. A system as shown in claim 3 for collecting data from a deployed sensor network and providing predictive analysis for use in system operations further comprising:

Said stored object state data is created using a particle filtering framework algorithm that uses level-sets analysis for each update step.

11. A system as shown in claim 3 for collecting data from a deployed sensor network and providing predictive analysis for use in system operations further comprising:

Said predicted object change data is created by a partially observed Markov decision policy (POMDP) algorithm;

12. A system as shown in claim 11 for collecting data from a deployed sensor network and providing predictive analysis for use in system operations further comprising:

Means for said POMDP statistical model algorithm to use inputs of collected sensor state, action, observation, and cost data to produce said object change data.

13. A system as shown in claim 1 for collecting data from a deployed sensor network and providing predictive analysis for use in system operations further comprising:

Means for utilizing a POMDP algorithm to identify previously unknown events or objects within said collected sensor data without prior identification;

Providing said previously unknown event and object data as input to said decision agent module.

14. A system as shown in claim 4 for collecting data from a deployed sensor network and providing predictive analysis for use in system operations further comprising:

Wherein said cost is associated with deploying sensors and collecting data from said sensors;

And wherein said cost further comprises a fixed cost for performing a sensor measurement and a predicted cost for the difficulty of requesting said sensor measurement.

15. A system as shown in claim 4 for collecting data from a deployed sensor network and providing predictive analysis for use in system operations further comprising:

Wherein said sensor management agent updates object state information;

Said sensor management agent utilizes sensor planning data in combination with said updated object state information to create prediction data for the value of said object state data to be collected by the next collection measurement action.

16. A system as shown in claim 4 for collecting data from a deployed sensor network and providing predictive analysis for use in system operations further comprising:

Wherein said policy decisions are those decisions that cause sensor measurement activities to be initiated.

17. A system as shown in claim 5 for collecting data from a deployed sensor network and providing predictive analysis for use in system operations further comprising:

Wherein said training feedback data is provided by interaction with a user of the system to initialize object and object state data base tables;

And wherein said training feedback data is requested by the system on a periodic bases only, after initialization of said object and object state data base tables.

18. A method for collecting data from a deployed sensor network and providing predictive analysis for use in system operations comprising:

deploying at least two sensors located in geospatially separate areas;

means for transporting collected data from said sensors to a system server;

storing data into a memory storage unit within said server including software modules for tracking, activity evaluation, sensor management agent, sensor control, and issuing system alerts to users;

said software modules using statistical modeling means for predictive state management based upon a plurality of parameters to produce a probabilistic evaluation for an occurrence of event change in the modeled sensor data;

using said predicted probabilistic evaluation data to preferentially select portions of said collected sensor data for continued evaluation;

without human input, identifying previously unknown events or objects within said collected sensor data and provide said information to a decision agent software process;

said software modules accepting feedback from said users to update a learning database for defining said preferentially selected sensor data within said system server;

issuing sensor control signals from said sensor management agent software module to said sensors located in geospatially separate areas to request additional sensor data collection, or to modify parameters for sensor data collection;

without human supervision, comparing said preferentially selected portions of collected sensor data to a predefined set of events and causing said decision agent process to issue said system alert to users when any of said predefined events is detected and a pre-set risk threshold is exceeded.

19. A method as shown in claim 18 for collecting data from a deployed sensor network and providing predictive analysis for use in system operations further comprising:

deploying sensors that collect video, audio, radar, infrared, ultrasonic, or hyper-spectral data, or any combination of said sensor types.

20. A method as shown in claim 18 for collecting data from a deployed sensor network and providing predictive analysis for use in system operations further comprising:

Said tracking software module receiving sensor input data from deployed sensor devices;

Said tracking software module modifying a sensor input data base;

Said tracking software transforming sensor input data into object data and storing said object data into an object and object state data base;

Said tracking software module operating upon received sensor data to reconcile data changes between predicted object change and observed object change in said received sensor data and update said sensor input data base;

Said tracking software module utilizing said data changes to produce state data for objects defined in said sensor input data;

Said tracking software module transferring said object state data to said sensor management agent software module.

21. A method as shown in claim 18 for collecting data from a deployed sensor network and providing predictive analysis for use in system operations further comprising:

Said sensor management agent software module accepting object state data from said tracking software module;

Said sensor management agent software module establishing an information value for each object state based upon a cost for acquiring new observed data for said object state, user feedback, state update data, state prediction data, and a risk assessment value as input from said activity evaluation software module;

Said sensor management agent software module using statistical modeling algorithms to calculate an expected relative valuation for each object and sensor measurement action and provides this data to said decision agent process;

Said sensor management agent software module, without human intervention, developing a policy for decisions regarding escalation of object state data for further action by the system and relaying sensor control and system alert information to sensors and users of the system.

22. A method as shown in claim 18 for collecting data from a deployed sensor network and providing predictive analysis for use in system operations further comprising:

Said activity evaluation software module accepting evaluated object state data from said sensor management agent and training feedback data from a system user;

Said activity evaluation software module utilizing training feedback data to actively identify new objects and update the object model data base stored within said server;

Said activity evaluation software module evaluating object state data through the use of a Bayesian modeling means to identify a level of risk that each identified object is a normal object for the given data model and relaying said risk assessment to said sensor management agent software module.

23. A method as shown in claim 18 for collecting data from a deployed sensor network and providing predictive analysis for use in system operations further comprising:

Said statistical modeling means utilizing Hidden Markov Model statistical modeling.

24. A method as shown in claim 18 for collecting data from a deployed sensor network and providing predictive analysis for use in system operations further comprising:

Said statistical modeling means utilizing principal components analysis.

25. A method as shown in claim 18 for collecting data from a deployed sensor network and providing predictive analysis for use in system operations further comprising:

Said statistical modeling means utilizing nonlinear object ID tracking.

26. A method as shown in claim 20 for collecting data from a deployed sensor network and providing predictive analysis for use in system operations further comprising:

creating said stored object data using a parametric representation of the distance between the object centroid and the external object boundary as a function of angle;

27. A method as shown in claim 20 for collecting data from a deployed sensor network and providing predictive analysis for use in system operations further comprising:

creating said stored object state data using a particle filtering framework algorithm that uses level-sets analysis for each update step.

28. A method as shown in claim 20 for collecting data from a deployed sensor network and providing predictive analysis for use in system operations further comprising:

creating said predicted object change data by a partially observed Markov decision policy (POMDP) algorithm;

29. A method as shown in claim 28 for collecting data from a deployed sensor network and providing predictive analysis for use in system operations further comprising:

Means for initializing said POMDP statistical model algorithm using inputs of collected sensor state, action, observation, and cost data to produce said object change data.

30. A method as shown in claim 18 for collecting data from a deployed sensor network and providing predictive analysis for use in system operations further comprising:

Means for utilizing a POMDP algorithm to identify previously unknown events or objects within said collected sensor data without prior identification;

Providing said previously unknown event and object data as input to said decision agent module.

31. A method as shown in claim 21 for collecting data from a deployed sensor network and providing predictive analysis for use in system operations further comprising:

Wherein said cost is associated with deploying sensors and collecting data from said sensors;

And wherein said cost further comprises a fixed cost for performing a sensor measurement and a predicted cost for the difficulty of requesting said sensor measurement.

32. A method as shown in claim 21 for collecting data from a deployed sensor network and providing predictive analysis for use in system operations further comprising:

Wherein said sensor management agent updates object state information;

Said sensor management agent utilizing sensor planning data in combination with said updated object state information to create prediction data for the value of said object state data to be collected by the next collection measurement action.

33. A method as shown in claim 21 for collecting data from a deployed sensor network and providing predictive analysis for use in system operations further comprising:

Wherein said policy decisions are those decisions causing sensor measurement activities to be initiated.

34. A method as shown in claim 22 for collecting data from a deployed sensor network and providing predictive analysis for use in system operations further comprising:

Wherein said training feedback data is provided by interacting with a user of the system to initialize object and object state data base tables;

And wherein said training feedback data is requested by the system on a periodic bases only, after initialization of said object and object state data base tables.