System and method for processing image data

• US 9,911,065 B2
• Filed: 10/27/2015
• Issued: 03/06/2018
• Est. Priority Date: 09/19/2012
• Status: Active Grant

• Current Assignee: Placemeter, LLC
• Original Assignee: Placemeter, LLC
• Inventors: Winter, Alexandre
• Primary Examiner(s): Dunphy, David F
• Application Number:
  US14/924,142
• Publication Number:
  US 20160048740A1
• Time in Patent Office: 861 Days
• CPC Class Codes: G06F 11/00 Error detection; Error correction; Monitoring error detection, correction or monitoring in information storage based on relative movement between record carrier and transducer G11B20/18; monitoring, i.e. supervising the progress of recording or reproducing G11B27/36; in static stores G11C29/00
• CPC Class Codes: G06F 11/3058 Monitoring arrangements for monitoring environmental properties or parameters of the computing system or of the computing system component, e.g. monitoring of power, currents, temperature, humidity, position, vibrations thermal management in cooling arrangements of a computing system G06F1/206
• CPC Class Codes: G06F 11/34 Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 11/3476 Data logging G06F11/14, G06F11/2205 take precedence
• CPC Class Codes: G06F 16/90 Details of database functions independent of the retrieved data types
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 11/206 Drawing of charts or graphs
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 20/48 Matching video sequences
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

The PLACEMETER PLATFORM APPARATUSES, METHODS AND SYSTEMS (“PM-PLATFORM”) transform sensor data and/or feedback via PM-PLATFORM components into notifications, updates, coupons, promotions, transactions and/or activities notifications, updates, coupons, promotions, transactions and/or activities. In one implementation, the PM-PLATFORM comprises a sensor, a memory, and a processor disposed in communication with the sensor and memory, the memory storing processor-issuable instructions to receive raw environment data at a sensor for at least two discrete points in time, analyze the received raw environment data locally to determine an at least one occupancy metric, store the occupancy metric, receive further raw environment data for a further point in time, process the further raw environment data to determine a further occupancy metric, compare the further occupancy metric to at least one previous occupancy metric, and issue a notification based on the comparison.

Claims

1. A computer implemented method for analyzing electronic signal data, the method comprising:

a processor receiving raw signal data from at least one sensor in an environment at different points in time, wherein the raw signal data comprises,

one or more video streams;

environmental data;

electronic thermometer readings; and

one or more audio streams;

the processor analyzing the raw signal data received at each of the different points in time wherein the raw signal data comprises object trajectories;

the processor generating a plurality of metrics derived from the raw signal data for each of the different points in time, wherein the plurality of metrics comprise a time at which an object trajectory crosses a predefined threshold;

the processor analyzing the plurality of metrics to determine changes in the environment over time;

the processor generating an end user view that graphically illustrates,

the changes in the environment over time, wherein the changes comprise changes in occupancy of a venue;

changes in wait time to go through a line;

changes in foot traffic in the environment;

historical information regarding the changes in the environment over time; and

predicted future changes in the environment over time.

2. The method of claim 1, wherein the raw signal data comprises:

wired video streams;

wireless video streams;

wired audio streams;

wireless audio streams; and

radio frequency data.

3. The method of claim 1, wherein the raw signal data comprises one or more video streams, and wherein the processor analyzing the plurality of metrics to determine changes in the environment over time comprises comparing similar metrics from two images of a same scene from two different points in time.

4. The method of claim 3, wherein the similar metrics comprises gray levels.

5. The method of claim 4, wherein the wherein the processor analyzing the plurality of metrics to determine changes in the environment over time further comprises estimating a gray level probability density function for each of the two images.

6. The method of claim 4, wherein the processor analyzing the plurality of metrics to determine changes in the environment over time further comprises using the probability density functions of the two images to estimate generate a measure of normalized mutual information.

7. The method of claim 6, wherein the processor analyzing the plurality of metrics to determine changes in the environment over time further comprises drawing a difference map of the differences between the two images.

8. The method of claim 6, wherein the processor analyzing the plurality of metrics to determine changes in the environment over time further comprises computing multiple difference maps over a sequence of images captured over time.

9. The method of claim 3, wherein the processor analyzing the plurality of metrics to determine changes in the environment over time further comprises comparing similar metrics sequentially over a succession of images captured over time, and further comprises selecting an image as a background image.

10. The method of claim 9, further comprising determining a mask.

11. The method of claim 10, wherein the mask is determined as a difference map between a current image and a current background image.

12. The method of claim 11, further comprising:

the processor determining subsequent masks for subsequent images; and

the processor storing a list of current object tracks, wherein object tracks comprise consecutive masks representing a same object that moves in a video.

13. The method of claim 12, further comprising:

the processor splitting a newly created mask into connected components;

the processor comparing each connected component to existing objects; and

the processor associating a current connected component with the object that best matches it.

14. The method of claim 13, further comprising archiving an object when the object has not been associated with a connected component of a mask for a predetermined period, wherein a trajectory of an archived object represents the trajectory of the object through an image over time.

15. The method of claim 14, further comprising the processor analyzing multiple archived object trajectories to determine what the archived object trajectories represent.

16. The method of claim 15, wherein the analysis of archived object trajectories yields one or more of the plurality of metrics.

17. The method of claim 16, further comprising storing the plurality of metrics in a database and associating each of the plurality of metrics with data regarding its origination.

18. The method of claim 1, wherein analyzing the raw signal data further comprises generating object tracks comprising consecutive masks representing a same moving object in the one or more video streams, wherein a current mask is detected as a difference map between a current frame and a current background frame.

19. The method of claim 18, wherein when a new mask is generated, it is split into connected components, and each connected component is compared to existing objects.

20. The method of claim 19, wherein if a surface of a mask intersects one of several objects in most recently generated masks, a current connected component is associated with an object that best matches the mask.

21. The method of claim 20, wherein if an object has not had added masks for than a predetermined time, the object is considered archived.