SYSTEM AND METHOD FOR IMPROVING SITE OPERATIONS BY DETECTING ABNORMALITIES

• US 20130027561A1
• Filed: 07/29/2011
• Published: 01/31/2013
• Est. Priority Date: 07/29/2011
• Status: Abandoned Application

• Current Assignee: Panasonic Intellectual Property Management Co., Ltd. (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Inventors: OZDEMIR, Hasan Timucin, LIU, Lipin, LEE, Kuo-Chu, MIWA, Michio, YU, Jannite
• Application Number:
  US13/194,010
• Publication Number:
  US 20130027561A1
• Time in Patent Office: Days
• US Class Current: 348/150
• CPC Class Codes: G06Q 10/06 Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
• CPC Class Codes: G06Q 10/06311 Scheduling, planning or task assignment for a person or group
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06V 40/174 Facial expression recognition
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

A system for improving site operations by detecting abnormalities includes a first sensor abnormality detector connected to a first sensor and configured to learn a first normal behavior sequence, a second sensor abnormality detector connected to a second sensor and configured to learn a second normal behavior sequence, an abnormality correlation server configured to receive abnormally scored first sensor data and abnormally scored second sensor data, the abnormality correlation server further configured to correlate the received abnormally scored first sensor data and abnormally scored second sensor data sensed at the same time by the first and second sensors and determine an abnormal event; and an abnormality report generator configured to generate an abnormality report based on the correlated the received abnormally scored first sensor data and abnormally scored second sensor data.

Claims

1. A system for improving site operations by detecting abnormalities, comprising:

a first sensor;

a first sensor abnormality detector connected to the first sensor, and configured to learn a first normal behavior sequence based on detected data sent from the first sensor, the first sensor abnormality detector comprising a first scorer configured to assign a normal score to first sensor data corresponding to the learned normal behavior sequence and an abnormal score to first sensor data having a value outside of the value of the first sensor data corresponding to the learned normal behavior sequence;

a second sensor;

a second sensor abnormality detector connected to the second sensor, and configured to learn a second normal behavior sequence based on detected data sent from the second sensor, the second sensor abnormality detector comprising a second scorer configured to assign a normal score to second sensor data corresponding to the learned normal behavior sequence and an abnormal score to second sensor data having a value outside of the value of the second sensor data corresponding to the learned normal behavior sequence;

an abnormality correlation server configured to receive abnormally scored first sensor data and abnormally scored second sensor data, the abnormality correlation server further configured to correlate the received abnormally scored first sensor data and abnormally scored second sensor data sensed at the same time by the first and second sensors and determine an abnormal event; and

an abnormality report generator configured to generate an abnormality report based on the correlated received abnormally scored first sensor data and abnormally scored second sensor data.

2. The system according to claim 1, wherein the first sensor and the second sensor are different sensor types and generate different types of data.

3. The system according to claim 2, wherein at least one of the first sensor and the second sensor is a video camera.

4. The system according to claim 1, wherein:

at least one of the first sensor abnormality detector and the second sensor abnormality detector comprises a memory configured to records sensor data, the recorded sensor data comprising distribution of sensor variables and metadata of event frequency; and

the at least one of the first sensor abnormality detector and the second sensor abnormality detector is configured to detect a change of the distribution and a change of the metadata over time.

5. The system according to claim 1, further comprising a protocol adapter positioned between the first and second sensors and the first and second sensor abnormality detectors.

6. The system according to claim 1, further comprising an intervention detector connected to the abnormality correlation server and configured to detect whether an abnormal event has been acknowledged by an entity external to the system.

7. The system according to claim 1, further comprising a pager connected to the abnormality report generator and configured to send an alert to a user when the abnormality report is generated.

8. At least one non-transitory computer-readable medium readable by a computer for improving site operations by detecting abnormalities, the at least one non-transitory computer-readable medium comprising:

a first sensor abnormality detecting code segment that, when executed, learns a first normal behavior sequence based on detected data sent from a first sensor, the first sensor abnormality detecting code segment comprising a first scoring code segment configured to assign a normal score to first sensor data corresponding to the learned first normal behavior sequence and an abnormal score to first sensor data having a value outside of the value of the first sensor data corresponding to the learned first normal behavior sequence;

a second sensor abnormality detecting code segment that, when executed, learns a second normal behavior sequence based on detected data sent from a second sensor, the second sensor abnormality detecting code segment comprising a second scoring code segment configured to assign a normal score to second sensor data corresponding to the learned second normal behavior sequence and an abnormal score to second sensor data having a value outside of the value of the second sensor data corresponding to the learned second normal behavior sequence;

an abnormality correlation code segment that, when executed, receives abnormally scored first sensor data and abnormally scored second sensor data, the abnormality correlation code segment further configured to correlate the received abnormally scored first sensor data and abnormally scored second sensor data sensed at the same time by the first and second sensors and determine an abnormal event; and

an abnormality report generating code segment that, when executed, generates an abnormality report based on the correlated the received abnormally scored first sensor data and abnormally scored second sensor data.

9. The at least one non-transitory computer-readable medium according to claim 8, wherein the first and second sensors are different types.

10. The at least one non-transitory computer-readable medium according to claim 9, wherein at least one of the first and second sensors is a video camera.

11. The at least one non-transitory computer-readable medium according to claim 8, wherein:

at least one of the first sensor abnormality detecting code segment and the second sensor abnormality detecting code segment, that when executed, actuates a memory configured to record sensor data, the recorded sensor data comprising distribution of sensor variables and metadata of event frequency; and

the at least one of the first sensor abnormality detecting code segment and the second sensor abnormality detecting code segment, when executed, detects a change of the distribution and a change of the metadata over time.

12. The at least one non-transitory computer-readable medium according to claim 8, further comprising an intervention detecting code segment that, when executed, detects whether an abnormal event has been acknowledged by an external entity.

13. The at least one non-transitory computer-readable medium according to claim 8, further comprising a paging code segment that, when executed, sends an alert to a user when the abnormality report is generated.

14. A method for improving site operations by detecting abnormalities, comprising:

learning a first normal behavior sequence based on detected data sent from a first sensor;

assigning a normal score to first sensor data corresponding to the learned normal behavior sequence and an abnormal score to first sensor data having a value outside of the value of the first sensor data corresponding to the learned first normal behavior sequence;

learning a second normal behavior sequence based on detected data sent from a second sensor;

assigning a normal score to second sensor data corresponding to the learned normal behavior sequence and an abnormal score to second sensor data having a value outside of the value of the second sensor data corresponding to the learned second normal behavior sequence;

receiving abnormally scored first sensor data and abnormally scored second sensor data;

correlating the received abnormally scored first sensor data and the received abnormally scored second sensor data sensed at a same time by the first and second sensors and determining an abnormal event; and

generating an abnormality report based on the correlated received abnormally scored first sensor data and the abnormally scored second sensor data.

15. The method of claim 14, wherein the first and second sensors are positioned at different regions of the site.

16. A method of processing an order from a mobile device, the method comprising:

detecting at least one nearest facility based on a location of the mobile device;

communicating the detected at least one more nearest facility to a user;

selecting a detected facility of the at least one nearest facility;

selecting at least one item from items available for purchase at the selected detected facility;

sending an order for the at least one item to a site for order processing; and

receiving a confirmation of the ordered at least one item.

17. The method of claim 16, further comprising sending payment for the one or more items.

18. A method of verifying an identity of a customer picking up an order at a site, the method comprising:

receiving an order from a mobile device, the order including customer identification data;

generating an order confirmation for the customer; and

associating the customer identification data with the order confirmation.

19. The method of verifying an identity of a customer picking up an order at a site of claim 18, wherein the customer identification data includes vehicle tag data, the method further comprising:

detecting the vehicle tag data upon arrival of a vehicle of the customer at the site;

determining a sequence of vehicles arriving at the site; and

preparing customer orders corresponding to the sequence of the vehicles arriving at the site.

20. The method of verifying the identity of a customer picking up an order at a site of claim 18, further comprising:

obtaining a location of the customer;

estimating a time of arrival of the customer; and

preparing the order based on the estimated time of arrival of the customer.

21. The method of verifying an identity of a customer picking up an order at a site of claim 18, further comprising:

sending an image of a worker of the site to the customer; and

routing the customer to the worker corresponding to the sent image upon the customer'"'"'s arrival at the site.

22. A method for preventing merchandise loss at a site, the method comprising:

storing video recordings of a plurality of videos, each video of the plurality of videos including video images and metadata of the video image, the metadata including data corresponding to a face value of a unique face;

comparing face values of the plurality of videos;

obtaining a degree of correlation between a face value of one video of the plurality of videos and a face value of another video of the plurality of videos; and

generating a report when a predetermined correlation threshold is reached between the one video and the another video.

23. The method for preventing merchandise loss at a site according to claim 22,

wherein the metadata further includes at least one of video recording time interval and camera field of view, the method further comprising comparing the at least one video recording time interval and camera field of view to obtain a composite value;

and

obtaining a degree of correlation between composite values of the one video of the plurality of videos and composite values of the another video of the plurality of videos.

24. A method of managing a workforce at a site, the method comprising:

monitoring the location of at least one employee at the site;

monitoring the location of at least one customer at the site;

determining a positional relationship between the at least one employee and the at least one customer;

determining that the at least one customer is being assisted by the at least one employee when the determined positional relationship is within a predetermined value range;

determining that the at least one customer is not being assisted by the at least one employee when the determined positional relationship is outside of the predetermined value range; and

generating a report when the determined positional relationship is outside of the predetermined value range.

25. The method of managing a workforce at a site of claim 24, wherein said monitoring a location of at least one customer at the site comprises monitoring locations of a plurality of customers, the method further comprising determining a period of time each customer is not assisted by the at least one employee.

26. The method of managing a workforce at a site of claim 24, wherein said monitoring a location of at least one customer at the site comprises monitoring locations of a plurality of customers, the method further comprising determining a site arrival time of each customer that is not being assisted by the at least one employee.

27. A method of determining an identity of a customer at a site, the method comprising:

detecting, using at least one video imager, a unique customer based on a customer face at the site based on face data corresponding to a face value of a unique face;

obtaining unique customer data at a point of sale terminal of the site, the unique customer data including at least customer name and previously stored face data; and

comparing the detected face data with the previously stored face data and determining whether the identity of the unique customer corresponds to the unique customer data.