MONITORING DEVICE, MONITORING SYSTEM, AND MONITORING METHOD

• US 20170295953A1
• Filed: 10/27/2015
• Published: 10/19/2017
• Est. Priority Date: 11/26/2014
• Status: Active Grant

• Current Assignee: Panasonic Intellectual Property Management Co., Ltd. (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Intellectual Property Management Co., Ltd. (Panasonic Holdings Corporation)
• Inventors: SAKATA, Yoshifumi, TAKEMOTO, Yuichiro, IWAI, Kazuhiko
• Granted Patent: US 10,334,965 B2
• Time in Patent Office: Days
• CPC Class Codes: A47F 10/02 for self-service type systems, e.g. supermarkets A47F10/06 takes precedence; hand-carts B62B; baskets B65D
• CPC Class Codes: A47F 2010/025 using stock management systems
• CPC Class Codes: A47F 3/002 Devices for protection against sunlight or theft
• CPC Class Codes: A47F 5/00 Show stands, hangers, or shelves characterised by their constructional features
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: G06Q 30/06 Buying, selling or leasing transactions
• CPC Class Codes: G06V 10/761 Proximity, similarity or dissimilarity measures
• CPC Class Codes: G06V 20/44 Event detection
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

A monitoring device includes a fixed area setter that sets an area of a part of a display shelf to be monitored as a fixed area, based on a captured image of the display shelf in a store, a shelf detector that detects the display shelf, by determining a similarity between an image of the fixed area set by the fixed area setter and an image at the same position as the fixed area, from the captured image of the store, and an alert notifier that notifies a user that the display shelf to be monitored is not detected, when it is determined that there is no similarity.

Claims

1. A monitoring device comprising:

a fixed area setter that sets an area of a part of a display shelf to be monitored as a fixed area, based on a captured image of the display shelf in a store;

a shelf detector that detects the display shelf, by determining a similarity between an image of the fixed area set by the fixed area setter and an image at the same position as the fixed area, from a captured image of the store; and

an alert notifier that notifies a user that the display shelf to be monitored is not detected, when it is determined that there is no similarity.

2. The monitoring device of claim 1,

wherein the shelf detector extracts an area without a change over a predetermined period as a fixed area candidate, from the captured image of the store, and

wherein the fixed area setter sets an area selected by the user as the fixed area, from among the extracted fixed area candidates.

3. The monitoring device of claim 1,

wherein the fixed area setter stores position information of the area set as the fixed area, a captured image of the display shelf at a setting time, and an identifier of the display shelf, in a storage.

4. The monitoring device of claim 1,

wherein the fixed area setter sets a plurality of areas as a plurality of the fixed areas,

wherein the shelf detector determines similarity between an image at each of the plurality of fixed area and an image at the same position as each of the plurality of fixed areas, and

wherein the alert notifier notifies the user that the display shelf is not detected when it is determined that at least one of the plurality of fixed areas is not similar.

5. The monitoring device of claim 1,

wherein the shelf detector determines similarity by comparing brightness of the image of the fixed area with brightness of the image at the same position as the fixed area.

6. The monitoring device of claim 1, further comprising:

an automatic tracker that when the shelf detector determines that an image of the fixed area is not similar to an image at the same position as the fixed area, searches for a similar image which is similar to the image of the fixed area, from the captured image of the store, and controls an angle of view of a camera which is capturing an image such that the position of the searched similar image is the position of the fixed area,

wherein the alert notifier notifies the user that the display shelf is not detected when the similar image cannot be searched.

8. The monitoring device of claim 1, further comprising:

a goods detector that detects goods from the display shelf detected by the shelf detector; and

a display screen generator that generates a monitoring screen for displaying the detection result of goods and the captured image of the display shelf on a monitor.

7. A monitoring device comprising:

a shelf detector that detects a display shelf, by searching for a marking image which is attached to the display shelf and is uniquely identifiable, from a captured image of the display shelf in a store, and determining whether or not a position of the searched marking image matches an initial position of the marking image which is stored in advance;

an automatic tracker that controls an angle of view of a camera which is capturing an image such that the position of the marking image is the initial position, when the shelf detector determines that the position of the marking image does not match the initial position; and

an alert notifier that notifies a user that the display shelf is not detected, when the shelf detector is not able to search for the marking image.

9. A monitoring system which monitors a display shelf, based on a captured image of the display shelf in a store, comprising:

a camera that captures the display shelf; and

an information processing device,

wherein the information processing device includes

a fixed area setter that sets an area of a part of the display shelf to be monitored as a fixed area, based on a captured image of the display shelf in the store;

a shelf detector that detects the display shelf, by determining a similarity between an image of the fixed area set by the fixed area setter and an image at the same position as the fixed area, from the captured image of the store, and

an alert notifier that notifies a user that the display shelf to be monitored is not detected, when it is determined that there is no similarity.

10. A monitoring method causing an information processing device to implement a process of monitoring a display shelf, based on a captured image of the display shelf in a store, comprising:

setting an area of a part of the display shelf to be monitored as a fixed area, from the captured image of the display shelf; and

determining a similarity between an image of the set fixed area and an image at the same position as the fixed area, from a captured image of the store, and notifying a user that the display shelf to be monitored is not detected, when it is determined that there is no similarity.