Monitoring shelves with pressure and light sensors

• US 10,521,646 B2
• Filed: 03/15/2019
• Issued: 12/31/2019
• Est. Priority Date: 01/10/2018
• Status: Active Grant

• Current Assignee: Trax Technology Solutions Pte. Ltd. (Trax Technologies (SG) Pte Ltd.)
• Original Assignee: Trax Technology Solutions Pte. Ltd. (Trax Technologies (SG) Pte Ltd.)
• Inventors: Adato, Yair, Lishner, Itai
• Primary Examiner(s): Wilder, A. Hunter
• Application Number:
  US16/354,608
• Publication Number:
  US 20190213546A1
• Time in Patent Office: 291 Days
• CPC Class Codes: G06F 16/23 Updating
• CPC Class Codes: G06F 16/235 Update request formulation
• CPC Class Codes: G06F 16/288 Entity relationship models
• CPC Class Codes: G06F 16/55 Clustering; Classification
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06F 16/90335 Query processing
• CPC Class Codes: G06F 17/18 for evaluating statistical data , e.g. average values, frequency distributions, probability functions, regression analysis forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04
• CPC Class Codes: G06F 18/2115 by evaluating different subsets according to an optimisation criterion, e.g. class separability, forward selection or backward elimination
• CPC Class Codes: G06Q 10/06311 Scheduling, planning or task assignment for a person or group
• CPC Class Codes: G06Q 10/063112 Skill-based matching of a person or a group to a task
• CPC Class Codes: G06Q 10/06316 Sequencing of tasks or work
• CPC Class Codes: G06Q 10/08 Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: G06Q 10/0875 Itemisation or classification of parts, supplies or services, e.g. bill of materials
• CPC Class Codes: G06Q 20/203 Inventory monitoring
• CPC Class Codes: G06Q 30/0629 for generating comparisons
• CPC Class Codes: G06Q 30/0639 Item locations
• CPC Class Codes: G06Q 30/0643 Graphical representation of items or shoppers
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30232 : Surveillance
• CPC Class Codes: G06T 2207/30242 : Counting objects in image
• CPC Class Codes: G06T 7/0002 : Inspection of images, e.g. flaw detection
• CPC Class Codes: G06T 7/13 : Edge detection
• CPC Class Codes: G06T 7/20 : Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/521 : from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06T 7/55 : from multiple images
• CPC Class Codes: G06T 7/70 : Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/75 : involving models
• CPC Class Codes: G06V 10/17 : using hand-held instruments
• CPC Class Codes: G06V 20/00 : Scenes; Scene-specific elements control of digital cameras H04N23/60
• CPC Class Codes: G06V 20/20 : in augmented reality scenes
• CPC Class Codes: G06V 20/52 : Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/62 : Text, e.g. of license plates, overlay texts or captions on TV images
• CPC Class Codes: G06V 20/64 : Three-dimensional objects
• CPC Class Codes: G06V 20/68 : Food, e.g. fruit or vegetables
• CPC Class Codes: G06V 30/10 : Character recognition
• CPC Class Codes: G06V 40/10 : Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G07G 1/0036 : Checkout procedures
• CPC Class Codes: G08B 21/18 : Status alarms G08B21/02 takes precedence
• CPC Class Codes: G08B 21/182 : Level alarms, e.g. alarms responsive to variables exceeding a threshold
• CPC Class Codes: H04N 23/51 : Housings
• CPC Class Codes: H04N 23/54 : Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
• CPC Class Codes: H04N 23/60 : Control of cameras or camera modules
• CPC Class Codes: H04N 23/611 : where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/65 : Control of camera operation in relation to power supply
• CPC Class Codes: H04N 23/66 : Remote control of cameras or camera parts, e.g. by remote control devices
• CPC Class Codes: H04N 23/661 : Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/80 : Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/90 : Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

A system for monitoring planogram compliance on a store shelf is disclosed. The system may include a plurality of detection elements and a processor. The processor may be configured to receive first signals from a first subset of detection elements after a plurality of products are placed on the store shelf, and use the first signals to identify at least one pattern associated with a product type of the products. The processor may further be configured to receive from a second subset of detection elements second signals indicative of no products being placed on at least one area of the store shelf, and use the second signals to determine at least one empty space on the store shelf. The processor may also be configured to determine, based on the at least one pattern and the at least one empty space, at least one aspect of planogram compliance.

Claims

1. A sensing device for monitoring planogram compliance on a store shelf, the sensing device comprising:

a plurality of pressure sensors configured to detect placement of products on the store shelf; and

at least one processor configured to;

receive first weight signals indicative of pressure levels from a first subset of pressure sensors from among the plurality of pressure sensors after at least one of a plurality of products is placed on at least one area of the store shelf associated with the first subset of pressure sensors;

identify, using the first weight signals at least one pattern of pressure points associated with a base shape and size of the at least one of the plurality of products;

determine a product type of at least one of the plurality of products by comparing the identified at least one pattern of pressure points to a database of profile patterns;

determine a facing direction of the at least one of the plurality of products by identifying, using the first weight signals, an orientation of the at least one of the plurality of products;

receive second weight signals indicative of pressure levels from a second subset of pressure sensors from among the plurality of pressure sensors, the second weight signals being equal to or within a threshold of a default atmospheric pressure value indicative of an absence of a product on at least one area of the store shelf associated with the second subset of pressure sensors;

determine, using the second weight signals, at least one empty space on the store shelf;

determine, using the first and second weight signals, a location and quantity of the at least one of the plurality of products on the store shelf;

determine, based on the determined product type, facing direction, location and quantity, at least one aspect of planogram compliance; and

transmit an indicator of the at least one aspect of planogram compliance to a remote server.

2. The sensing device of claim 1, wherein the at least one processor is further configured to use the first weight signals to identify at least one product of a first type using a first pattern of pressure points and at least one product of a second type using a second pattern of pressure points.

3. The sensing device of claim 2, wherein a second at least one aspect of planogram compliance includes product homogeneity, and the at least one processor is further configured to count occurrences where a product of the second type is placed on an area of the store shelf associated with the first type of product.

4. The sensing device of claim 1, wherein the at least one processor is further configured to determine, based on product dimension data for the determined product type stored in the database, a number of additional products of the determined product type that can be placed on the at least one area of the store shelf associated with the second subset of pressure sensors.

5. The sensing device of claim 4, wherein a second at least one aspect of planogram compliance includes a restocking rate, and the at least one processor is further configured to use the second weight signals to determine the restocking rate based on a sensed rate at which products are added to the at least one area of the store shelf associated with the second subset of pressure sensors.

6. The sensing device of claim 4, wherein the at least one processor is further configured to determine the facing direction based on a number of products determined to be placed on a selected area of the store shelf at a front of the store shelf.

7. The sensing device of claim 1, wherein the plurality of pressure sensors are embedded into a fabric configured to be positioned on the store shelf.

8. The sensing device of claim 1, wherein the plurality of pressure sensors are configured to be integrated with the store shelf.

9. The sensing device of claim 1, wherein the at least one processor is further configured to receive the first weight signals from the first subset of pressure sensors as the plurality of products are placed above the first subset of pressure sensors.

10. The sensing device of claim 1, wherein the plurality of pressure sensors are substantially uniformly distributed across the store shelf.

11. The sensing device of claim 1, wherein the plurality of pressure sensors are distributed relative to the store shelf such that a first area of the store shelf has a higher density of pressure sensors than a second area of the store shelf.

12. The sensing device of claim 1, wherein the at least one processor R further configured to identify a change in at least one characteristic, associated with one or more of the first weight signals, and in response to the identified change, trigger an acquisition of at least one image of the store shelf.

13. The sensing device of claim 12, wherein the change in at least one characteristic associated with one or more of the first weight signals is indicative of removal of at least one product from a location associated with the at least one area of the store shelf associated with the first subset of pressure sensors.

14. The sensing device of claim 1, wherein the at least one processor is further configured to:

determine a change in at least one characteristic associated with one or more of the first weight signals;

determine a type of event associated with the change;

trigger an acquisition of at least one image of the store shelf when the change is associated with a first event type; and

forgo the acquisition of at least one image of the store shelf when the change is associated with a second event type.

15. The sensing device of claim 1, wherein the at least one processor is further configured to identify a change in at least one characteristic associated with one or more of the first weight signals;

and in response to the identified change, trigger a product-related task For an employee of a retail store.

16. A method for monitoring planogram compliance on a store shelf, the method comprising:

receiving, by at least one processor, first weight signals indicative of pressure levels from a first subset of a plurality of pressure sensors after at least one of a plurality of products is placed on at least one area of the store shelf associated with the first subset of pressure sensors;

identifying, by the at least one processor and using the first weight signals, at least one pattern of pressure points associated with a base shape and size of the at least one of the plurality of products;

determining, by the at least one processor, a product type of the at least one of the plurality of products by comparing the identified at least one pattern of pressure points to a database of profile patterns;

determining, by the at least one processor, a facing direction of the at least one of the plurality of products by identifying, using the first weight signals, an orientation of the at least one of the plurality of products;

receiving, by the at least one processor, second weight signals indicative of pressure levels from a second subset of the plurality of pressure sensors, the second weight signals being equal to or within a threshold of a default atmospheric pressure value indicative of an absence of a product on at least one area of the store shelf associated with the second subset of pressure sensors;

determining, by the at least one processor and using the second weight signals, at least one empty space on the store shelf;

determining, by the at least one processor and using the first and second weight signals, a location and quantity of the at least one of the plurality of products on the store shelf;

determining, by the at least one processor and based on the determined product type, facing direction, location and quantity, at least one aspect of planogram compliance; and

transmitting, by at least one processor, an indicator of the at least one aspect of planogram compliance to a remote server.

17. A computer program product for monitoring planogram compliance a store shelf embodied in a non-transitory computer-readable medium and executable by at least one processor, the computer program product including instructions for causing the at least one processor to execute a method comprising:

receiving first weight signals indicative of pressure levels from a first subset of a plurality of pressure sensors after at least one of a plurality of products is placed on at least one area of the store shelf associated with the first subset of pressure sensors;

identifying, using the first weight signals, at least one pattern of pressure points associated with a base shape and size of the at least one of the plurality of products;

determining a product type of the at least one of the plurality of products by comparing the identified at least one pattern of pressure points to a database of profile patterns;

determining a facing direction of the at least one of the plurality of products by identifying, using the first weight signals, an orientation of the at least one of the plurality of products;

receiving second weight signals indicative of pressure levels from a second subset of the plurality of pressure sensors, the second weight signals being equal to or within a threshold of a default atmospheric pressure value indicative of an absence of a product on at least one area of the store shelf associated with the second subset of pressure sensors;

determining, using the second weight signals, at least one empty space on the store shelf;

determining, using the first and second weight signals, a location and quantity of the at least one of the plurality of products on the store shelf;

determining, based on the determined product type, facing direction, location and quantity, at least one aspect of planogram compliance; and

transmitting an indicator of the at least one aspect of planogram compliance to a remote server.