Camera configured to be mounted to store shelf

• US 10,643,060 B2
• Filed: 03/15/2019
• Issued: 05/05/2020
• Est. Priority Date: 01/10/2018
• Status: Active Grant

• Current Assignee: Trax Technology Solutions Pte. Ltd. (Trax Technologies (SG) Pte Ltd.)
• Original Assignee: Trax Technology Solutions Pte. Ltd. (Trax Technologies (SG) Pte Ltd.)
• Inventors: Adato, Yair, Lishner, Itai, Cohen, Daniel Shimon
• Primary Examiner(s): Billah, Masum
• Application Number:
  US16/354,652
• Publication Number:
  US 20190215424A1
• Time in Patent Office: 417 Days
• Field of Search: 348150
• CPC Class Codes: G06F 16/23 Updating
• CPC Class Codes: G06F 16/235 Update request formulation
• CPC Class Codes: G06F 16/288 Entity relationship models
• CPC Class Codes: G06F 16/55 Clustering; Classification
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06F 16/90335 Query processing
• CPC Class Codes: G06F 17/18 for evaluating statistical data , e.g. average values, frequency distributions, probability functions, regression analysis forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04
• CPC Class Codes: G06F 18/2115 by evaluating different subsets according to an optimisation criterion, e.g. class separability, forward selection or backward elimination
• CPC Class Codes: G06Q 10/06311 Scheduling, planning or task assignment for a person or group
• CPC Class Codes: G06Q 10/063112 Skill-based matching of a person or a group to a task
• CPC Class Codes: G06Q 10/06316 Sequencing of tasks or work
• CPC Class Codes: G06Q 10/08 Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: G06Q 10/0875 Itemisation or classification of parts, supplies or services, e.g. bill of materials
• CPC Class Codes: G06Q 20/203 Inventory monitoring
• CPC Class Codes: G06Q 30/0629 for generating comparisons
• CPC Class Codes: G06Q 30/0639 Item locations
• CPC Class Codes: G06Q 30/0643 Graphical representation of items or shoppers
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30232 : Surveillance
• CPC Class Codes: G06T 2207/30242 : Counting objects in image
• CPC Class Codes: G06T 7/0002 : Inspection of images, e.g. flaw detection
• CPC Class Codes: G06T 7/13 : Edge detection
• CPC Class Codes: G06T 7/20 : Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/521 : from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06T 7/55 : from multiple images
• CPC Class Codes: G06T 7/70 : Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/75 : involving models
• CPC Class Codes: G06V 10/17 : using hand-held instruments
• CPC Class Codes: G06V 20/00 : Scenes; Scene-specific elements control of digital cameras H04N23/60
• CPC Class Codes: G06V 20/20 : in augmented reality scenes
• CPC Class Codes: G06V 20/52 : Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/62 : Text, e.g. of license plates, overlay texts or captions on TV images
• CPC Class Codes: G06V 20/64 : Three-dimensional objects
• CPC Class Codes: G06V 20/68 : Food, e.g. fruit or vegetables
• CPC Class Codes: G06V 30/10 : Character recognition
• CPC Class Codes: G06V 40/10 : Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G07G 1/0036 : Checkout procedures
• CPC Class Codes: G08B 21/18 : Status alarms G08B21/02 takes precedence
• CPC Class Codes: G08B 21/182 : Level alarms, e.g. alarms responsive to variables exceeding a threshold
• CPC Class Codes: H04N 23/51 : Housings
• CPC Class Codes: H04N 23/54 : Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
• CPC Class Codes: H04N 23/60 : Control of cameras or camera modules
• CPC Class Codes: H04N 23/611 : where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/65 : Control of camera operation in relation to power supply
• CPC Class Codes: H04N 23/66 : Remote control of cameras or camera parts, e.g. by remote control devices
• CPC Class Codes: H04N 23/661 : Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/80 : Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/90 : Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

A system for acquiring images of products in a retail store is disclosed. The system may include a first housing configured for location on a retail shelving unit, and an image capture device included in the first housing and configured relative to the first housing such that an optical axis of the image capture device is directed toward an opposing retail shelving unit when the first housing is fixedly mounted on the retail shelving unit. The system may further include a second housing configured for location on the retail shelving unit separate from the first housing, the second housing containing a processor configured to control the image capture device and a network interface for communicating with a remote server. The system may further include a data conduit configured to enable transfer of control signals from the processor to the image capture device.

Claims

1. A system for acquiring images of products in a retail store, the system comprising:

at least one first housing configured for location on a first side of a retail shelving unit;

at least one image capture device included in the at least one first housing and configured relative to the at least one first housing such that an optical axis of the at least one image capture device is directed toward an opposing retail shelving unit when the at least one first housing is fixedly mounted on the retail shelving unit;

a second housing configured for location on a second side of the retail shelving unit separate from the at least one first housing, the second housing containing at least one processor configured to control the at least one image capture device and also to control a network interface for communicating with a remote server; and

at least one data conduit extending between the at least one first housing and the second housing, the at least one data conduit being configured to enable transfer of control signals from the at least one processor to the at least one image capture device and to enable collection of image data acquired by the at least one image capture device for transmission by the network interface, wherein;

the at least one processor is further configured to enable real-time display of a field of view of the at least one image capture device on a handheld device of a user installing the image capturing device, and

the real-time display of the field of view of the at least one image capture device includes augmented markings indicating;

a location of a field of view of an adjacent image capture device,

a region of interest in the opposing retail shelving unit, and

a misplaced product.

2. The system of claim 1, wherein the at least one processor is further configured to detect an individual in a selected area between the retail shelving unit and the opposing retail shelving unit, and to cause the at least one image capture device to forgo image acquisition while the individual is within the selected area.

3. The system of claim 1, wherein the at least one processor is further configured to:

analyze a plurality of images acquired by the at least one image capture device;

identify at least one of the plurality of images that includes a representation of at least a portion of an individual located in an area between the retail shelving unit and the opposing retail shelving unit; and

avoid transmission of the at least one identified image to the remote server.

4. The system of claim 1, wherein the at least one processor is further configured to:

receive an output signal from at least one sensor located on the opposing retail shelving unit, the output signal being indicative of a sensed lifting of a product from the opposing retail shelving unit; and

cause the at least one image capture device to capture an image of the opposing retail shelving unit in response to receiving the output signal from the at least one sensor located on the opposing retail shelving unit.

5. The system of claim 1, wherein the at least one processor is further configured to cause the at least one image capture device to periodically capture images of products located on the opposing retail shelving unit.

6. The system of claim 1, wherein the retail shelving unit includes two adjacent horizontal shelves forming a continuous surface for product placement, and wherein the at least one first housing is configured to be fixedly mounted on the retail shelving unit in a slit between the two adjacent horizontal shelves.

7. The system of claim 1, wherein the retail shelving unit includes two adjacent horizontal shelves forming a continuous surface for product placement, and wherein the at least one first housing is configured to be fixedly mounted on a first shelf and the second housing is configured to be fixedly mounted on a second shelf.

8. The system of claim 1, wherein the retail shelving unit includes a horizontal shelf with at least one designated place for mounting a housing of an image capturing device, the at least one designated place is associated with connectors;

and wherein the at least one first housing is configured to be fixedly mounted on a side of the horizontal shelf facing the opposing retail shelving unit using the connectors.

9. The system of claim 1, wherein the at least one first housing is configured to be fixedly mounted on the retail shelving unit on a first side of a horizontal shelf facing the opposing retail shelving unit.

10. The system of claim 9, wherein the second housing is configured to be fixedly mounted on the retail shelving unit on a second side of the horizontal shelf orthogonal to the first side.

11. The system of claim 10, wherein the retail shelving unit includes a third side of the horizontal shelf parallel to the first side, and the second housing is further configured to be fixedly mounted on the retail shelving unit on the second side closer to the third side than to the first side.

12. The system of claim 11, wherein the at least one data conduit includes flexible printed circuits and has an adjustable length for extending the at least one data conduit between the at least one first housing and the second housing when the at least one first housing is fixedly mounted on the first side and the second housing is fixedly mounted on the second side closer to the third side than to the first side.

13. The system of claim 1, wherein the at least one data conduit is configured for banding around an edge of a horizontal shelf of the retail shelving unit.

14. The system of claim 1, wherein the retail shelving unit includes a horizontal shelf and the at least one data conduit is configured for location inside an envelope of the horizontal shelf.

15. The system of claim 1, wherein the at least one image capture device includes a plurality of image capture devices, and wherein the at least one processor contained in the second housing is further configured to control the plurality of image capture devices, wherein each of the plurality of image capture devices has a field of view that at least partially overlaps with a field of view of at least one other image capture device from among the plurality of image capture devices.

16. The system of claim 15, wherein the at least one processor is further configured to control the plurality of image capture devices such that each of the plurality of image capture devices captures an image at a different time.

17. The system of claim 15, wherein the plurality of image capture devices includes at least a first image capture device and a second image capture device configured for location on a single horizontal shelf, and wherein image data acquired by the first image capture device and the second image capture device is configured to enable a calculation of depth information associated with at least one product positioned on the opposing retail shelving unit.

18. The system of claim 17, wherein a first distance between the first image capture device and the second image capture device is selected based on a second distance between the retail unit and the opposing retail shelving unit, wherein the first distance is at least 10 cm.

19. The system of claim 1, wherein the at least one processor is further configured to modify the image data acquired by the at least one image capture device based on at least one attribute associated with the opposing retail shelving unit.

20. The system of claim 1, wherein the at least one image capture device includes a plurality of image capturing devices configured for location on the retail shelving unit, wherein at least some of the plurality of image capturing devices are configured to be fixedly mounted on shelves at different heights.

21. The system of claim 1, wherein the at least one first housing includes at least one lens having a fixed focal length corresponding to a distance between the retail shelving unit and the opposing retail shelving unit, and wherein the at least one lens is associated with a fixed focal length having a fixed value between 2.5 mm and 4.5 mm.

22. The system of claim 1, wherein the at least one processor is further configured to control a projector and an image capture device configured for location on a single horizontal shelf, and wherein image data acquired by the image capture device includes reflections of light from the projector and enables a calculation of depth information associated with at least one product positioned on the opposing retail shelving unit.

23. The system of claim 1, wherein the second housing includes a power port for conveying energy from a power source in the second housing to the at least one first housing.

24. The system of claim 1, wherein the at least one first housing includes a field of view adjustment mechanism for setting the field of view of the at least one image capture device such that the field of view will at least partially encompass products placed both on a bottom shelf of the opposing retail shelving unit and on a top shelf the opposing retail shelving unit.

25. The system of claim 1, wherein an anti-theft device is located in at least one of the at least one first housing and the second housing.

26. The system of claim 1, wherein the at least one image capture device includes an image sensor having sufficient resolution to enable detection of text associated with labels on the opposing retail shelving unit.

27. A method for acquiring images of products in a retail store, the method comprising:

fixedly mounting on a first side of a retail shelving unit at least one first housing containing at least one image capture device such that an optical axis of the at least one image capture device is directed to an opposing retail shelving unit;

fixedly mounting on a second side of the retail shelving unit a second housing at a location spaced apart from the at least one first housing, the second housing containing at least one processor;

extending at least one data conduit between the at least one first housing and the second housing;

capturing images of the opposing retail shelving unit using the at least one image capture device;

transmitting at least some of the captured images from the second housing to a remote server configured to determine planogram compliance relative to the opposing retail shelving unit; and

enabling real-time display of a field of view of the at least one image capture device on a handheld device of a user installing the image capturing device, wherein the real-time display of the field of view of the at least one image capture device includes augmented markings indicating;

a location of a field of view of an adjacent image capture device,

a region of interest in the opposing retail shelving unit, and

a misplaced product.