Identifying bottle sizes based on bottle proportions

• US 10,546,182 B2
• Filed: 03/15/2019
• Issued: 01/28/2020
• Est. Priority Date: 01/10/2018
• Status: Active Grant

• Current Assignee: Trax Technology Solutions Pte. Ltd. (Trax Technologies (SG) Pte Ltd.)
• Original Assignee: Trax Technology Solutions Pte. Ltd. (Trax Technologies (SG) Pte Ltd.)
• Inventors: Adato, Yair, Michael, Yotam, Adar, Yonatan, Kushnir, Maria, Yashpe, Dror
• Primary Examiner(s): Liew, Alex Kok S
• Application Number:
  US16/354,672
• Publication Number:
  US 20190213752A1
• Time in Patent Office: 319 Days
• CPC Class Codes: G06F 16/23 Updating
• CPC Class Codes: G06F 16/235 Update request formulation
• CPC Class Codes: G06F 16/288 Entity relationship models
• CPC Class Codes: G06F 16/55 Clustering; Classification
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06F 16/90335 Query processing
• CPC Class Codes: G06F 17/18 for evaluating statistical data , e.g. average values, frequency distributions, probability functions, regression analysis forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04
• CPC Class Codes: G06F 18/2115 by evaluating different subsets according to an optimisation criterion, e.g. class separability, forward selection or backward elimination
• CPC Class Codes: G06Q 10/06311 Scheduling, planning or task assignment for a person or group
• CPC Class Codes: G06Q 10/063112 Skill-based matching of a person or a group to a task
• CPC Class Codes: G06Q 10/06316 Sequencing of tasks or work
• CPC Class Codes: G06Q 10/08 Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: G06Q 10/0875 Itemisation or classification of parts, supplies or services, e.g. bill of materials
• CPC Class Codes: G06Q 20/203 Inventory monitoring
• CPC Class Codes: G06Q 30/0629 for generating comparisons
• CPC Class Codes: G06Q 30/0639 Item locations
• CPC Class Codes: G06Q 30/0643 Graphical representation of items or shoppers
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30232 : Surveillance
• CPC Class Codes: G06T 2207/30242 : Counting objects in image
• CPC Class Codes: G06T 7/0002 : Inspection of images, e.g. flaw detection
• CPC Class Codes: G06T 7/13 : Edge detection
• CPC Class Codes: G06T 7/20 : Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/521 : from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06T 7/55 : from multiple images
• CPC Class Codes: G06T 7/70 : Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/75 : involving models
• CPC Class Codes: G06V 10/17 : using hand-held instruments
• CPC Class Codes: G06V 20/00 : Scenes; Scene-specific elements control of digital cameras H04N23/60
• CPC Class Codes: G06V 20/20 : in augmented reality scenes
• CPC Class Codes: G06V 20/52 : Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/62 : Text, e.g. of license plates, overlay texts or captions on TV images
• CPC Class Codes: G06V 20/64 : Three-dimensional objects
• CPC Class Codes: G06V 20/68 : Food, e.g. fruit or vegetables
• CPC Class Codes: G06V 30/10 : Character recognition
• CPC Class Codes: G06V 40/10 : Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G07G 1/0036 : Checkout procedures
• CPC Class Codes: G08B 21/18 : Status alarms G08B21/02 takes precedence
• CPC Class Codes: G08B 21/182 : Level alarms, e.g. alarms responsive to variables exceeding a threshold
• CPC Class Codes: H04N 23/51 : Housings
• CPC Class Codes: H04N 23/54 : Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
• CPC Class Codes: H04N 23/60 : Control of cameras or camera modules
• CPC Class Codes: H04N 23/611 : where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/65 : Control of camera operation in relation to power supply
• CPC Class Codes: H04N 23/66 : Remote control of cameras or camera parts, e.g. by remote control devices
• CPC Class Codes: H04N 23/661 : Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/80 : Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/90 : Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

A system for processing images captured in a retail store is provided. The system may include at least one processor configured to receive an image depicting a store shelf having at least one bottle displayed thereon and analyze the image to detect a representation in the image of the at least one bottle. The at least one bottle may have an outline design. The at least one processor is also configured to identify in the image two outline elements being associated with the outline design of the at least one bottle. Each of the two outline elements may have a specific length. The at least one processor may further be configured to determine a size of the at least one bottle based on a comparison of the lengths of the two outline elements.

Claims

1. A system for processing images captured in a retail store, the system comprising:

at least one processor configured to;

receive an image depicting a store shelf having a first product comprising at least one bottle and a second product displayed thereon, a type of the first product being different from a type of the second product;

analyze the image to detect a representation in the image of the at least one bottle, wherein the at least one bottle has an outline design;

identify in the image two outline elements being associated with the outline design of the at least one bottle, wherein each of the two outline elements has a specific length;

determine a size of the at least one bottle of the first product based on a comparison of the lengths of the two outline elements; and

determine, based on the determined sized of the at least one bottle of the first product, a type of the second product.

2. The system of claim 1, wherein the two outline elements are selected from a group including:

a top of a bottle, an orifice of the bottle, a lip of the bottle, a collar of the bottle, a neck of the bottle, a heel of the bottle, and a base of the bottle.

3. The system of claim 1, wherein the at least one processor is further configured to determine the size of a bottle when at least 50% of the outline design of the bottle is obscured.

4. The system of claim 1, wherein the at least one processor is further configured to determine a product type of the at least one bottle based on the determined size.

5. The system of claim 1, wherein the at least one processor is further configured to:

analyze the image to obtain a distribution of lengths of a group of outline elements, the group of outline elements comprising at least the two outline elements associated with the outline design of the at least one bottle; and

use a property of the distribution of the lengths of the group of outline elements to determine the size of the at least one bottle.

6. The system of claim 1, wherein the at least one processor is further configured to:

determine a confidence level for the determined size;

when the confidence level is below a predetermined threshold, analyze the image to identify another product on the store shelf, wherein the other product includes another outline element associated with a specific length; and

confirm the size of the bottle based on a comparison of the specific length of the other outline element of the other product and the length of one of the two outline elements of the at least one bottle.

7. The system of claim 1, wherein the at least one processor is further configured to:

determine a confidence level for the determined size;

when the confidence level is below a threshold, identify another outline element of the at least one bottle, the other outline element having a specific length; and

confirm the size of the bottle based on a comparison of the specific length of the other outline element of the at least one bottle and the length of one of the two outline elements of the at least one bottle.

8. The system of claim 1, wherein the at least one processor is further configured to:

determine a confidence level for the determined size;

when the confidence level is below a threshold, analyze the image to identify a label attached to the store shelf, wherein the label includes a textual element associated with a specific length; and

confirm the size of the bottle based on a comparison of the specific length of the textual element of the label and the length of one of the two outline elements of the at least one bottle.

9. The system of claim 1, wherein the image depicts a first bottle and a second bottle located adjacent each other, wherein the first and second bottles are associated with a common brand, but wherein the first and second bottles have different sizes.

10. The system of claim 9, wherein the at least one processor is further configured to use a same two outline elements for the first and second bottles to determine a first product size for the first bottle and a second product size for the second bottle differs than the first product size.

11. The system of claim 5, wherein the at least one processor is further configured to use a Fano factor of the distribution of the lengths of the group of outline elements to determine the size of the at least one bottle.

12. The system of claim 9, wherein the at least one processor is further configured to use different two outline elements for the first and second bottles to determine a first product size for the first bottle and a second product size for the second bottle differs than the first product size.

13. The system of claim 9, wherein the at least one processor is further configured to:

perform a first action associated with the first bottle; and

perform a second action associated with the second bottle, wherein the second action differs from the first action when the determined size of the second bottle differs from the determined size of the first bottle.

14. The system of claim 1, wherein the at least one processor is further configured to determine a product type of the at least one bottle based on the determined size of the at least one bottle and at least one of:

a brand associated with the at least one bottle, a logo associated with the at least one bottle, text associated with the at least one bottle, a price associated with the at least one bottle, and a shape of the at least one bottle.

15. A method for processing images captured in a retail store, the method comprising:

receiving an image depicting a store shelf having a first product comprising at least one bottle and a second product displayed thereon, a type of the first product being different from a type of the second product;

analyzing the image to detect a representation in the image of the at least one bottle, wherein the bottle has an outline design;

identifying in the image two outline elements being associated with the outline design of the at least one bottle, wherein each of the two outline elements has a specific length;

determining a size of the at least one bottle of the first product based on a comparison of the lengths of the two outline elements; and

determining, based on the determined sized of the at least one bottle of the first product, a type of the second product.

16. The method of claim 15, wherein the two outline elements are selected from a group including:

an orifice of a bottle, a lip of the bottle, a collar of the bottle, a neck of the bottle, a heel of the bottle, and a base of the bottle.

17. The method of claim 15, further comprising:

determining a product type of the at least one bottle based on the determined size.

18. The method of claim 15, further comprising:

analyzing the image to obtain a distribution of lengths of a group of outline elements, the group of outline elements comprising at least the two outline elements associated with the outline design of the at least one bottle; and

using a property of the distribution of the lengths of the group of outline elements to determine the size of the at least one bottle.

19. The method of claim 15, further comprising:

determining a confidence level for the determined size;

when the confidence level is below a threshold, attempt to recognize the product type of the at least one bottle based on at least one of;

a brand associated with a detected bottle, a logo associated with the detected bottle, text associated with the detected bottle, a price associated with the bottle, and a shape of the detected bottle; and

confirm the size of the detected bottle based on the product type.

20. A computer program product for processing images captured in a retail store embodied in a non-transitory computer-readable medium and executable by at least one processor, the computer program product including instructions for causing the at least one processor to execute a method comprising:

receiving an image depicting a store shelf having a first product comprising at least one bottle and a second product displayed thereon, a type of the first product being different from a type of the second product;

analyzing the image to detect a representation in the image of the at least one bottle, wherein the at least one bottle has an outline design;

identifying in the image two outline elements being associated with the outline design of the at least one bottle, wherein each of the two outline elements has a specific length;

determining a size of the at least one bottle of the first product based on a comparison of the lengths of the two outline elements; and

determining, based on the determined sized of the at least one bottle of the first product, a type of the second product.