USING PRICE IN VISUAL PRODUCT RECOGNITION

• US 20190213418A1
• Filed: 03/14/2019
• Published: 07/11/2019
• Est. Priority Date: 01/10/2018
• Status: Active Grant

• Current Assignee: Trax Technology Solutions Pte. Ltd. (Trax Technologies (SG) Pte Ltd.)
• Original Assignee: Trax Technology Solutions Pte. Ltd. (Trax Technologies (SG) Pte Ltd.)
• Inventors: Adato, Yair, Eisenschtat, Aviv, Mhabary, Ziv, Cohen, Daniel Shimon
• Granted Patent: US 10,558,843 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/23 Updating
• CPC Class Codes: G06F 16/235 Update request formulation
• CPC Class Codes: G06F 16/288 Entity relationship models
• CPC Class Codes: G06F 16/55 Clustering; Classification
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06F 16/90335 Query processing
• CPC Class Codes: G06F 17/18 for evaluating statistical data , e.g. average values, frequency distributions, probability functions, regression analysis forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04
• CPC Class Codes: G06F 18/2115 by evaluating different subsets according to an optimisation criterion, e.g. class separability, forward selection or backward elimination
• CPC Class Codes: G06Q 10/06311 Scheduling, planning or task assignment for a person or group
• CPC Class Codes: G06Q 10/063112 Skill-based matching of a person or a group to a task
• CPC Class Codes: G06Q 10/06316 Sequencing of tasks or work
• CPC Class Codes: G06Q 10/08 Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: G06Q 10/0875 Itemisation or classification of parts, supplies or services, e.g. bill of materials
• CPC Class Codes: G06Q 20/203 Inventory monitoring
• CPC Class Codes: G06Q 30/0629 for generating comparisons
• CPC Class Codes: G06Q 30/0639 Item locations
• CPC Class Codes: G06Q 30/0643 Graphical representation of items or shoppers
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30232 : Surveillance
• CPC Class Codes: G06T 2207/30242 : Counting objects in image
• CPC Class Codes: G06T 7/0002 : Inspection of images, e.g. flaw detection
• CPC Class Codes: G06T 7/13 : Edge detection
• CPC Class Codes: G06T 7/20 : Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/521 : from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06T 7/55 : from multiple images
• CPC Class Codes: G06T 7/70 : Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/75 : involving models
• CPC Class Codes: G06V 10/17 : using hand-held instruments
• CPC Class Codes: G06V 20/00 : Scenes; Scene-specific elements control of digital cameras H04N23/60
• CPC Class Codes: G06V 20/20 : in augmented reality scenes
• CPC Class Codes: G06V 20/52 : Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/62 : Text, e.g. of license plates, overlay texts or captions on TV images
• CPC Class Codes: G06V 20/64 : Three-dimensional objects
• CPC Class Codes: G06V 20/68 : Food, e.g. fruit or vegetables
• CPC Class Codes: G06V 30/10 : Character recognition
• CPC Class Codes: G06V 40/10 : Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G07G 1/0036 : Checkout procedures
• CPC Class Codes: G08B 21/18 : Status alarms G08B21/02 takes precedence
• CPC Class Codes: G08B 21/182 : Level alarms, e.g. alarms responsive to variables exceeding a threshold
• CPC Class Codes: H04N 23/51 : Housings
• CPC Class Codes: H04N 23/54 : Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
• CPC Class Codes: H04N 23/60 : Control of cameras or camera modules
• CPC Class Codes: H04N 23/611 : where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/65 : Control of camera operation in relation to power supply
• CPC Class Codes: H04N 23/66 : Remote control of cameras or camera parts, e.g. by remote control devices
• CPC Class Codes: H04N 23/661 : Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/80 : Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/90 : Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

A system for processing images captured in a retail store and differentiating between products with similar visual appearances is provided. The system may comprise at least one processor configured to receive at least one image depicting a store shelf having a product displayed thereon, analyze the at least one image to detect the product, determine that the detected product is either a first type of product or a second type of product, analyze the at least one image to determine a price associated with the detected product, determine that the detected product is of the first type of product when the determined price falls within a first price range associated with the first type of product, and determine that the detected product is of the second type of product when the determined price falls within a second price range associated with the second type of product.

Claims

1-113. -113. (canceled)

114. A system for processing images captured in a retail store and differentiating between products with similar visual appearances, the system comprising:

at least one processor configured to;

receive at least one image depicting a store shelf having a product displayed thereon;

analyze the at least one image to detect the product represented in the at least one image;

determine that the detected product is either a first type of product or a second type of product, wherein each of the first type of product and the second type of product is associated with a different price range;

analyze the at least one image to determine a price associated with the detected product;

determine that the detected product is of the first type of product when the determined price falls within a first price range associated with the first type of product; and

determine that the detected product is of the second type of product when the determined price falls within a second price range associated with the second type of product.

115. The system of claim 114, wherein a determination of the price associated with the detected product includes detecting a label attached to the product, and the at least one processor is further configured to analyze the at least one image to recognize a price indicator on the detected label.

116. The system of claim 114, wherein a determination of the price associated with the detected product includes detecting a label attached to the store shelf, and the at least one processor is further configured to analyze the at least one image to recognize a price indicator on the detected label.

117. The system of claim 114, wherein the first price range associated with the first type of product is higher than the second price range associated with the second type of product.

118. The system of claim 117, wherein the at least one processor is further configured to:

analyze the at least one image to detect a promotion associated with the detected product; and

based on the detected promotion determine that the type of the detected product is the first type of product even when the identified price does not fall within the first price range.

119. The system of claim 117, wherein the at least one processor is further configured to determine that the type of the detected product is the first type of product when the determined price is greater than the first price range.

120. The system of claim 117, wherein the at least one processor is further configured to determine that the type of the detected product is the second type of product when the determined price is less than the second price range.

121. The system of claim 114, wherein the at least one processor is further configured to access a database including a first catalog price for the first type of product and a second catalog price for second type of product, and wherein the determined price is different from the first catalog price and the second catalog price.

122. The system of claim 121, wherein the at least one processor is further configured to determine that the type of the detected product is the first type of product when the determined price is closer to the first catalog price than to the second catalog price.

123. The system of claim 114, wherein when the determined price does not fall within the prices ranges associated with the first type of product and the second type of product, the at least one processor is further configured to:

provide a visual representation of the detected product to a user;

receive input from the user indicative of a correct type of the detected product; and

initiate an action based on the received input and the identified price.

124. The system of claim 123, wherein the action comprises updating at least one of the first or second prices ranges.

125. The system of claim 123, wherein the action comprises informing the user about a mismatch between a price of the correct type of the detected product and the identified price.

126. The system of claim 114, wherein the first type of product and the second type of product are associated with different sizes.

127. The system of claim 114, wherein the first type of product and the second type of product are associated with different ingredients.

128. The system of claim 114, wherein the at least one processor is further configured to determine that the type of the detected product is either the first type of product or the second type of product based on at least one of:

a visual appearance of the detected product, a brand associated with the detected product, a logo associated with the detected product, text associated with the detected product, and a shape of the detected product.

129. A method for processing images captured in a retail store, the method comprising:

receiving at least one image depicting a store shelf having a product displayed thereon;

analyzing the at least one image to detect the product represented in the at least one image;

determining that the detected product is either a first type of product or a second type of product, wherein the first type of product and the second type of product have similar visual appearances and each is associated with a different prices range, a first price range associated with the first type of product is higher than a second price range associated with the second type of product;

analyzing the at least one image to determine a price associated with the detected product;

determining that the detected product is of the first type of product when the determined price falls within the first price range; and

determining that the detected product is of the second type of product when the determined price falls within the second price range.

130. The method of claim 129, further comprising:

determining that the type of the detected product is of the first type of product when the determined price is higher than the first price range; and

determining that the type of the detected product is of the second type of product when the determined price is lower than the second price range.

131. The method of claim 129, wherein when the determined price does not fall within the prices ranges of the first type of product and the second type of product, method further comprising:

providing a visual representation of the detected product to a user;

receiving input from the user indicative of a correct type of the detected product; and

initiating an action based on the received input and the identified price, wherein the action includes at least one of;

updating at least one of the first and second prices ranges, or informing the user about a mismatch between a price of the correct type of the detected product and the identified price.

132. The method of claim 129, wherein a determination of the price associated with the detected product includes detecting a label attached to the product, and the method further includes analyzing the at least one image to recognize a price indicator on the detected label.

133. A computer program product for processing images captured in a retail store embodied in a non-transitory computer-readable medium and executable by at least one processor, the computer program product including instructions for causing the at least one processor to execute a method comprising:

receiving at least one image depicting a store shelf having a product displayed thereon;

analyzing the at least one image to detect the product represented in the at least one image;

determining that the detected product is either a first type of product or a second type of product, wherein each of the first type of product and the second type of product is associated with a different price range;

analyzing the at least one image to determine a price associated with the detected product; and

determining that the detected product is of the first type of product when the determined price falls within a first price range associated with the first type of product; and

determining that the detected product is of the second type of product when the determined price falls within a second price range associated with the second type of product.

134-284. -284. (canceled)