Item identification using video recognition to supplement bar code or RFID information

• US 9,171,442 B2
• Filed: 11/19/2010
• Issued: 10/27/2015
• Est. Priority Date: 11/19/2010
• Status: Active Grant

• Current Assignee: Sensormatic Electronics LLC (Johnson Controls International plc)
• Original Assignee: Tyco Fire & Security GmbH (Johnson Controls International plc)
• Inventors: Clements, Scott M.
• Primary Examiner(s): Pe, Geepy
• Assistant Examiner(s): AMR, SALAME M
• Application Number:
  US12/950,149
• Publication Number:
  US 20120127314A1
• Time in Patent Office: 1,803 Days
• Field of Search: 235/462.01, 235/439, 348/150, 348/135, 382/181, 705/28, 705/8, 705/64, 353/70, 386/126
• US Class Current: 1/1
• CPC Class Codes: G07G 1/0063 with means for detecting the geometric dimensions of the article of which the code is read, such as its size or height, for the verification of the registration
• CPC Class Codes: G08B 13/246 Check out systems combined with EAS, e.g. price information stored on EAS tag anti-theft systems in point of sale systems G07G3/003
• CPC Class Codes: G08B 13/248 EAS system combined with another detection technology, e.g. dual EAS and video or other presence detection system

Abstract

A method and system for confirming the identity of a product in a security system and for determining sales-related information pertaining to the products. One or more products are associated with a product identification element, such as a bar code or an RFID tag. A reader or a scanner within the system receives a signal from the product identification element containing product identity information about the product. The system further includes a camera that captures a video image of the product. Signals representing the product identity information and the video image of the product are forwarded to a data correlation unit that determines if the product associated with the product identity information matches the product in the video image.

Claims

1. A method of using video analytics to obtain sales related information, the method comprising:

capturing a video of a plurality of products residing on a plurality of product support structures, the video including image of persons in proximity to the plurality of product support structures; and

analyzing the captured video to;

determine characteristics of the plurality of product support structures and of the plurality of products residing on the plurality of product support structures, the determined characteristics including;

information relating to interaction of at least one person with at least one of the plurality of product support structures;

information relating to interaction of at least one person with at least one product residing on one of the plurality of product support structures; and

information relating to when at least one product residing on at least one of the plurality of product support structures was removed from the at least one of the plurality of product support structures;

determine if the at least one person removed one of the plurality of products from associated product packaging;

determine if the product support structure is in a predefined location; and

if the product support structure is determined not to be in the predefined location, generate an alert indicating the product support structure is not in the predefined location.

2. The method of claim 1, wherein the determining of characteristics includes determining demographics of the persons in proximity to the product support structure based at least in part on captured video.

3. The method of claim 1, wherein the determining of characteristics includes determining if the removed at least one product was replaced on the product support structure.

4. The method of claim 1, further comprising analyzing the signal from the video capture device to determine if one of the plurality of products was removed from associated product packaging and the product packaging replaced back on the support structure without the product.

5. The method of claim 1, further comprising analyzing the signal from the video capture device to determine if one of the plurality of products on the product support structure does not belong on the product support structure based at least in part on captured video.

6. A system for obtaining sales related information of a plurality of products on a plurality of product support structures, the system comprising:

a video capture device, the video capture device capturing video of a plurality of products residing on a plurality of product support structures, the video including image of persons in proximity to the plurality of product support structures; and

a data correlation unit in communication with the video capture device, the data correlation unit;

receiving a signal from the video capture device, the signal representing the video of the plurality of products residing on the plurality of product support structures; and

analyzing the signal from the video capture device to;

determine characteristics of the plurality of product support structures and of the plurality of products residing on the plurality of product support structures, the determined characteristics including;

information relating to interaction of at least one person with at least one of the plurality of product support structures;

information relating to interaction of at least one person with at least one product residing on one of the plurality of product support structures; and

information relating to when at least one product residing on at least one of the plurality of product support structures was removed from the at least one of the plurality of product support structures;

determine if the at least one person removed one of the plurality of products from associated product packaging;

determine if the product support structure is in a predefined location; and

if the product support structure is determined not to be in the predefined location, generate an alert indicating the product support structure is not in the predefined location.

7. The system of claim 6, further comprising a reader in communication with a product identification element, the product identification element associated with one of the plurality of products, the reader receiving a signal containing product identity information from the product identification element.

8. The system of claim 7, wherein the data correlation unit:

receives a signal from the reader, the signal including the product identity information; and

Determine if the one of the plurality of products associated with the product identity information matches an image of the one of the plurality of products captured by the video capture device.

9. The system of claim 7, wherein the product identification element is a radio frequency identification (RFID) tag and the signal containing the product identity information is an RFID signal.

10. The system of claim 7, wherein the product identification element is a bar code.

11. The system of claim 6, wherein the video capture device transmits a signal corresponding to the image of the persons to the data correlation unit;

and

the data correlation unit further analyzes the signal from the video capture device to determine demographics of the persons in proximity to the product support structure based at least in part on the captured images.

12. The system of claim 6, wherein the data correlation unit further analyzes the signal from the video capture device to determine if one of the plurality of products was removed from associated product packaging and the product packaging replaced back on the support structure without the product.