Still image shopping event monitoring and analysis system and method

• US 8,873,794 B2
• Filed: 02/12/2008
• Issued: 10/28/2014
• Est. Priority Date: 02/12/2007
• Status: Active Grant

• Current Assignee: Shopper Scientist LLC
• Original Assignee: Shopper Scientist LLC
• Inventors: Sorensen, Herb, Albers, David, Byram, Mark A., Gibney, Michael D., Wynne, Michael S.
• Primary Examiner(s): Zeender, Ryan
• Assistant Examiner(s): Amsdell, Dana
• Application Number:
  US12/030,132
• Publication Number:
  US 20080215462A1
• Time in Patent Office: 2,450 Days
• US Class Current: 382/100
• CPC Class Codes: G06Q 10/06 Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: G06Q 30/06 Buying, selling or leasing transactions
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

The still image shopping event analysis systems and methods provided herein may implement low frequency still image sampling and perform a computer analysis of the still images captured, including discriminating differences between frames of the still images based on changes of pixels between the frames and detecting and/or analyzing one or more shopping events based on the discriminated differences between frames of the still images. The systems and methods provided herein may further count and/or analyze the shopping events based on patterns of changes between frames, including for example, numbers of customers visiting and amounts of time customers spent visiting a shopping area, whether the visit was a transitory visit or involved more detailed shopping, whether a purchase occurred, and/or which and number of item(s) purchased.

Claims

1. A still image shopping event analysis method, comprising:

capturing at a low frequency a sequence of still images from a fixed position camera of one or more monitored stationary shelves; and

performing, by an analysis computer, an analysis of the still images captured, including;

converting each still image into a pixel matrix comprising, for each pixel, only a pixel luminosity value such that each pixel matrix is lacking information usable to reconstruct the pixel matrix back to a displayable image;

breaking down each pixel matrix into different regions, said regions comprising an aisle region having a plurality of zones, a shelf region, and a plurality of product regions in the shelf region, each product region having an associated product;

discriminating differences between frames of the still images based on changes in pixels between the frames;

detecting one or more shopping events occurring in a zone of the aisle region that is adjacent a first product region of the shelf region based on the discriminated pixel differences between frames of the still images alone without recognition of image features within the still images;

categorizing the one or more shopping events, based on changes in pixels between frames, into constant-events during which no customer is visiting the zone of the aisle region adjacent the first product region when changes in pixels in the zone between frames are below a threshold, and visiting-events during which one or more customers are visiting the zone of the aisle region adjacent the first product region when changes in pixels in the zone between frames are above a threshold;

detecting a purchasing-event, in which a purchase of a product located in the first product region has been made by a visiting customer during a visiting-event, by comparing a first frame captured during a constant-event preceding the visiting-event to a second frame captured during the constant-event following the visiting-event and detecting changes in pixels of the first product region between the first frame and the second frame, the changes in pixels indicating that the product from the first product region has been removed from the shelf region during the purchasing-event between the first frame and the second frame, the detecting being performed based on changes in pixels alone without recognition of image features within the still images, wherein the product that was removed during the purchasing-event is identified based upon the location of the changed pixels in the product region.

2. The method of claim 1, wherein the monitored stationary shelves include one or more areas of a grocery store, pharmacy, fresh meat counter, and/or deli counter.

3. The method of claim 1, wherein the still images are captured using one or more power over Ethernet (POE) cameras that are configured to snap still images at a desired rate that is lower than a video frame rate.

4. The method of claim 1, wherein the computer analysis is carried out by an analysis computer, and the method further comprises transmitting the still images to the analysis computer via a wireless virtual private network (VPN) over an internet, prior to performing the computer analysis.

5. The method of claim 1, wherein performing the computer analysis of the still images further includes measuring duration of the one or more shopping events.

6. The method of claim 1, wherein performing the computer analysis of the still images further includes estimating a number and/or percentage of customers who have visited the zone of the aisle region of the monitored stationary shelves based on a number of visiting-events identified in a particular time period.

7. The method of claim 1, wherein performing the computer analysis further includes determining an entrance direction of a visiting customer during the visiting-event based on characteristics of a line graph of the visiting-event, wherein the line graph of the visiting-event plots differences between frames.

8. The method of claim 1, further comprising:

categorizing the visiting-events, based on duration of the visiting-events, into transitory-events and stopping-to-shop-events, wherein during each of the visiting-events, one or more customers are passing the monitored stationary shelves, and during each of the stopping-to-shop-events, one or more customers visiting a zone of an aisle region of the monitored stationary shelves also stopped to shop;

wherein performing the computer analysis of the still images further includes estimating a number and/or a percentage of customers who stopped to shop at the monitored stationary shelves based on a number of stopping-to-shop-events occurring in a particular time period.

9. The method of claim 1, wherein performing the computer analysis of the still images further includes estimating a number of visiting customers during a visiting-event based on a share of and/or number of pixels of the still images that have been changed from the still images of the constant-event, and/or changes in pixels throughout the visiting-event.

10. The method of claim 1, wherein performing the computer analysis of the still images further includes calculating a total number of the products being purchased based on a number of purchasing-events during which customers have purchased the product.

11. The method of claim 10, wherein performing the computer analysis further includes identifying product stock-outs and/or pending product stock-outs of the product, and/or providing alerts regarding product stock-outs and/or pending product stock-outs of the product based on the calculated total number of the products being purchased.

12. The method of claim 1, wherein performing the computer analysis of the still images further includes identifying a number of purchasing-events that occurred in a particular time period, and estimating a number and/or percentage of customers visiting the monitored stationary shelves who also made a purchase of an item located in the monitored stationary shelves based on the number of purchasing-events.

13. The method of claim 1, wherein the method further includes tagging the still images with tagging information and using the tagging information to assist the computer analysis of the still images.

14. The method of claim 13, wherein the tagging information enables single images and/or groups of images of the shopping event to be viewed to assist in the computer analysis of the shopping event.

15. A still image shopping event analysis system, comprising:

a fixed position camera configured to capture at a low frequency a sequence of pixel matrices each comprising pixel array data captured while imaging one or more monitored stationary shelves, wherein the pixel array data comprises, for each pixel, information usable for statistical analysis but lacking information usable to construct the pixel matrix to a display image; and

a data analysis module executed by an analysis computer, configured to;

break down each pixel matrix into different regions, said regions comprising an aisle region having a plurality of zones, a shelf region, and product regions in the shelf region, each product region having an associated product;

perform computer analysis of the pixel matrices to discriminate differences between the pixel matrices based on changes in pixels;

detect one or more shopping events occurring in a zone of the aisle region that is adjacent a first product region of the shelf region based on the discriminated pixel differences between the pixel matrices;

categorize the shopping events, based on changes in pixels between pixel matrices, into constant-events during which no customer is visiting the zone of the aisle region that is adjacent the first product region when changes in pixels in the zone between frames are below a threshold, and visiting-events during which one or more customers are visiting the zone of the aisle region that is adjacent the first product region when changes in pixels in the zone between pixel matrices are above a threshold;

detect a purchasing-event, in which a purchase of a product located in the first product region has been made by a visiting customer during a visiting-event, by comparing a first pixel matrix captured during a constant-event preceding the visiting-event to a second pixel matrix captured during a constant-event following the visiting-event and detecting changes in pixels of the first product region between the first pixel matrix and the second pixel matrix, the changes in pixels indicating that the product from the first product region has been removed from the shelf region during the purchasing-event between the first pixel matrix and the second pixel matrix, the detecting being performed based on changes in pixels alone without recognition of image features, wherein a product that was removed during the purchasing-event is identified based upon a location of the changed pixels in the product region.

16. The still image shopping event analysis system of claim 15, further comprising a still image tagging module configured to tag the pixel matrices to assist in the computer analysis of the still pixel matrices.