Video analytics for simulating the motion tracking functionality of a surveillance camera

• US 9,049,348 B1
• Filed: 11/10/2010
• Issued: 06/02/2015
• Est. Priority Date: 11/10/2010
• Status: Active Grant

• Current Assignee: Target Brands Inc. (Target Corporation)
• Original Assignee: Target Brands Inc. (Target Corporation)
• Inventors: Foster, Rob, Persian, Steve
• Primary Examiner(s): Anyikire, Chikaodili E
• Application Number:
  US12/943,791
• Time in Patent Office: 1,665 Days
• Field of Search: 348/143, 348/150, 348/169
• US Class Current: 1/1
• CPC Class Codes: G06Q 30/0268 at point-of-sale [POS]
• CPC Class Codes: G06Q 30/0271 Personalized advertisement
• CPC Class Codes: G07G 3/003 Anti-theft control theft alarms G08B13/00
• CPC Class Codes: G08B 13/24 by interference with electromagnetic field distribution
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

In some implementations, a camera captures a video stream of a region of a business without panning or tilting, where the camera includes a field of view that identifies a portion of the region that is captured. An analytics system optionally analyzes the video stream to determine a path of a customer moving across the camera'"'"'s field of view. In selected embodiments, an image display system can extract a subset of the video stream, where the subset of the video stream depicts the customer during substantially all of the customer'"'"'s movement across the camera'"'"'s field of view. A public facing monitor may present the subset of the video stream such that the system effectively emulates the output of a pan, tilt, zoom camera that is tracking the customer across the field of view.

Claims

1. A method comprising:

receiving, from a camera, a first sequence of images of a retail environment, wherein a field of view of each of the images in the first sequence is the same as the field of view of the other images in the first sequence;

analyzing the first sequence of images to identify an object moving in the field of view;

analyzing the first sequence of images to identify a product for sale in the retail environment in the field of view;

determining a relationship between the object moving in the field of view and the product in the field of view;

identifying a rule previously stored in an analytics rules database associated with the object moving in the field of view and the product in the field of view and associated with the relationship between the object and the product, the rule specifying a procedure to be performed based on the analysis of the first sequence of images;

identifying a second sequence of images based on the rule associated with the object moving in the field of view, wherein each image in the second sequence is cropped from the first sequence of images such that each image in the second sequence is included within a respective image in the first sequence and each image in the second sequence includes the object;

displaying the second sequence of images on a public facing monitor, wherein the public facing monitor depicts the object during substantially all of the object'"'"'s movement in the field of view; and

displaying content on the public facing monitor relating to the relationship between the object and the product.

2. The method of claim 1, wherein the analyzing comprises analyzing a subset of the first sequence of images to determine a current rate of movement of the object in the field of view and a current direction of movement of the object in the field of view and wherein the identifying comprises selecting the second sequence of images based on the current rate of movement of the object in the field of view and the current direction of movement of the object in the field of view.

3. The method of claim 1, wherein the analyzing comprises performing object recognition on each of the images in the first sequence to identify a position of the object in each of the images in the first sequence and wherein the identifying comprises selecting each of the images in the second sequence based on the position of the object in the respective image in the first sequence of images.

4. The method of claim 1, wherein the first sequence of images comprises a single first image, and displaying the second sequence of images comprises statically displaying a single second image on the public facing monitor, wherein the single second image is included within the single first image.

5. The method of claim 1, further comprising detecting movement of the object in the field of view and wherein the receiving is based on the detecting.

6. The method of claim 5, further comprising presenting an advertisement on the public facing monitor when no movement is detected in the field of view and wherein the displaying is based on the detecting.

7. The method of claim 5, wherein the camera and the public facing monitor are physically located in a region of the retail environment.

8. The method of claim 1, wherein the object comprises a person.

9. The method of claim 1, wherein displaying the second sequence of images comprises displaying a message on the public facing monitor, and wherein the message is overlaid on top of the second sequence of images.

22. The method of claim 1, wherein the object comprises a customer, and

wherein the step of determining a relationship between the object moving in the field of view and the product in the field of view comprises determining that the customer is interacting with the product while walking across the field of view, and wherein the step of displaying content on the public facing monitor comprises displaying a message related to the product on the public facing monitor.

23. The method of claim 22, wherein the step of displaying a message related to the product comprises displaying an advertisement concurrently with the second sequence of images on the public facing monitor.

24. The method of claim 1, wherein the object comprises a customer, and wherein the step of displaying content on the public facing monitor relating to the relationship between the object and the product comprises displaying a message that an employee of the retail environment is coming to help the customer.

25. The method of claim 1, wherein the object comprises a customer, and wherein the step of displaying content on the public facing monitor relating to the relationship between the object and the product comprises displaying product comparison information between the product and another similar product.

10. A non-transitory physical computer-readable medium,

embodied in a device, on which medium is encoded program code configured for execution by a processor to perform a method for identifying a sequence of images that simulate tracking functionality of an articulating camera, the method comprising:

(a) receiving a sequence of first images, wherein the sequence of first images depicts a first area;

(b) analyzing a portion of the sequence of first images to identify an entity moving across the first area;

(c) identifying a sequence of second images, wherein the sequence of second images is a cropped portion of the sequence of first images and the sequence of second images includes the entity, wherein the analyzing comprises analyzing the sequence of first images to determine a velocity and direction of the entity across the first area, and wherein the identifying comprises selecting the sequence of second images based on the velocity and the direction of the entity across the first area;

(d) when the velocity of the entity exceeds a threshold velocity, exporting the sequence of second images for presentation on a display;

(e) when the velocity exceeds the threshold velocity, repeating steps (a) through (d) to present the sequence of second images so the display depicts the entity during at least a substantial majority of the entity'"'"'s movement across the first area such that the sequence of said second images presented on the display simulates the tracking functionality of an articulating camera, wherein step (b) comprises predicting a location of the entity in the portion of the first images sequence not analyzed based on the velocity and the direction of the entity determined in step (c); and

(f) precluding exporting of the sequence of second images when the velocity does not exceed the threshold velocity.

11. The medium of claim 10, wherein the entity comprises a human.

12. The medium of claim 10, wherein a first number of images in the sequence of said first images is less than a second number of images in the sequence of said second images.

13. The medium of claim 10, wherein the analyzing comprises performing object recognition on a sequence of said first images to identify a position of the entity in each of the images in the sequence of said first images and wherein the identifying comprises selecting each of the images in the sequence of said second images based on the position of the entity in the image from the sequence of said first images corresponding to the image in the sequence of said second images.

14. The medium of claim 10, wherein the exporting comprises selecting a message for presentation on the display, and wherein the display presents the message and the second image simultaneously.

15. The medium of claim 14, wherein the message comprises product information for a product associated with the entity.

16. A system comprising:

a static camera device, wherein the static camera device is located in a region of a business, the static camera device is configured to capture a video stream of a field of capture of the static camera device wherein the field of capture includes a portion of the region, and the static camera device does not pan or tilt during capture of the video stream;

a computer-implemented analytics system configured to analyze the video stream and determine a path of an item across the field of capture;

an analytics rules database storing rules based on characteristics of the analyzed video stream;

a computer-implemented image display system configured to identify a cropped video stream, wherein the cropped video stream is based on the path of the item across the field of capture and the cropped video stream is extracted from the video stream captured by the static camera device; and

a closed circuit television configured to present the cropped video stream based on a rule previously stored in the analytics rules database associated with the item moving in the field of capture, the rule specifying a procedure to be performed based on the analysis of the video stream, the cropped video stream being presented while the object is moving across the field of capture, wherein the closed circuit television is located in the region of the business and wherein the cropped video stream from the static camera device emulates a tracking functionality of an articulating camera device when presented on the closed circuit television,

wherein the analytics system analyzes a portion of the video stream captured by the camera device to determine a current rate of movement of the item across the field of capture and a current direction of movement of the item across the field of capture, and wherein the image display system identifies the cropped video stream by extrapolating location information for a non-analyzed portion of the video stream based on the path of the item identified from the analyzed portion of the video stream.

17. The system of claim 16, wherein the item comprises a human.

18. The system of claim 16, wherein the portion of the video stream analyzed by the analytics system comprises less than fifty percent of the video stream.

19. The system of claim 17, wherein the video stream captured by the camera device comprises a plurality of frames, wherein the cropped video stream comprises a plurality of cropped frames and each of the plurality of cropped frames corresponds to a respective one of the plurality of frames the cropped frame is extracted from, wherein the analytics system performs object recognition on each of the plurality of frames to identify a position of the item in the frame, and wherein the image display system identifies each of the plurality of cropped frames based on the position of the item in the frame corresponding to the cropped frame.

20. The system of claim 16, wherein the cropped video stream is presented on the closed circuit television based on the camera device detecting movement in the field of capture.

21. The system of claim 20, wherein the closed circuit television is configured to present a plurality of customer messages when the cropped video stream is not presented.