Image Processing for Identifying Individuals

• US 20200134339A1
• Filed: 10/30/2018
• Published: 04/30/2020
• Est. Priority Date: 10/30/2018
• Status: Active Grant

• Current Assignee: NCR Voyix Corporation
• Original Assignee: NCR Corporation
• Inventors: Zucker, Brent Vance, Lieberman, Adam Justin
• Granted Patent: US 10,929,697 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/217 Validation; Performance evaluation; Active pattern learning techniques
• CPC Class Codes: G06Q 30/0633 Lists, e.g. purchase orders, compilation or processing
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/292 Multi-camera tracking
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

Cameras capture time-stamped images of predefined areas. At least one image includes a representation of a portion of an individual with other portions of the individual occluded within the image. Pixel attributes for the portion of the individual are identified and provided as a box or set of coordinates for tracking the individual within the image and in subsequent images taken.

Claims

1. A method, comprising:

receiving an image that depicts a portion of an individual with other portions of the individual occluded within the image;

identifying pixel attributes associated with the portion and ignoring other pixel attributes of the image; and

providing metadata that uniquely identifies the portion within the image based on the identifying.

2. The method of claim 1, wherein receiving further includes obtaining the image from a person tracker that is unable to identify the individual from the image.

3. The method of claim 2, wherein identifying further includes passing the image to a machine-learning algorithm and receiving as output from the machine-learning algorithm coordinates representing a box within the image for the pixel attributes.

4. The method of claim 1, wherein identifying further includes dividing the image into a grids and evaluating each grid cell for candidate attributes associated with the individual.

8. The method of claim 1, wherein providing further includes providing the metadata to a person tracker that is tracking the individual within a frictionless store system.

9. The method of claim 1, wherein providing further includes providing the metadata as coordinates that identify locations within the image associated with the individual.

10. The method of claim 1 further comprising, assigning an item identifier to a shopping cart associated with the individual based on the metadata and the other pixel attributes for the image.

5. The method of claim 5, wherein dividing further includes weighting each grid cell with a confidence value indicating a confidence that one or more of the candidate attributes appear in that grid cell.

6. The method of claim 5, wherein weighting further includes comparing each confidence value against a threshold confidence value and retaining select ones of the grid cells associated with the confidence values that meet or exceed the threshold confidence value.

7. The method of claim 6, wherein comparing further includes providing coordinates representing a box within the image for the select ones of the grid cells to a person tracker as the metadata.

11. A method, comprising:

training a machine-learning algorithm with first images of first individuals that are occluded within the first images;

providing second images to the machine-learning algorithm with second individuals that are occluded within the second images;

receiving metadata as output from the machine-learning algorithm that identifies locations within the second images that are associated with the second individuals; and

maintaining shopping carts for items picked up by the second individuals based on the metadata and other metadata associated with the second images.

12. The method of claim 11, wherein training further includes training the machine-learning algorithm with a first set of the first images taken from first cameras having a first angle that focuses downward on the first individuals.

13. The method of claim 12, wherein training further includes training the machine-learning algorithm with a second set of the first images taken from second cameras having a second angle that focuses upward on the first individuals.

14. The method of claim 13, wherein training further includes training the machine-learning algorithm with a third set of the first images taken from the first and second cameras where a first set of the first individuals are blocked in varying degrees by objects present in the third set of the first images.

15. The method of claim 11, wherein receiving further includes obtaining the metadata as coordinates for boxes that encloses the locations within the second images.

16. The method of claim 11, wherein receiving further includes assigning a unique person identifier for each second individual being tracked within the second images based on the metadata.

17. The method of claim 11, wherein receiving further includes processing the metadata with third images to track each of the second individuals in the third images.

18. The method of claim 11, wherein maintaining further includes assigning item identifiers provided by an item tracker for the items picked up to a transaction manager with individual identifiers for each of the second individuals based on the metadata.

19. A system, comprising:

cameras configured to capture images within a store;

a server that includes a processor;

the processor configured to execute executable instructions from a non-transitory computer-readable storage medium as individual detector;

the individual detector when executed on the processor configured to;

track individuals identified in images;

provide occluded images to a machine-learning algorithm that is trained on identifying persons in the occluded images;

receive metadata as output from the machine-learning algorithm that specifies locations for the individuals within the occluded images;

process the metadata to track the individuals in the occluded images; and

interact with an item identifier and a transaction manager to track items picked up by the individuals and to assign item identifiers for the items to shopping carts maintained by transaction manager for each of the individuals based on the images and the occluded images.

20. The system of claim 19, wherein the individual detector is further configured to process the metadata as coordinates representing boxes within the occluded images for the individuals.