Methods and systems for person detection in a video feed

• US 10,380,429 B2
• Filed: 07/11/2016
• Issued: 08/13/2019
• Est. Priority Date: 07/11/2016
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Chaudhry, Rizwan Ahmed, Dalal, Navneet, Ben-Meshulam, Jonathan Z., Heitz, III, George Alban
• Primary Examiner(s): Bella, Matthew C
• Assistant Examiner(s): Shin, Brian D
• Application Number:
  US15/207,459
• Publication Number:
  US 20180012463A1
• Time in Patent Office: 1,128 Days
• Field of Search: None
• CPC Class Codes: G06F 18/231 Hierarchical techniques, i.e. dividing or merging pattern sets so as to obtain a dendrogram
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/40 Extraction of image or video features
• CPC Class Codes: G06V 10/62 relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
• CPC Class Codes: G06V 10/7625 Hierarchical techniques, i.e. dividing or merging patterns to obtain a tree-like representation; Dendograms
• CPC Class Codes: G06V 20/44 Event detection
• CPC Class Codes: G06V 20/49 Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

The various embodiments described herein include methods, devices, and systems for providing event alerts. In one aspect, a method includes: (1) obtaining a video feed, the video feed comprising a plurality of images; and, (2) for each image, analyzing the image to determine whether the image includes a person, the analyzing including: (a) determining that the image includes a potential instance of a person by analyzing the image at a first resolution; (b) in accordance with the determination that the image includes the potential instance, denoting a region around the potential instance; (c) determining whether the region includes an instance of the person by analyzing the region at a second resolution, greater than the first resolution; and (d) in accordance with a determination that the region includes the instance of the person, determining that the image includes the person.

Claims

1. A method, comprising:

at a computing system having one or more processors and memory;

obtaining a video feed, the video feed comprising a plurality of images; and

for each image in the plurality of images, analyzing the image to determine whether the image includes a person, the analyzing including;

determining that the image includes a potential instance of a person by analyzing the image at a first resolution;

in accordance with the determination that the image includes the potential instance, denoting a region around the potential instance, wherein the area of the region is less than the area of the image;

determining whether the region includes an instance of the person by analyzing the region at a second resolution, greater than the first resolution; and

in accordance with a determination that the region includes the instance of the person;

determining that the image includes the person; and

storing an indication of the determination for use in subsequent alert notification processing.

2. The method of claim 1, further comprising, for each image of the plurality of images, assigning a confidence score to the image.

3. The method of claim 2, wherein the confidence score is based on the analysis of the image at the first resolution.

4. The method of claim 2, wherein the confidence score is based on the analysis of the region at the second resolution.

5. The method of claim 2, wherein the video feed includes a motion event, and the method further comprises, in accordance with a determination that the confidence score for at least one image of the plurality of images exceeds a predetermined threshold, denoting the motion event as involving a person.

6. The method of claim 1, wherein determining that the image includes the potential instance of the person comprises:

detecting the potential instance of the person;

assigning a confidence score to the potential instance of the person; and

in accordance with a determination that the confidence score meets one or more predetermined criteria, determining that the images include the potential instance of the person.

7. The method of claim 1, further comprising:

determining whether the region includes one or more persons other than the potential person; and

in accordance with a determination that the region includes the one or more other persons, denoting the image as containing a person.

8. The method of claim 1, further comprising:

determining that one or more images of the plurality of images includes a person;

obtaining a second plurality of images, the second plurality of images preceding the motion event;

for each image in the second plurality of images, analyzing the image to determine whether the image includes the person;

in accordance with a determination that one or more images of the second plurality of images do not include the person, denoting the motion event as involving the person; and

in accordance with a determination that one or more images of the second plurality of images include the person, forgoing denoting the motion event as involving the person.

9. The method of claim 1, further comprising:

in accordance with a determination that an image includes multiple potential instances of a person, denoting a region encompassing each potential instance; and

determining whether the region includes one or more instances of a person by analyzing the region at a second resolution, greater than the first resolution.

10. The method of claim 1, further comprising:

for each image in the plurality of images, analyzing the image to determine whether the image includes a particular object, the analyzing including;

determining whether the image includes a potential instance of the particular object by analyzing the image at a first resolution;

in accordance with a determination that the image includes a potential instance, denoting a region around the potential instance, wherein the area of the region is less than the area of the image;

determining whether the region includes an instance of the particular object by analyzing the region at a second resolution, greater than the first resolution; and

in accordance with a determination that the region includes an instance of the particular object, determining that the image includes the particular object.

11. The method of claim 1, further comprising determining whether the motion event involves a person by analyzing one or more relationships between images including persons of the plurality of images.

12. A computing system comprising:

one or more processors; and

memory coupled to the one or more processors, the memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for;

obtaining a video feed, the video feed comprising a plurality of images; and

for each image in the plurality of images, analyzing the image to determine whether the image includes a person, the analyzing including;

determining that the image includes a potential instance of a person by analyzing the image at a first resolution;

in accordance with the determination that the image includes the potential instance, denoting a region around the potential instance, wherein the area of the region is less than the area of the image;

determining whether the region includes an instance of the person by analyzing the region at a second resolution, greater than the first resolution; and

in accordance with a determination that the region includes the instance of the person;

determining that the image includes the person; and

storing an indication of the determination for use in subsequent alert notification processing.

13. The computing system of claim 12, wherein the one or more programs further include instructions for assigning a confidence score to each image of the plurality of images.

14. The computing system of claim 13, wherein the video feed includes a motion event, and the one or more programs further include instructions for, in accordance with a determination that the confidence score for at least one image of the plurality of images exceeds a predetermined threshold, denoting the motion event as involving a person.

15. The computing system of claim 12, wherein determining that the image includes the potential instance of the person comprises:

detecting the potential instance of the person;

assigning a confidence score to the potential instance of the person; and

in accordance with a determination that the confidence score meets one or more predetermined criteria, determining that the images include the potential instance of the person.

16. The computing system of claim 12, wherein the one or more programs further include instructions for:

determining that one or more images of the plurality of images includes a person;

obtaining a second plurality of images, the second plurality of images preceding the motion event;

for each image in the second plurality of images, analyzing the image to determine whether the image includes the person;

in accordance with a determination that one or more images of the second plurality of images do not include the person, denoting the motion event as involving the person; and

in accordance with a determination that one or more images of the second plurality of images include the person, forgoing denoting the motion event as involving the person.

17. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a computing system, cause the system to:

obtain a video feed, the video feed comprising a plurality of images; and

for each image in the plurality of images, analyze the image to determine whether the image includes a person, the analyzing including;

determining that the image includes a potential instance of a person by analyzing the image at a first resolution;

in accordance with the determination that the image includes the potential instance, denoting a region around the potential instance, wherein the area of the region is less than the area of the image;

determining whether the region includes an instance of the person by analyzing the region at a second resolution, greater than the first resolution; and

in accordance with a determination that the region includes the instance of the person;

determining that the image includes the person; and

storing an indication of the determination for use in subsequent alert notification processing.

18. The non-transitory computer-readable storage medium of claim 17, wherein the one or more programs comprising instructions, which when executed by the computing system, cause the system to assign a confidence score to each image of the plurality of images.

19. The non-transitory computer-readable storage medium of claim 17, wherein determining that the image includes the potential instance of the person comprises:

detecting the potential instance of the person;

assigning a confidence score to the potential instance of the person; and

in accordance with a determination that the confidence score meets one or more predetermined criteria, determining that the images include the potential instance of the person.

20. The non-transitory computer-readable storage medium of claim 17, wherein the one or more programs comprising instructions, which when executed by the computing system, cause the system to:

determine that one or more images of the plurality of images includes a person;

obtain a second plurality of images, the second plurality of images preceding the motion event;

for each image in the second plurality of images, analyze the image to determine whether the image includes the person;

in accordance with a determination that one or more images of the second plurality of images do not include the person, denote the motion event as involving the person; and

in accordance with a determination that one or more images of the second plurality of images include the person, forgo denoting the motion event as involving the person.