Method and associated apparatus for performing cooperative counting with aid of multiple cameras

• US 10,223,592 B2
• Filed: 05/15/2017
• Issued: 03/05/2019
• Est. Priority Date: 10/14/2016
• Status: Active Grant

• Current Assignee: Synology Incorporated
• Original Assignee: Synology Incorporated
• Inventors: Hsu, Szu-Lu, Chiu, Yu-Hsiang, Lee, Szu-Hsien
• Primary Examiner(s): Lu, Tom Y
• Application Number:
  US15/595,910
• Publication Number:
  US 20180107878A1
• Time in Patent Office: 659 Days
• CPC Class Codes: G06F 18/2113 by ranking or filtering the set of features, e.g. using a measure of variance or of feature cross-correlation
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30242 Counting objects in image
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/44 Event detection
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/53 Recognition of crowd images, e.g. recognition of crowd congestion
• CPC Class Codes: G07C 9/00 Individual registration on entry or exit
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A method for performing cooperative counting and an associated apparatus are provided, where the method is applicable to a counter system, and the counter system includes a plurality of cameras. The method includes: setting a plurality of points on an electronic map as a plurality of predetermined points according to user inputs; determining at least one rule related to the predetermined points according to rule information, where the rule information is stored in the counter system; respectively performing video object detection upon a plurality of images captured by the cameras to generate detection results respectively corresponding to the cameras; and merging the detection results respectively corresponding to the cameras, to count events complying with the at least one rule.

Claims

1. A method for performing cooperative counting, wherein the method is applicable to a counter system, the counter system comprises a plurality of cameras, and the method comprises the following steps:

setting a plurality of points on an electronic map as a plurality of predetermined points according to user inputs;

determining at least one rule related to the predetermined points according to rule information, wherein the rule information is stored in the counter system;

performing video object detection upon a plurality of images captured by the cameras, respectively, to generate detection results respectively corresponding to the cameras;

merging the detection results respectively corresponding to the cameras, to count events complying with the at least one rule;

wherein the step of performing the video object detection upon the images to generate the detection results further comprises;

performing the video object detection upon the images captured by the cameras, respectively, to automatically generate a plurality of sets of candidate coordinate values, wherein image capture ranges of the cameras are mapped to a plurality of areas on the electronic map, respectively, the sets of candidate coordinate values represent candidate positions of a plurality of video objects on the electronic map, and a subset of the sets of candidate coordinate values is within an overlapping area of at least one portion of the areas;

wherein the step of merging the detection results to count the events complying with the at least one rule further comprises;

selectively altering the subset of the sets of candidate coordinate values to transform the sets of candidate coordinate values into a plurality of sets of coordinate values, wherein the sets of coordinate values represent positions of the video objects on the electronic map; and

monitoring the video objects according to the sets of coordinate values to count the events complying with the at least one rule.

2. The method of claim 1, wherein the predetermined points indicate at least one predetermined line segment on the electronic map;

and the events complying with the at least one rule comprise an event of a video object passing through the predetermined line segment.

3. The method of claim 1, wherein the predetermined points indicate a boundary of a predetermined area on the electronic map, for designating the predetermined area;

and the events complying with the at least one rule comprise an event of a video object entering the predetermined area.

4. The method of claim 1, wherein the video object detection comprises image detection of a specific type of objects.

5. The method of claim 1, wherein the video object detection comprises pedestrian detection, and the video objects represent a plurality of pedestrians, wherein the sets of coordinate values represent positions of the pedestrians on the electronic map;

and

the step of performing the video object detection upon the images to generate the detection results further comprises;

performing the pedestrian detection upon the images captured by the cameras, respectively, to automatically generate the sets of candidate coordinate values.

6. The method of claim 5, wherein the predetermined points indicate at least one predetermined line segment on the electronic map;

the events complying with the at least one rule comprises an event of a pedestrian passing through the predetermined line segment; and

the step of monitoring the video objects according to the sets of coordinate values to count the events complying with the at least one rule further comprises;

monitoring the plurality of pedestrians according to the sets of coordinate values to count one or more pedestrians passing through the predetermined line segment among the plurality of pedestrians.

7. The method of claim 1, wherein the step of performing the video object detection upon the images captured by the cameras, respectively, to automatically generate the sets of candidate coordinate values further comprises:

performing video object image recognition upon an image of the images, to automatically generate at least one set of image coordinate values in an image coordinate system of the image, wherein the set of image coordinate values indicate a position of at least one video object within the video objects in the image; and

performing a coordinate transformation upon the set of image coordinate values according to coordinate transformation reference information, to automatically generate at least one set of candidate coordinate values within the sets of candidate coordinate values, wherein the set of candidate coordinate values represent a candidate position of the video object within the video objects on the electronic map.

8. The method of claim 7, wherein the image is captured by a camera within the cameras, and the coordinate transformation reference information comprises transformation information of an image capture range of the camera for performing the coordinate transformation.

9. The method of claim 1, wherein the step of selectively altering the subset of the sets of candidate coordinate values to transform the sets of candidate coordinate values into the sets of coordinate values further comprises:

referring to a predetermined distance threshold to determine whether multiple sets of candidate coordinate values respectively corresponding to different cameras within the plurality of sets of candidate coordinate values represent a same video object, in order to selectively merge the multiple sets of candidate coordinate values into a set of coordinate values within the sets of coordinate values.

10. The method of claim 9, wherein the step of selectively altering the subset of the sets of candidate coordinate values to transform the sets of candidate coordinate values into the sets of coordinate values further comprises:

when a distance between multiple points represented by the multiple sets of candidate coordinate values is less than the predetermined distance threshold, merging the multiple sets of candidate coordinate values into the set of coordinate values within the sets of coordinate values.

11. The method of claim 10, wherein the step of selectively altering the subset of the sets of candidate coordinate values to transform the sets of candidate coordinate values into the sets of coordinate values further comprises:

when the distance between the multiple points is less than the predetermined distance threshold, selecting a set of candidate coordinate values within the multiple sets of candidate coordinate values as the set of coordinate values within the sets of coordinate values.

12. The method of claim 10, wherein the step of selectively altering the subset of the sets of candidate coordinate values to transform the sets of candidate coordinate values into the sets of coordinate values further comprises:

when the distance between the multiple points is less than the predetermined distance threshold, performing an average calculation upon the multiple sets of candidate coordinate values to generate the set of coordinate values within the sets of coordinate values.

13. The method of claim 12, wherein the average calculation is a weighted average calculation.

14. An apparatus for performing cooperative counting, wherein the apparatus is applicable to a counter system, the counter system comprises a plurality of cameras, and the apparatus comprises:

an interface circuit, arranged to couple to the cameras; and

a processing circuit, coupled to the interface circuit, the processing circuit arranged to control operations of the apparatus, wherein;

the processing circuit sets a plurality of points on an electronic map as a plurality of predetermined points according to user inputs;

the processing circuit determines at least one rule related to the predetermined points according to rule information, wherein the rule information is stored in the counter system;

the processing circuit performs video object detection upon a plurality of images captured by the cameras, respectively, to generate detection results respectively corresponding to the cameras;

the processing circuit merges the detection results respectively corresponding to the cameras, to count events complying with the at least one rule;

wherein the processing circuit performs the video object detection upon the images captured by the cameras to automatically generate a plurality of sets of candidate coordinate values, respectively, wherein image capture ranges of the cameras are mapped to a plurality of areas on the electronic map, respectively, and the sets of candidate coordinate values represent candidate positions of a plurality of video objects on the electronic map, and a subset of the sets of candidate coordinate values is within an overlapping area of at least one portion of the areas;

the processing circuit selectively alters the subset of the sets of candidate coordinate values in order to transform the sets of candidate coordinate values into a plurality of sets of coordinate values, wherein the sets of coordinate values represent positions of the video objects on the electronic map; and

the processing circuit monitors the video objects according to the sets of coordinate values, in order to count the events complying with the at least one rule.

15. The apparatus of claim 14, wherein the video object detection comprises pedestrian detection, and the video objects represent a plurality of pedestrians, wherein the sets of coordinate values represent positions of the pedestrians on the electronic map;

and the processing circuit performs the pedestrian detection upon the images captured by the cameras to automatically generate the sets of candidate coordinate values.

16. The apparatus of claim 15, wherein the predetermined points indicate at least one predetermined line segment on the electronic map;

events complying with the at least one rule comprise an event of a pedestrian passing through the predetermined line segment; and

the processing circuit monitors the pedestrians according to the sets of coordinate values, in order to count the number of pedestrians passing through the predetermined line segment.

17. The apparatus of claim 14, wherein the processing circuit performs video object image recognition upon an image within the images to automatically generate at least one set of image coordinate values in an image coordinate system of the image, wherein the set of image coordinate values indicate positions of at least one of the video objects in the image;

and

the processing circuit performs a coordinate transformation upon the set of image coordinate values according to coordinate transformation reference information, to automatically generate at least one set of candidate coordinate values in the sets of candidate coordinate values, wherein the set of candidate coordinate values represent a candidate position of the video object within the video objects on the electronic map.

18. The apparatus of claim 14, wherein the processing circuit determines whether multiple sets of candidate coordinate values within the sets of candidate coordinate values that belong to different cameras represent a same video object according to a predetermined distance threshold, in order to selectively merge the multiple sets of candidate coordinate values into a set of coordinate values within the sets of coordinate values.