System and method for automatic calculation of scene geometry in crowded video scenes

• US 9,576,204 B2
• Filed: 03/24/2015
• Issued: 02/21/2017
• Est. Priority Date: 03/24/2015
• Status: Active Grant

• Current Assignee: Qognify Ltd. (PSBU Midco SARL)
• Original Assignee: Qognify Ltd. (PSBU Midco SARL)
• Inventors: Goldner, Vladimir, Gurwicz, Yaniv
• Primary Examiner(s): Liew, Alex
• Application Number:
  US14/666,338
• Publication Number:
  US 20160283798A1
• Time in Patent Office: 700 Days
• US Class Current: 1/1
• CPC Class Codes: G06V 10/50 by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
• CPC Class Codes: G06V 20/53 Recognition of crowd images, e.g. recognition of crowd congestion
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions

Abstract

A system or method for automatic calculation of scene geometry may include receiving, by a processor, a video stream including a plurality of video frames. A video frame may be divided into blocks, and a vertical vanishing point of the video stream may be determined based on a dominant orientation of detected edges in the blocks. A horizon and scale of the video stream may be estimated based on optical flow of pixels between two consecutive video frames. A set of human heads may be detected in a video frame of the video stream based on the estimated horizon and scale of the video stream.

Claims

1. A method for automatic calculation of scene geometry, comprising:

receiving, by a processor, a video stream including a plurality of video frames;

dividing, by the processor, a video frame into blocks;

determining a vertical vanishing point of the video stream based on a dominant orientation of detected edges in the blocks;

estimating a horizon and scale of the video stream based on optical flow of pixels between two consecutive video frames; and

detecting a set of human heads in a video frame of the video stream based on the estimated horizon and scale of the video stream.

2. The method of claim 1, wherein estimating the horizon and scale of the video stream comprises generating a set of estimated human head sizes at different locations within a video frame.

3. The method of claim 1, wherein detecting the set of human heads in the video frame comprises generating a set of candidate points locating a human head'"'"'s center.

4. The method of claim 3, wherein detecting the set of human heads in the video frame further comprises creating a Hough transform using the generated set of candidate points.

5. The method of claim 4, wherein the Hough transform extracts ellipses from the video frame which approximate a human head'"'"'s size.

6. The method of claim 1, comprising adjusting the estimated horizon and scale if a standard deviation of y-coordinates of the detected set of human heads is less than a threshold.

7. The method of claim 6, wherein adjusting the estimated horizon and scale comprises detecting a set of human heads in a second video frame of the video stream using a different estimated horizon and scale.

8. The method of claim 1, wherein estimating the horizon and scale of the video stream is further based on the detected edges in the blocks.

9. A system for automatic calculation of scene geometry, comprising:

a memory; and

a processor configured to;

receive a video stream including a plurality of video frames;

divide a video frame into blocks;

determine a vertical vanishing point of the video stream based on a dominant orientation of edges in the blocks;

estimate a horizon and scale of the video stream based on optical flow of pixels between two consecutive video frames; and

detect a set of human heads in a video frame of the video stream based on the estimated horizon and scale of the video stream.

10. The system of claim 9, wherein the processor is configured to estimate the horizon and scale of the video stream by generating a set of estimated human head sizes at different locations within a video frame.

11. The system of claim 9, wherein the processor is configured to detect the set of human heads in the video frame comprises generating a set of candidate points locating a human head'"'"'s center.

12. The system of claim 11, wherein the processor is configured to detect the set of human heads in the video frame by further creating a Hough transform using the generated set of candidate points.

13. The system of claim 12, wherein the Hough transform extracts ellipses from the video frame which approximate a human head'"'"'s size.

14. The system of claim 9, wherein the processor is configured to adjust the estimated horizon and scale if a standard deviation of y-coordinates of the detected set of human heads is less than a threshold.

15. The system of claim 14, wherein the processor is configured to adjust the estimated horizon and scale by detecting a set of human heads in a second video frame of the video stream using a different estimated horizon and scale.

16. The system of claim 9, wherein the processor is configured to determine a vertical vanishing point of the video stream based on a set of vanishing points, wherein each vanishing point is an intersection between a pair of detected edges in a pair of blocks.

17. The system of claim 9, wherein the processor is configured to detect humans using the vertical vanishing point, horizon, and scale and refine the horizon and scale based on the detected humans.