METHODS AND APPARATUS TO IMPROVE EFFICIENCY OF OBJECT TRACKING IN VIDEO FRAMES

• US 20200134837A1
• Filed: 12/19/2019
• Published: 04/30/2020
• Est. Priority Date: 12/19/2019
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Varadarajan, Srenivas, Murthy, Girish Srinivasa, Bodas, Anand, Tickoo, Omesh, Somayazulu, Vallabhajosyula
• Granted Patent: US 11,430,136 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/2113 by ranking or filtering the set of features, e.g. using a measure of variance or of feature cross-correlation
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/50 by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
• CPC Class Codes: G06V 10/62 relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
• CPC Class Codes: G06V 10/771 Feature selection, e.g. selecting representative features from a multi-dimensional feature space
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49

Abstract

Methods, apparatus, systems and articles of manufacture to improve efficiency of object tracking in video frames are disclosed. An example apparatus includes a clusterer to cluster a map of a video frame into blobs; a comparator to determine an intersection over union value between the blobs and bounding boxes in a second video frame; and an interface to initiate object detection by a neural network on the first video frame when the intersection over union does not satisfy a threshold.

Claims

1. An apparatus comprising:

a clusterer to cluster pixels of a foreground map of a first video frame into blobs;

a comparator to determine an intersection over union value between the blobs and bounding boxes in a second video frame; and

an interface to initiate object detection by a neural network on the first video frame when the intersection over union value does not satisfy a threshold.

2. The apparatus of claim 1, wherein the intersection over union value does not satisfy the threshold when a new object is not present in the second video frame is present in the first video frame.

3. The apparatus of claim 1, further including:

a bitstream analyzer to determine a global motion vector of the first video frame; and

a subtractor to generate the foreground map by subtracting the global motion vector from the first video frame.

4. The apparatus of claim 3, wherein the bitstream analyzer is to determine the global motion vector from a bitstream of at least one of the first video frame or a video stream including the first video frame.

5. The apparatus of claim 3, wherein the global motion vector corresponds to movement of a camera.

6. The apparatus of claim 1, further including a filter to remove a subset of the blobs that are below a threshold size.

7. The apparatus of claim 1, wherein the clusterer is to cluster a subsequent map of a third video frame into subsequent blobs when the intersection over union value satisfies the threshold, the third video frame subsequent to the first video frame.

8. The apparatus of claim 7, wherein the clusterer to cluster the subsequent map of the third video frame into the subsequent blobs without the object detection being performed on the first video frame.

9. The apparatus of claim 1, further including:

a counter to increment a frame count when the first video frame is obtained;

a count comparator to compare the frame count to an object detection count; and

the interface to initiate the object detection of the first video frame with the neural network when the frame count is greater than or equal to the object detection count, the neural network to generate one or more bounding boxes based on the first video frame, the counter to be reset when the frame count is greater than or equal to the object detection count.

10. The apparatus of claim 9, wherein:

the comparator is to compare a first number of the one or more bounding boxes based on the first video frame to a second number of the bounding boxes of the second video frame that have been processed by the neural network; and

the counter is to decrease the object detection count when the first number is greater than the second number.

11. The apparatus of claim 9, wherein the comparator is to (A) pair the one or more bounding boxes of the first video frame with the bounding boxes of the second video frame and (B) determine one or more intersection over union values between the pairs, and further including:

an averager to average the one or more intersection over union values, the counter to adjust the object detection count based on the average.

12. The apparatus of claim 11, wherein the threshold is a first threshold, the counter to increase the object detection count when the average is greater than a second threshold and decrease the object detection count when the average is less than the threshold.

13. The apparatus of claim 12, wherein the counter is to decrease the object detection count to a base number when the average is less than the threshold.

14. The apparatus of claim 9, wherein the comparator is to (A) pair the one or more bounding boxes of the first video frame with the bounding boxes of the second video frame and (B) determine one or more intersection over union values between the pairs, the counter to adjust the object detection count based on the one or more intersection over union values between the pairs.

15. A non-transitory computer readable storage medium comprising instructions which, when executed, cause a machine to at least:

group pixels a map of a first video frame into blobs;

determine an intersection over union value between the blobs and bounding boxes in a second video frame; and

provide the first video frame to a neural network for object detection when the intersection over union value does not satisfy a threshold.

16. The non-transitory computer readable storage medium of claim 15, wherein the instructions cause the machine to:

increment a frame count when the first video frame is obtained;

compare the frame count to an object detection count; and

initiate the object detection of the first video frame with the neural network when the frame count is greater than or equal to the object detection count, the neural network to generate one or more bounding boxes based on the video frame, the frame count to be reset when the frame count is greater than or equal to the object detection count.

17. The non-transitory computer readable storage medium of claim 16, wherein the instruction cause the machine to:

compare a first number of the one or more bounding boxes based on the first video frame to a second number of the bounding boxes of the second video frame that have been processed by the neural network; and

decrease the object detection count when the first number is greater than the second number.

18. The non-transitory computer readable storage medium of claim 16, wherein the instruction cause the machine to:

pair the one or more bounding boxes of the first video frame with the bounding boxes of the second video frame;

determine one or more intersection over union values between the pairs; and

average the one or more intersection over union values, the adjusting of the object detection count being based on the average.

19. The non-transitory computer readable storage medium of claim 18, wherein the threshold is a first threshold, wherein the instruction cause the machine to increase the object detection count when the average a second threshold and decrease the object detection count when the average is less than the threshold.

20. A method comprising:

generating, by executing an instruction with a processor, blobs from a map of a first video frame;

calculating, by executing an instruction with the processor, an intersection over union value between the blobs and bounding boxes in a second video frame; and

instructing a neural network to perform object detection on the first video frame when the intersection over union value does not satisfy a threshold.