DETECTED OBJECT TRACKER FOR A VIDEO ANALYTICS SYSTEM

• US 20190012761A1
• Filed: 03/23/2018
• Published: 01/10/2019
• Est. Priority Date: 09/23/2015
• Status: Active Grant

• Current Assignee: Intellective Ai, Inc.
• Original Assignee: Omni AI, Inc.
• Inventors: RISINGER, Lon W., SAITWAL, Kishor Adinath, COBB, Wesley Kenneth
• Granted Patent: US 10,679,315 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats

Abstract

Techniques are disclosed which provide a detected object tracker for a video analytics system. As disclosed, the detected object tracker provides a robust foreground object tracking component for a video analytics system which allow other components of the video analytics system to more accurately evaluate the behavior of a given object (as well as to learn to identify different instances or occurrences of the same object) over time. More generally, techniques are disclosed for identifying what pixels of successive video frames depict the same foreground object. Logic implementing certain functions of the detected object tracker can be executed on either a conventional processor (e.g., a CPU) or a hardware acceleration processing device (e.g., a GPU), allowing multiple camera feeds to be evaluated in parallel.

Claims

17-61. -61. (canceled)

62. A computer-implemented method for tracking foreground objects depicted in a video scene, the method comprising:

receiving via at least one processor, a background/foreground BG/FG segmentation of a current video frame from a plurality of video frames of the video scene, the current video frame including at least one appearance value for each of a plurality of pixels and wherein the BG/FG segmentation classifies at least one pixel in the plurality of the pixels as depicting at least one of scene foreground or scene background;

for each region of pixels in the current frame classified as depicting the video scene foreground, determining via the at least one processor, an ellipse to bound that region;

comparing via the at least one processor, a geometry of at least a first one of the ellipses in the current frame with a geometry of a second ellipse in at least a prior frame from the plurality of video frames;

classifying via the at least one processor, the first ellipse as corresponding to a first known foreground object tracked in at least the prior frame based at least in part on the comparison;

extending via the at least one processor, a trajectory of the first known foreground object based on the at least one appearance value of the pixels bound by the first ellipse to update at least one of size, orientation, and motion data of the first foreground object; and

tracking via the at least one processor, the first foreground object based at least in part on the trajectory.

63. The method of claim 62, wherein classifying the first ellipse as corresponding to the first known foreground object comprises matching at least one of a size, orientation, and location of the first ellipse with the second ellipse, the second ellipse being generated for the first known foreground object tracked in at least the prior frame.

64. The method of claim 62, wherein classifying the first ellipse as corresponding to the first known foreground object further comprises:

matching at least one of a size, orientation, and location of the first ellipse with the second ellipse, the second ellipse being generated for the first known foreground object tracked in at least the prior frame; and

comparing a weighted color distribution (WCD) generated using the appearance values of pixels bound by the first ellipse with a WCD generated using appearance values of pixels bound by the second ellipse generated for the first known foreground object tracked in at least the prior frame.

65. The method of claim 62, wherein extending the trajectory of the first known foreground object comprises generating at least one of appearance data, position data, velocity data, and direction data.

66. The method of claim 62, further comprising, classifying at least a third ellipse as depicting a new foreground object within the video scene.

67. The method of claim 62, further comprising:

classifying at least a third ellipse as depicting a new foreground object within the video scene; and

initializing a trajectory for the new foreground object.

68. The method of claim 62, further comprising:

classifying at least a third ellipse as depicting a new foreground object within the video scene; and

upon determining a size and orientation of a geometry of the third ellipse matches a geometry of an ellipse in a missing set;

reclassifying the third ellipse as corresponding to a second known foreground object; and

extending a trajectory of the second known foreground object based on the appearance values of the pixels bound by the third ellipse.

69. The method of claim 62, further comprising:

identifying an ellipse corresponding to a foreground object tracked in a prior frame, wherein the identified ellipse was not matched to any of the ellipses determined for the current frame; and

adding the identified ellipse to a set of missing ellipses.

70. A non-transitory computer-readable storage medium containing a program, which when executed on a processor, performs an operation for tracking foreground objects depicted in a video scene, the operation comprising:

receiving a background/foreground BG/FG segmentation of a current video frame from a plurality of video frames of the video scene, the current video frame including at least one appearance value for each of a plurality of pixels and wherein the BG/FG segmentation classifies at least one pixel in the plurality of the pixels as depicting either scene foreground or scene background;

for each region of pixels in the current frame classified as depicting the video scene foreground, determining an ellipse to bound that region;

comparing a geometry of at least a first one of the ellipses in the current frame with a geometry of a second ellipse in at least a prior frame from the plurality of video frames;

classifying the first ellipse as corresponding to a first known foreground object tracked in at least the prior frame based at least in part on the comparison;

extending a trajectory of the first known foreground object based on the at least one appearance value of the pixels bound by the first ellipse to update at least one of size, orientation, and motion data of the first foreground object; and

tracking the first foreground object based at least in part on the trajectory .

71. The non-transitory computer-readable storage medium of claim 70, wherein classifying the first ellipse as corresponding to the first known foreground object comprises matching at least one of a size, orientation, and location of the first ellipse with the second ellipse, the second ellipse generated for the first known foreground object tracked in at least the prior frame.

72. The non-transitory computer-readable storage medium of claim 70, wherein classifying the first ellipse as corresponding to the first known foreground object comprises:

matching at least one of a size, orientation, and location of the first ellipse with the second ellipse, the second ellipse generated for the first known foreground object tracked in at least the prior frame; and

comparing a weighted color distribution (WCD) generated using the appearance values of pixels bound by the first ellipse with a WCD generated using appearance values of pixels bound by the second ellipse generated for the first known foreground object tracked in at least the prior frame.

73. The non-transitory computer-readable storage medium of claim 70, wherein extending the trajectory of the first known foreground object comprises generating at least one of appearance data, position data, velocity data, and direction data.

74. The non-transitory computer-readable storage medium of claim 70, wherein the operation further comprises, classifying at least a third ellipse as depicting a new foreground object within the video scene.

75. The non-transitory computer-readable storage medium of claim 70, wherein the operation further comprises:

classifying at least a third ellipse as depicting a new foreground object within the video scene; and

initializing a trajectory for the new foreground object.

76. The non-transitory computer-readable storage medium of claim 70, wherein the operation further comprises:

identifying an ellipse corresponding to a foreground object tracked in a prior frame, wherein the identified ellipse was not matched to any of the ellipses determined for the current frame; and

adding the identified ellipse to a set of missing ellipses.

77. A system, comprising:

a video input source configured to provide a sequence of video frames, each depicting a video scene;

a central processing unit (CPU); and

a memory containing a program, which, when executed on the CPU is configured to perform an operation for tracking foreground objects depicted in the video scene, the operation comprising;

receiving a background/foreground BG/FG segmentation of a current video frame from a plurality of video frames of the video scene, wherein the current video frame includes at least one appearance value for each of a plurality of pixels and wherein the BG/FG segmentation classifies one or more of the pixels as depicting at least one of scene foreground or scene background,

for each region of pixels in the current frame classified as depicting the video scene foreground, determining an ellipse to bound that region,

comparing a geometry of at least a first one of the ellipses in the current frame with a geometry of a second ellipse in at least a prior frame from the plurality of video frames,

classifying the first ellipse as corresponding to a first known foreground object tracked in at least the prior frame based at least in part on the comparison,

extending a trajectory of the first known foreground object based on the at least one appearance value of the pixels bound by the first ellipse to update at least one of size, orientation, and motion data of the first foreground object, and

tracking the first foreground object based at least in part on the trajectory.

78. The system of claim 77, wherein classifying the first ellipse as corresponding to the first known foreground object comprises matching at least one of a size, orientation, and location of the first ellipse with the second ellipse, the second ellipse being generated for the first known foreground object tracked in at least the prior frame.

79. The system of claim 77, wherein:

classifying the first ellipse as corresponding to the first known foreground object comprises matching at least one of a size, orientation, and location of the first ellipse with the second ellipse, the second ellipse being generated for the first known foreground object tracked in at least the prior frame; and

extending the trajectory of the first known foreground object comprises generating at least one of appearance data, position data, velocity data, and direction data.

80. The system of claim 77, wherein the operation further comprises, classifying at least a third ellipse as depicting a new foreground object within the video scene.

81. The system of claim 77, wherein the operation further comprises:

identifying an ellipse corresponding to a foreground object tracked in a prior frame, wherein the identified ellipse was not matched to any of the ellipses determined for the current frame; and

adding the identified ellipse to a set of missing ellipses.