DETECTION OF STATIC OBJECT ON THOROUGHFARE CROSSINGS

• US 20140003724A1
• Filed: 06/28/2012
• Published: 01/02/2014
• Est. Priority Date: 06/28/2012
• Status: Active Grant

• Current Assignee: Kyndryl Incorporated
• Original Assignee: International Business Machines Corporation
• Inventors: Feris, Rogerio S., Zhai, Yun
• Granted Patent: US 9,008,359 B2
• Time in Patent Office: Days
• US Class Current: 382/197
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06T 7/269 using gradient-based methods
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

Foreground object image features are extracted from input video via application of a background subtraction mask, and optical flow image features from a region of the input video image data defined by the extracted foreground object image features. If estimated movement features indicate that the underlying object is in motion, a dominant moving direction of the underlying object is determined. If the dominant moving direction is parallel to an orientation of the second, crossed thoroughfare, an event alarm indicating that a static object is blocking travel on the crossing second thoroughfare is not generated. If the estimated movement features indicate that the underlying object is static, or that its determined dominant moving direction is not parallel to the second thoroughfare, an appearance of the foreground object region is determined and a static-ness timer run while the foreground object region comprises the extracted foreground object image features.

Claims

1. A method for applying computer vision techniques to automatically detect the presence of static objects within a first thoroughfare crossing of a second thoroughfare, the method comprising:

in response to input video image data from a scene of a level-grade crossing of a first thoroughfare by a second thoroughfare, a processing unit extracting and recording into a tangible data storage medium foreground object image features via application of a background subtraction mask, and optical flow image features from a region of the input video image data defined by the extracted foreground object image features;

the processing unit determining, by estimating movement features of the foreground object image features, whether an underlying object of the extracted foreground object image features is in motion or is static;

the processing unit determining a dominant moving direction of the underlying object as a function of the extracted optical flow image features if the estimated movement features indicate that the underlying object is in motion;

the processing unit resetting the process and not generating an event alarm if the determined dominant moving direction of the underlying object is parallel to an orientation of the second thoroughfare that is crossed by the first thoroughfare; and

if the estimated movement features indicate that the underlying object is static, or that the object is in motion and the determined dominant moving direction of the underlying object is not parallel to the orientation of the second thoroughfare that is crossed by the first thoroughfare, the processing unit;

determining an initial appearance of the foreground object region;

starting and running a static-ness timer that runs while the foreground object region comprises the extracted foreground object image features;

generating the event alarm if the started static-ness timer reaches a threshold while the determined appearance of the foreground object region comprises the extracted foreground object image features; and

wherein the event alarm indicates that a static object is blocking travel on the crossing second thoroughfare.

2. The method of claim 1, wherein the steps of the processing unit determining the appearance of the foreground object region and starting and running the static-ness timer further comprises:

extracting pre-recorded static region appearance data for the region of the input video image data defined by the extracted foreground object image features from the foreground object image features and optical flow image features recorded into the tangible data storage medium;

using the extracted pre-recorded static region appearance data to determine a current appearance for the region of the input video image data defined by the extracted foreground object image features; and

keeping the started timer running if the extracted foreground object image features are learned into background while the timer is running and the current region appearance matches the initial region appearance within a distance measure threshold.

3. The method of claim 2, wherein the step of the processing unit estimating the movement features of the foreground object image features comprises:

collecting optical flow statistics from frame data for the foreground object image features in a histogram comprising a first bin accumulating a frequency of pixels with zero optical flow magnitudes that represents a static area of the underlying object, and a plurality of other bins that each accumulate frequencies of pixels that possess optical flows pointing in each of a plurality of different, respective directions;

determining that the underlying object of the extracted foreground object image features is static if an area of the foreground region comprising the object that is represented by the first bin accumulated frequencies is at least about the same as a total of the areas represented by the accumulated frequencies of the plurality of the other different, respective direction bins; and

determining that the underlying object of the extracted foreground object image features is in motion if the foreground region area represented by the accumulated first bin frequencies is less than the total of the areas represented by the accumulated frequencies of the plurality of the other different, respective directions bins.

4. The method of claim 3, wherein the dominant moving direction of the underlying object is determined as an arithmetic mean of the accumulated frequencies of the plurality of the other different, respective direction bins.

5. The method of claim 4, wherein the determining the appearance of the region of the input video image data defined by the extracted foreground data comprises implementing and applying a color histogram over the area of the foreground region represented by the accumulated static frequencies of the first bin.

6. The method of claim 5, wherein the step of the processing unit extracting the foreground blob region object image features via application of the background subtraction mask comprises determining background subtraction mask features based on a Gaussian Mixture Model method.

7. The method of claim 5, wherein the processing unit extracting the optical flow image features comprises determining variational flow optical flow features for pixels within the region of the input video image data defined by the extracted foreground object image features.

8. The method of claim 5, further comprising, prior to the step of the processing unit determining whether the underlying object of the extracted foreground object image features is in motion or is static by estimating the movement features of the foreground object image features, if a foreground blob defined from the foreground object image features extracted by the background subtraction mask has a size or an aspect ratio that does not match a predetermined size or aspect ratio value:

filtering out the foreground object image features;

resetting the process; and

not generating the event alarm.

9. The method of claim 5, further comprising:

integrating computer-readable program code into a computer system comprising a processing unit, a computer readable memory and a computer readable tangible storage medium, wherein the computer readable program code is embodied on the computer readable tangible storage medium and comprises instructions that, when executed by the processing unit via the computer readable memory, cause the processing unit to perform the steps of extracting and recording into the tangible data storage medium the foreground object image features and the optical flow image features, determining whether the underlying object of the extracted foreground object image features is in motion or is static, determining the dominant moving direction of the underlying object in motion, resetting the process and not generating an event alarm if the determined dominant moving direction of the underlying object in motion is parallel to the orientation of the second thoroughfare that is crossed by the first thoroughfare, and determining the initial appearance of the foreground object region, starting and running the static-ness timer and generating the event alarm for the static underlying object and the object in motion having the determined dominant moving direction that is not parallel to the orientation of the second thoroughfare that is crossed by the first thoroughfare.

10. A system, comprising:

a processing unit in communication with a computer readable memory and a tangible computer-readable storage medium;

wherein the processing unit, when executing program instructions stored on the tangible computer-readable storage medium via the computer readable memory;

in response to input video image data from a scene of a level-grade crossing of a first thoroughfare by a second thoroughfare, extracts and records into a tangible data storage medium foreground object image features via application of a background subtraction mask, and optical flow image features from a region of the input video image data defined by the extracted foreground object image features;

determines, by estimating movement features of the foreground object image features, whether an underlying object of the extracted foreground object image features is in motion or is static;

determines a dominant moving direction of the underlying object as a function of the extracted optical flow image features if the estimated movement features indicate that the underlying object is in motion;

resets the process and does not generate an event alarm if the determined dominant moving direction of the underlying object is parallel to an orientation of the second thoroughfare that is crossed by the first thoroughfare; and

if the estimated movement features indicate that the underlying object is static, or that the object is in motion and the determined dominant moving direction of the underlying object is not parallel to the orientation of the second thoroughfare;

determines an initial appearance of the foreground object region;

starts and runs a static-ness timer that runs while the foreground object region comprises the extracted foreground object image features; and

generates the event alarm if the started static-ness timer reaches a threshold while the determined appearance of the foreground object region comprises the extracted foreground object image features; and

wherein the event alarm indicates that a static object is blocking travel on the crossing second thoroughfare.

11. The system of claim 10, wherein the processing unit, when executing the program instructions stored on the computer-readable storage medium via the computer readable memory, determines the appearance of the foreground object region and starts and runs the static-ness timer by further:

extracting pre-recorded static region appearance data for the region of the input video image data defined by the extracted foreground object image features from the foreground object image features and optical flow image features recorded into the tangible data storage medium;

using the extracted pre-recorded static region appearance data to determine a current appearance for the region of the input video image data defined by the extracted foreground object image features; and

keeping the started timer running if the extracted foreground object image features are learned into background while the timer is running and the current region appearance matches the initial region appearance within a distance measure threshold.

12. The system of claim 11, wherein the processing unit, when executing the program instructions stored on the computer-readable storage medium via the computer readable memory, further estimates the movement features of the foreground object image features by:

collecting accumulative optical flow statistics from frame data for the foreground object image features in a histogram comprising a first bin accumulating a frequency of pixels with zero optical flow magnitudes that represents a static area of the underlying object, and a plurality of other bins that each accumulate frequencies of pixels that possess optical flows pointing in each of a plurality of different, respective directions;

determining that the underlying object of the extracted foreground object image features is static if an area of the foreground region comprising the object that is represented by the first bin accumulated frequencies is at least about the same as a total of the areas represented by the accumulated frequencies of the plurality of the other different, respective direction bins; and

determining that the underlying object of the extracted foreground object image features is in motion if the foreground region area represented by the first bin accumulated frequencies is less than the total of the areas represented by the accumulated frequencies of the plurality of the other different, respective direction bins.

13. The system of claim 12, wherein the processing unit, when executing the program instructions stored on the computer-readable storage medium via the computer readable memory, determines the dominant moving direction of the underlying object as an arithmetic mean of the accumulated frequencies of the plurality of the other different, respective direction bins.

14. The system of claim 13, wherein the processing unit, when executing the program instructions stored on the computer-readable storage medium via the computer readable memory, prior to determining whether the underlying object of the extracted foreground object image features is in motion or is static by estimating the movement features of the foreground object image features, if a foreground blob defined from the foreground object image features extracted by the background subtraction mask has a size or an aspect ratio that does not match a predetermined size or aspect ratio value, further:

filters out the foreground object image features;

resets the process; and

does not generate the event alarm.

15. The system of claim 14, wherein the processing unit, when executing the program instructions stored on the computer-readable storage medium via the computer readable memory, extracts the optical flow image features by determining variational flow optical flow features for pixels within the region of the input video image data defined by the extracted foreground object image features.

16. An article of manufacture, comprising:

a computer readable tangible storage medium having computer readable program code embodied therewith, the computer readable program code comprising instructions that, when executed by a computer processing unit, cause the computer processing unit to;

in response to input video image data from a scene of a level-grade crossing of a first thoroughfare by a second thoroughfare, extract and record into a tangible data storage medium foreground object image features via application of a background subtraction mask, and optical flow image features from a region of the input video image data defined by the extracted foreground object image features;

determine, by estimating movement features of the foreground object image features, whether an underlying object of the extracted foreground object image features is in motion or is static;

determine a dominant moving direction of the underlying object as a function of the extracted optical flow image features if determined that the underlying object is in motion;

reset the process and not generate an event alarm if the determined dominant moving direction of the underlying object is parallel to an orientation of the second thoroughfare that is crossed by the first thoroughfare; and

if determined that the underlying object is static, or that the object is in motion and the determined dominant moving direction of the underlying object is not parallel to the orientation of the second thoroughfare;

determine an initial appearance of the foreground object region;

start and run a static-ness timer that runs while the foreground object region comprises the extracted foreground object image features;

generate the event alarm if the started static-ness timer reaches a threshold while the determined appearance of the foreground object region comprises the extracted foreground object image features; and

wherein the event alarm indicates that a static object is blocking travel on the crossing second thoroughfare.

17. The article of manufacture of claim 16, wherein the computer readable program code instructions, when executed by the computer processing unit, further cause the computer processing unit to determine the appearance of the foreground object region and start and run the static-ness timer by further:

extracting pre-recorded static region appearance data for the region of the input video image data defined by the extracted foreground object image features from the foreground object image features and optical flow image features recorded into the tangible data storage medium;

using the extracted pre-recorded static region appearance data to determine a current appearance for the region of the input video image data defined by the extracted foreground object image features; and

keeping the started timer running if the extracted foreground object image features are learned into background while the timer is running and the current region appearance matches the initial region appearance within a distance measure threshold.

18. The article of manufacture of claim 17, wherein the computer readable program code instructions, when executed by the computer processing unit, further cause the computer processing unit to estimate the movement features of the foreground object image features by:

collecting accumulative optical flow statistics from frame data for the foreground object image features in a histogram comprising a first bin accumulating a frequency of pixels with zero optical flow magnitudes that represents a static area of the underlying object, and a plurality of other bins that each accumulate frequencies of pixels that possess optical flows pointing in each of a plurality of different, respective directions;

determining that the underlying object of the extracted foreground object image features is static if an area of the foreground region comprising the object that is represented by the first bin accumulated frequencies is at least about the same as a total of the areas represented by the accumulated frequencies of the plurality of the other different, respective direction bins; and

determining that the underlying object of the extracted foreground object image features is in motion if the foreground region area represented by the first bin accumulated frequencies is less than the total of the areas represented by the accumulated frequencies of the plurality of the other different, respective directions bins.

19. The article of manufacture of claim 18, wherein the computer readable program code instructions, when executed by the computer processing unit, further cause the computer processing unit to determine the dominant moving direction of the underlying object as an arithmetic mean of the accumulated frequencies of the plurality of the other different, respective direction bins.

20. The article of manufacture of claim 19, wherein the computer readable program code instructions, when executed by the computer processing unit, further cause the computer processing unit to, prior to determining whether the underlying object of the extracted foreground object image features is in motion or is static by estimating the movement features of the foreground object image features, if a foreground blob defined from the foreground object image features extracted by the background subtraction mask has a size or an aspect ratio that does not match a predetermined size or aspect ratio value:

filter out the foreground object image features;

reset the process; and

not generate the event alarm.