Determination of train presence and motion state in railway environments
First Claim
Patent Images
1. A method for applying computer vision techniques to automatically detect and classify the presence or absence of a train within a railway track area, the method comprising:
- determining foreground feature data for each of a plurality of frames of video input data via applying a background subtraction mask to each of the frames, wherein the video input data is acquired from a region of interest defined around a train track area within an image scene of a stream of the video image data that comprises a portion of a railway track area;
determining for each of the plurality of frames a frame differencing ratio of a value of motion feature data determined for the each frame to a value of motion feature data determined for an immediately previous frame of the plurality of frames;
labeling each of the frames as “
train present”
if the determined foreground feature data value is equal to or is greater than a background subtraction threshold value that is selected as indicative of the presence of a foreground object having the size of a train car, or as “
train absent”
if the determined foreground feature data value is less than the background subtraction threshold value;
labeling each of the frames as “
motion present”
if the determined frame differencing data value is equal to or greater than a motion threshold value that is selected as indicative of a change in the visual features consistent with the movement of a train car within the frame data, or as “
static”
if the determined frame differencing data value is less than the motion threshold value;
classifying a segment of the input video data comprising a group of consecutive video frames that each comprise the “
train absent”
label and the “
static”
label as a “
no train present”
segment;
classifying a segment of the input video data comprising a group of consecutive video frames that each comprise the “
train present”
label and the “
motion present”
label as a “
train present and in transition”
segment;
classifying a segment of the input video data comprising a group of consecutive video frames that each comprise the “
train present”
label and the “
static”
label as a “
train present and stopped”
segment;
determining a presence or a motion state of a train within the region of interest of the video scene at a time of inquiry as the “
no train present”
, the “
train present and in transition”
or the “
train present and stopped”
classification of the segment comprising the video frame of the time of inquiry;
determining a distribution of the “
train present” and
“
train absent”
labels of the plurality of the video segments within an auto-correction time period;
in response to the determined distribution of the “
train present” and
“
train absent”
labels not meeting an expected distribution for the auto-correction time period, exchanging the “
train present” and
the “
train absent”
labels that are determined for each of the video frames; and
repeating the steps of classifying the segments of the input video data of consecutive video frames that each comprise the “
train absent”
label and the “
static”
label into the “
no train present”
segment groups, the segments of the input video data of consecutive video frames that each comprise the “
train present”
label and the “
motion present”
label as the “
train present and in transiton”
segments, and the segments of the input video data of consecutive video frames that each comprise the “
train present”
label and the “
static”
label as the “
train present and stopped”
segment groups, as a function of the exchanged “
train present” and
“
train absent”
labels.
2 Assignments
0 Petitions
Accused Products
Abstract
Foreground feature data and motion feature data is determined for frames of video data acquired from a train track area region of interest. The frames are labeled as “train present” if the determined foreground feature data value meets a threshold value, else as “train absent”; and as “motion present” if the motion feature data meets a motion threshold, else as “static.” The labels are used to classify segments of the video data comprising groups of consecutive video frames, namely as within a “no train present” segment for groups with “train absent” and “static” labels; within a “train present and in transition” segment for groups “train present” and “motion present” labels; and within a “train present and stopped” segment for groups with “train present” and “static” labels. The presence or motion state of a train at a time of inquiry is thereby determined from the respective segment classification.
10 Citations
16 Claims
-
1. A method for applying computer vision techniques to automatically detect and classify the presence or absence of a train within a railway track area, the method comprising:
-
determining foreground feature data for each of a plurality of frames of video input data via applying a background subtraction mask to each of the frames, wherein the video input data is acquired from a region of interest defined around a train track area within an image scene of a stream of the video image data that comprises a portion of a railway track area; determining for each of the plurality of frames a frame differencing ratio of a value of motion feature data determined for the each frame to a value of motion feature data determined for an immediately previous frame of the plurality of frames; labeling each of the frames as “
train present”
if the determined foreground feature data value is equal to or is greater than a background subtraction threshold value that is selected as indicative of the presence of a foreground object having the size of a train car, or as “
train absent”
if the determined foreground feature data value is less than the background subtraction threshold value;labeling each of the frames as “
motion present”
if the determined frame differencing data value is equal to or greater than a motion threshold value that is selected as indicative of a change in the visual features consistent with the movement of a train car within the frame data, or as “
static”
if the determined frame differencing data value is less than the motion threshold value;classifying a segment of the input video data comprising a group of consecutive video frames that each comprise the “
train absent”
label and the “
static”
label as a “
no train present”
segment;classifying a segment of the input video data comprising a group of consecutive video frames that each comprise the “
train present”
label and the “
motion present”
label as a “
train present and in transition”
segment;classifying a segment of the input video data comprising a group of consecutive video frames that each comprise the “
train present”
label and the “
static”
label as a “
train present and stopped”
segment;determining a presence or a motion state of a train within the region of interest of the video scene at a time of inquiry as the “
no train present”
, the “
train present and in transition”
or the “
train present and stopped”
classification of the segment comprising the video frame of the time of inquiry;determining a distribution of the “
train present” and
“
train absent”
labels of the plurality of the video segments within an auto-correction time period;in response to the determined distribution of the “
train present” and
“
train absent”
labels not meeting an expected distribution for the auto-correction time period, exchanging the “
train present” and
the “
train absent”
labels that are determined for each of the video frames; andrepeating the steps of classifying the segments of the input video data of consecutive video frames that each comprise the “
train absent”
label and the “
static”
label into the “
no train present”
segment groups, the segments of the input video data of consecutive video frames that each comprise the “
train present”
label and the “
motion present”
label as the “
train present and in transiton”
segments, and the segments of the input video data of consecutive video frames that each comprise the “
train present”
label and the “
static”
label as the “
train present and stopped”
segment groups, as a function of the exchanged “
train present” and
“
train absent”
labels. - View Dependent Claims (2, 3, 4, 5, 6, 7)
-
-
8. A method for providing a service for applying computer vision techniques to automatically detect and classify the presence or absence of a train within a railway track area, the method comprising:
-
integrating computer-readable program code into a computer system comprising a processing unit, a computer readable memory and a computer readable tangible storage medium, wherein the computer readable program code is embodied on the computer readable tangible storage medium and comprises instructions for execution by the processing unit via the computer readable memory that cause the processing unit to; determine foreground feature data for each of a plurality of frames of video input data via applying a background subtraction mask to each of the frames, wherein the video input data is acquired from a region of interest defined around a train track area within an image scene of a stream of the video image data that comprises a portion of a railway track area; determine for each of the plurality of frames a frame differencing ratio of a value of motion feature data determined for the each frame to a value of motion feature data determined for an immediately previous frame of the plurality of frames; label each of the frames as “
train present”
if the determined foreground feature data value is equal to or is greater than a background subtraction threshold value that is selected as indicative of the presence of a foreground object having the size of a train car, or as “
train absent”
if the determined foreground feature data value is less than the background subtraction threshold value;label each of the frames as “
motion present”
if the determined frame differencing data value is equal to or greater than a motion threshold value that is selected as indicative of a change in the visual features consistent with the movement of a train car within the frame data, or as “
static”
if the determined frame differencing data value is less than the motion threshold value;classify a segment of the input video data comprising a group of consecutive video frames that each comprise the “
train absent”
label and the “
static”
label as a “
no train present”
segment;classify a segment of the input video data comprising a group of consecutive video frames that each comprise the “
train present”
label and the “
motion present”
label as a “
train present and in transition”
segment;classify a segment of the input video data comprising a group of consecutive video frames that each comprise the “
train present”
label and the “
static”
label as a “
train present and stopped”
segment;determine a presence or a motion state of a train within the region of interest of the video scene at a time of inquiry as the “
no train present”
, the “
train present and in transition”
or the “
train present and stopped”
classification of the segment comprising the video frame of the time of inquiry;determine a distribution of the “
train present” and
“
train absent”
labels of the plurality of the video segments within an auto-correction time period;in response to the determined distribution of the “
train present” and
“
train absent”
labels not meeting an expected distribution for the auto-correction time period, exchanging the “
train present” and
the “
train absent”
labels that are determined for each of the video frames; andrepeat the classification of the segments of the input video data of consecutive video frames that each comprise the “
train absent”
label and the “
static”
label into the “
no train present”
segment groups, the segments of the input video data of consecutive video frames that each comprise the “
train present”
label and the “
motion present”
label as the “
train present and in transiton”
segments, and the segments of the input video data of consecutive video frames that each comprise the “
train present”
label and the “
static”
label as the “
train present and stopped”
segment groups, as a function of the exchanged “
train present” and
“
train absent”
labels. - View Dependent Claims (9, 10)
-
-
11. An article of manufacture, comprising:
-
a computer readable hardware storage device having computer readable program code embodied therewith, the computer readable program code comprising instructions for execution by a computer processing unit that cause the computer processing unit to; determine foreground feature data for each of a plurality of frames of video input data via applying a background subtraction mask to each of the frames, wherein the video input data is acquired from a region of interest defined around a train track area within an image scene of a stream of the video image data that comprises a portion of a railway track area; determine for each of the plurality of frames a frame differencing ratio of a value of motion feature data determined for the each frame to a value of motion feature data determined for an immediately previous frame of the plurality of frames; label each of the frames as “
train present”
if the determined foreground feature data value is equal to or is greater than a background subtraction threshold value that is selected as indicative of the presence of a foreground object having the size of a train car, or as “
train absent”
if the determined foreground feature data value is less than the background subtraction threshold value;label each of the frames as “
motion present”
if the determined frame differencing data value is equal to or greater than a motion threshold value that is selected as indicative of a change in the visual features consistent with the movement of a train car within the frame data, or as “
static”
if the determined frame differencing data value is less than the motion threshold value;classify a segment of the input video data comprising a group of consecutive video frames that each comprise the “
train absent”
label and the “
static”
label as a “
no train present”
segment;classify a segment of the input video data comprising a group of consecutive video frames that each comprise the “
train present”
label and the “
motion present”
label as a “
train present and in transition”
segment;classify a segment of the input video data comprising a group of consecutive video frames that each comprise the “
train present”
label and the “
static”
label as a “
train present and stopped”
segment;determine a presence or a motion state of a train within the region of interest of the video scene at a time of inquiry as the “
no train present”
, the “
train present and in transition”
or the “
train present and stopped”
classification of the segment comprising the video frame of the time of inquiry;determine a distribution of the “
train present” and
“
train absent”
labels of the plurality of the video segments within an auto-correction time period;in response to the determined distribution of the “
train present” and
“
train absent”
labels not meeting an expected distribution for the auto-correction time period, exchanged the “
train present” and
the “
train absent”
labels that are determined for each of the video frames; andrepeat the classification of the segments of the input video data of consecutive video frames that each comprise the “
train absent”
label and the “
static”
label into the “
no train present”
segment groups, the segments of the input video data of consecutive video frames that each comprise the “
train present”
label and the “
motion present”
label as the “
train present and in transition”
segments, and the segments of the input video data of consecutive video frames that each comprise the “
train present”
label and the “
static”
label as the “
train present and stopped”
segment groups, as a function of the exchanged “
train present” and
“
train absent”
labels. - View Dependent Claims (12, 13)
-
-
14. A system, comprising:
-
a processing unit; a computer readable memory in communication with the processing unit; and computer-readable storage medium in communication with the processing unit; wherein the processing unit executes program instructions stored on the computer-readable storage medium via the computer readable memory and thereby; determines foreground feature data for each of a plurality of frames of video input data via applying a background subtraction mask to each of the frames, wherein the video input data is acquired from a region of interest defined around a train track area within an image scene of a stream of the video image data that comprises a portion of a railway track area; determines for each of the plurality of frames a frame differencing ratio of a value of motion feature data determined for the each frame to a value of motion feature data determined for an immediately previous frame of the plurality of frames; labels each of the frames as “
train present”
if the determined foreground feature data value is equal to or is greater than a background subtraction threshold value that is selected as indicative of the presence of a foreground object having the size of a train car, or as “
train absent”
if the determined foreground feature data value is less than the background subtraction threshold value;labels each of the frames as “
motion present”
if the determined frame differencing data value is equal to or greater than a motion threshold value that is selected as indicative of a change in the visual features consistent with the movement of a train car within the frame data, or as “
static”
if the determined frame differencing data value is less than the motion threshold value;classifies a segment of the input video data comprising a group of consecutive video frames that each comprise the “
train absent”
label and the “
static”
label as a “
no train present”
segment;classifies a segment of the input video data comprising a group of consecutive video frames that each comprise the “
train present”
label and the “
motion present”
label as a “
train present and in transition”
segment;classifies a segment of the input video data comprising a group of consecutive video frames that each comprise the “
train present”
label and the “
static”
label as a “
train present and stopped”
segment;determines a presence or a motion state of a train within the region of interest of the video scene at a time of inquiry as the “
no train present”
, the “
train present and in transition”
or the “
train present and stopped”
classification of the segment comprising the video frame of the time of inquiry;determines a distribution of the “
train present” and
“
train absent”
labels of the plurality of the video segments within an auto-correction time period;in response to the determined distribution of the “
train present” and
“
train absent”
labels not meeting an expected distribution for the auto-correction time period, exchanges the “
train present” and
the “
train absent”
labels that are determined for each of the video frames; andrepeats the classification of the segments of the input video data of consecutive video frames that each comprise the “
train absent”
label and the “
static”
label into the “
no train present”
segment groups, the segments of the input video data of consecutive video frames that each comprise the “
train present”
label and the “
motion present”
label as the “
train present and in transition”
segments, and the segments of the input video data of consecutive video frames that each comprise the “
train present”
label and the “
static”
label as the “
train present and stopped”
segment groups, as a function of the exchanged “
train present” and
“
train absent”
labels. - View Dependent Claims (15, 16)
-
Specification
-
- Resources
Litigation Campaign Assessment
Thank you for your request. You will receive a custom alert email when the Litigation Campaign Assessment is available.
×
-
Current AssigneeKyndryl Incorporated
-
Original AssigneeInternational Business Machines Corporation
-
InventorsBobbitt, Russell P., Feris, Rogerio S., Zhai, Yun
-
Primary Examiner(s)Koziol, Stephen R
-
Assistant Examiner(s)SCHWARTZ, RAPHAEL M
-
Application NumberUS13/590,269Publication NumberTime in Patent Office1,043 DaysField of SearchNoneUS Class Current1/1CPC Class CodesG06T 7/11 Region-based segmentationG06V 20/41 Higher-level, semantic clus...G06V 20/49 Segmenting video sequences,...G06V 20/52 Surveillance or monitoring ...
