Operations Monitoring in an Area

• US 20140339374A1
• Filed: 05/16/2014
• Published: 11/20/2014
• Est. Priority Date: 05/17/2013
• Status: Active Grant

• Current Assignee: International Electronic Machines Corp.
• Original Assignee: International Electronic Machines Corp.
• Inventors: Mian, Zahid F., Gamache, Ronald W., Spoor, Ryk E.
• Granted Patent: US 10,202,135 B2
• Time in Patent Office: Days
• US Class Current: 246/473.1
• CPC Class Codes: B61L 23/041 Obstacle detection
• CPC Class Codes: B61L 29/30 Supervision, e.g. monitoring arrangements
• CPC Class Codes: G06F 18/256 of results relating to different input data, e.g. multimodal recognition
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06V 10/811 the classifiers operating on different input data, e.g. multi-modal recognition
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G08B 13/1436 with motion detection
• CPC Class Codes: G08G 1/005 including pedestrian guidance indicator
• CPC Class Codes: G08G 1/017 identifying vehicles G08G1/015, G08G1/054 take precedence
• CPC Class Codes: G08G 1/0175 by photographing vehicles, e.g. when violating traffic rules
• CPC Class Codes: G08G 1/04 : using optical or ultrasonic detectors
• CPC Class Codes: G08G 1/166 : for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

An assembly for monitoring an area is provided. The assembly can include two or more cameras sensitive to radiation of distinct wavelength ranges. The fields of view of the cameras can be substantially co-registered at the area to be monitored. The assembly can include a computer system which can process the image data to monitor the area. The computer system can be configured to identify relevant objects present in the area, update tracking information for the relevant objects, and evaluate whether an alert condition is present using the tracking information.

Claims

1. A system comprising:

a monitoring assembly deployed to monitor an area, wherein the monitoring assembly includes;

a plurality of cameras including at least two cameras sensitive to radiation of distinct wavelength ranges, wherein the fields of view of the at least two cameras are substantially co-registered at the area; and

a computer system configured to monitor the area by performing a method including;

operating the plurality of cameras to substantially concurrently acquire image data of the area;

identifying a set of relevant objects in the area using the image data acquired by the plurality of cameras;

updating tracking information for each object in the set of relevant objects; and

evaluating an alert condition in response to the tracking information for each of the set of relevant objects.

2. The system of claim 1, wherein the identifying includes:

independently processing the image data acquired by each of the plurality of cameras to identify objects present in the area;

fusing similar objects identified in the image data acquired by each of the plurality of cameras; and

verifying objects identified in the image data acquired by only a portion of the plurality of cameras.

3. The system of claim 1, wherein the identifying a set of relevant objects includes classifying objects identified in the area as being relevant or not relevant based on a physical size of the object.

4. The system of claim 1, wherein the identifying a set of relevant objects includes classifying objects identified in the area as being relevant or not relevant based on movement of the object.

5. The system of claim 1, wherein the computer system is further configured to self-calibrate using a set of static features in the area.

6. The system of claim 5, wherein the set of static features includes at least one of:

rails or ties for railroad tracks.

7. The system of claim 1, wherein the evaluating an alert condition includes evaluating a presence of an object in a restricted area using at least one of:

tracking data for the object or tracking data for another object.

8. The system of claim 7, wherein the evaluating further uses an amount of time to a critical event.

9. The system of claim 8, wherein the restricted area comprises an at-grade railroad crossing and the critical event comprises a train using the at-grade railroad crossing.

10. The system of claim 1, wherein the plurality of distinct wavelength ranges includes:

near infrared and visible radiation and infrared radiation, the monitoring assembly further comprising a near infrared illuminator, wherein the method further includes operating the near infrared illuminator to provide sufficient light for imaging the area with the first camera.

11. A method comprising:

acquiring multispectral image data of an area on a computer system of a monitoring assembly including a plurality of cameras including at least two cameras sensitive to radiation of distinct wavelengths;

the computer system identifying a set of relevant objects in the area using the multispectral image data;

the computer system updating tracking information for each object in the set of relevant objects;

the computer system evaluating an alert condition in response to the tracking information for each of the set of relevant objects; and

the computer system performing an action response to evaluating a presence of an alert condition.

12. The method of claim 11, further comprising the computer system self-calibrating and registering two-dimensional pixel data of the area into a three-dimensional model of the area using a plurality of static features in the area.

13. The method of claim 12, wherein the plurality of static features include at least one of:

rails or ties for railroad tracks in the area.

14. The method of claim 11, wherein the identifying a set of relevant objects includes classifying objects identified in the area as being relevant or not relevant based on a physical size of the object.

15. The method of claim 11, wherein the identifying a set of relevant objects includes classifying objects identified in the area as being relevant or not relevant based on movement of the object.

16. The method of claim 11, wherein the evaluating an alert condition includes evaluating a presence of an object in a restricted area using at least one of:

tracking data for the object or tracking data for another object.

17. An at-grade railroad crossing comprising:

a road intersecting with railroad tracks; and

a monitoring assembly deployed to monitor the railroad crossing, wherein the monitoring assembly includes;

a first camera sensitive to near infrared and visible radiation;

a second camera sensitive to infrared radiation, wherein the fields of view of the first and second cameras are substantially co-registered at the intersection of the road and the railroad tracks; and

a computer system configured to monitor the railroad crossing by performing a method including;

identifying a set of relevant objects in the area using image data substantially concurrently acquired by the first and second cameras;

updating location and tracking information for each object in the set of relevant objects; and

evaluating an alert condition using the location and tracking information for each of the set of relevant objects and an amount of time until a train next uses the railroad crossing.

18. The crossing of claim 17, wherein the updating uses a registration map mapping the image data to a three-dimensional model of the railroad crossing.

19. The crossing of claim 17, wherein the identifying a set of relevant objects includes:

identifying all objects visible in the image data;

determining a size and orientation of each object visible in the image data; and

classifying objects as not relevant based on at least one of;

the size and orientation of the object or tracking information corresponding to movement of the object.

20. The crossing of claim 17, the monitoring assembly further comprising a near infrared illuminator, wherein the method further includes operating the near infrared illuminator to provide sufficient light for imaging the railroad crossing with the first camera.