Process for monitoring territories in order to recognise forest and surface fires

• US 8,368,757 B2
• Filed: 10/20/2005
• Issued: 02/05/2013
• Est. Priority Date: 11/22/2004
• Status: Active Grant

• Current Assignee: Deutsches Zentrum Fã¼R Luft- Und Raumfahrt E.V., IQ Wireless GmbH
• Original Assignee: IQ Wireless GmbH
• Inventors: Jock, Andreas, Krane, Uwe, Neuss, Hartmut, Vogel, Holger, Mertens, Volker, Behnke, Thomas, Grser, Gnter, Knollenberg, Jorg, Krt, Ekkehard
• Primary Examiner(s): Nguyen, Thu
• Assistant Examiner(s): TRAN, NAM T
• Application Number:
  US11/791,169
• Publication Number:
  US 20100194893A1
• Time in Patent Office: 2,665 Days
• Field of Search: None
• US Class Current: 348/159
• CPC Class Codes: A62C 3/0271 Detection of area conflagration fires fire alarms for forest fires G08B17/005
• CPC Class Codes: G08B 17/005 for forest fires, e.g. detecting fires spread over a large or outdoors area fire fighting forest fires A62C3/02
• CPC Class Codes: G08B 17/125 by using a video camera to detect fire or smoke

Abstract

Disclosed are processes for the centralised monitoring of territories to recognize forest and surface fires. A swiveling and tiltable camera installed at a monitoring site supplies images of overlapping observation sectors. In each observation sector a sequence of images includes a plurality of images is taken, at an interval which corresponds to fire and smoke dynamics. An on-site image-processing software supplies event warnings with indication of the position of the event site in the analysed image. A total image and an image sequence with image sections of the event site are then transmitted to a central station and reproduced at the central station as a continuous sequence in quick-motion mode. Event warnings with relevant data are blended into electronic maps at the central station. Cross-bearing is made possible by blending event warnings from adjacent monitoring sites. False alarms are minimized by marking known false alarm sources as exclusion zones.

Claims

1. A method of monitoring territories and detecting forest and surface fires with a monitoring system including:

a first complex of means stationed at a minimum of one monitoring site, said complex comprising;

a camera mounted at an elevated location with the ability to tilt and swivel, the horizontal swivel range being at least 360°

, control and evaluation means connected to the camera and running image-processing software for detecting smoke and/or the fire in images from the camera, and having control software, memory for storing events and the images, and an interface to communication means;

a second complex of means installed at a manned central station and comprising a computer including an operating, display and monitoring workplace, control software, memory for the events and the images, means for mixing and outputting the images to at least one monitor, and at least two interfaces to the communication means;

the communication means including;

first bidirectional communication means for image files, data, and voice to interconnect said first and second complexes; and

second bidirectional data and voice communication means to connect said second complex with deployed firefighting crews,

the method comprising;

a) dividing an observation area of the monitoring site into observation sectors each corresponding to a horizontal aperture angle of a lens of the camera;

b) selecting a horizontal angular distance between adjacent observation sectors to create an overlap between them;

c) aiming the camera by positioning means at said observation sectors in automatic succession, or in any order under manual control from the central station;

d) after aiming the camera, providing a plurality of the images timed for adaptation to dynamics of the smoke and the fire;

e) sending the images to a control unit of the monitoring site for storage as an image sequence;

f) processing the images in the control unit of the monitoring site with the image-processing software for detecting the smoke and/or the fire, the image-processing software responding to a presence of the smoke and/or the fire by issuing an event message and data relating to a location and magnitude of the event;

g) if the event message is generated, using the control software of the monitoring site to mark the location of the event in a pertinent one of the images based on the data concerning the location and the magnitude of the event, and to compress the image and to transmit the image to the central station together with an alert message comprising an identity of the monitoring site, an identity of the observation sector, a direction of and an estimated distance to the location of the event;

h) visibly or audibly reproducing the alert message received at the central station, decompressing and storing the image, and displaying the image either automatically or in response to a manual request;

i) at the central station, entering a manual request and communicating the request to the monitoring site, causing the control software at the monitoring site to extract image portions corresponding to the marked location of the event from the images of a current image sequence, to compress the image portions, and to transmit the image portions as an image sequence to the central station;

j) when the image portions corresponding to the marked location of the event are received at the central station, the images portions are decompressed, stored, and displayed as a continuous sequence in a fast-motion display mode, and said sequence is inserted into an overall image, or is displayed in a large-scale format

the method further comprising;

eliminating sources of false alerts including settlements, streets and roads, and surfaces of bodies of water where the smoke may occur by

manually calling up and displaying at the central station images of the observation sectors, or a panoramic image with the marked observation sectors of the monitoring site,

causing the control software to outline by a polygon of a suitable shape the portions of an individual image, or of the panoramic image, which may lead, or have previously led, to other false alerts;

causing the control software of the central station to determine parameters of the polygon and to communicate the parameters as exclusion areas to the control software of the monitoring site;

determining manually at the central station whether event messages pertaining to exclusion areas are to be reported to the central station, and causing the control software at the central station to communicate results of the determining step to the control software of the monitoring site;

in case the image processing software issues the event message, the control software of the monitoring site checking whether the message pertains to a least one of the exclusion areas; and

in case the event message pertains to the exclusion area, the control software of the monitoring site proceeding if instructed to report the event messages to the central station, but without assigning an alert status to the event messages.

2. The method as in claim 1, in the control unit in the monitoring site, the method further comprising:

o) cropping the image vertically by removing from its top and/or bottom edges the horizontal image strips not relevant to detecting the forest fires and doing so before communicating the image to the image-processing software;

p) inputting the data-reduced images thus obtained to the image-processing software for detecting the smoke and/or the fire; and

q) inserting into an original image the data on the location and the magnitude of the event returned by the image-processing software, taking manipulations of step (o) into account.

3. The method as in claim 2, the method further comprising:

predefining the step of cropping the image vertically for each one of the observation sectors.

4. The method as in claim 3, the method further comprising;

combining the step of cropping the image vertically with a different camera tilt for each one of the observation sectors.

5. The method as in claim 2, the method further comprising;

combining the step of cropping the image vertically with a different camera tilt for each one of the observation sectors.

6. The method as in claim 2, the method further comprising:

r) using the operating, display and monitoring workplace of the computer unit to manually call up the images from the observation sectors, or a panoramic image with the observation sectors marked;

s) entering measures for a vertical image crop and a tilt of the camera defined for each of the observation sectors by means of the control software into the images of the individual observation sectors or into the panoramic image;

t) using the control software for determining parameters of the entered measures and transmitting the entered measures to the control software of the monitoring site;

u) repeating the step (r) to check the measures of steps (s) and (t) for correctness and repeating the steps (s) and (t) to increase precision.

7. The method as in claim 1, at the central station, the method further comprising:

r) using the operating, display and monitoring workplace of the computer unit to manually call up the images from the observation sectors, or a panoramic image with the observation sectors marked;

s) entering measures for a vertical image crop and a tilt of the camera defined for each of the observation sectors by means of the control software into the images of the individual observation sectors or into the panoramic image;

t) using the control software for determining parameters of the entered measures and transmitting the entered measures to the control software of the monitoring site;

u) is repeated repeating the step (r) to check the measures of steps (s) and (t) for correctness and repeating the steps (s) and (t) to increase precision.

8. The method as in claim 1, wherein the central station has electronic maps and/or digitized and stored aerial photographs of the areas monitored, the method comprising:

displaying a pertinent one of the maps automatically or in response to manual request in response to the message received at the central station, and automatically inserting into the pertinent map the data comprising the identity of the monitoring station, the observation sector, the direction, and the estimated distance to the location of the event in a graphic and an alphanumeric data format.

9. The method as in claim 8, at the central station, the method further comprising:

when two or more of the alert messages are received at the same or nearly the same time from adjacent monitoring sites, displaying information contained in all said alert messages on the pertinent map so that a cross bearing can be taken.

10. The method as in claim 9, at the central station, the method further comprising:

v) expanding displayed information can be expanded to the adjacent monitoring sites by zooming and shifting displayed portions of the pertinent map;

w) displaying the adjacent monitoring sites and the observation sectors thereof in response to a manual request;

x) determining from the pertinent map the observation sectors of the adjacent monitoring sites which are relevant to the received messages;

y) manually calling up the current images of the observation sectors of the adjacent monitoring site at the operating, display, and monitoring workplace of the computer unit;

z) visually analyzing the images so obtained for features of the smoke and the fire that the image-processing software failed to identify as an event;

aa) marking the location of a visually detected or suspected event in the image by the control software;

bb) deriving the alert message comprising the identity of the monitoring site by control software; and

cc) subjecting the alert message thus derived to further treatment.

11. The method as in claim 8, at the central station, the method further comprising:

v) expanding displayed information can be expanded to the adjacent monitoring sites by zooming and shifting displayed portions of the pertinent map;

w) displaying the adjacent monitoring sites and the observation sectors thereof in response to a manual request;

x) determining from the pertinent map the observation sectors of the adjacent monitoring sites which are relevant to the received messages;

y) manually calling up the current images of the observation sectors of the adjacent monitoring site at the operating, display, and monitoring workplace of the computer unit;

z) visually analyzing the images so obtained for features of the smoke and the fire that the image-processing software failed to identify as an event;

aa) marking the location of a visually detected or suspected event in the image by the control software;

bb) deriving the alert message comprising the identity of the monitoring site by control software; and

cc) subjecting the alert message thus derived to further treatment.

12. The method as in claim 8, method further comprising:

dd) equipping the deployed firefighting crews with global position determining means;

ee) communicating current positions of the deployed firefighting crews by radio to the central station on an automatic and continuous basis;

ff) upon automatic or manual call-up of the pertinent map, automatically showing the positions of the deployed firefighting crews in a displayed area of the pertinent map in the graphic and the alphanumeric data format.

13. The method as in claim 1, the method further comprising:

dd) equipping the deployed firefighting crews with global position determining means;

ee) communicating current positions of the deployed firefighting crews by radio to the central station on an automatic and continuous basis;

ff) upon automatic or manual call-up of a pertinent map, automatically showing the positions of the deployed firefighting crews in a displayed area of the pertinent map in a graphic and an alphanumeric data format.

14. The method as in claim 13, the method further comprising:

selectively displaying the image and the pertinent map according to a split-screen principle, or separately on two different screens.

15. The method as in claim 1, the method further comprising:

r) using the operating, display and monitoring workplace of the computer unit to manually call up the images from the observation sectors, or a panoramic image with the observation sectors marked;

s) entering measures for a vertical image crop and a tilt of the camera defined for each of the observation sectors by means of the control software into the images of the individual observation sectors or into the panoramic image;

t) using the control software for determining parameters of the entered measures and transmitting the entered measures to the control software of the monitoring site;

u) repeating step (r) to check the measures of steps (s) and (t) for correctness and repeating the steps (s) and (t) to increase precision.

16. The method as in claim 1, wherein when the image is transmitted from the monitoring site to the central station, no data reduction takes place in a horizontal direction.

17. A method of monitoring territories and detecting forest and surface fires with a monitoring system including:

a first complex of means stationed at a minimum of one monitoring site, said complex comprising;

a camera mounted at an elevated location with the ability to tilt and swivel, the horizontal swivel range being at least 360°

, control and evaluation means connected to the camera and running image-processing software for detecting smoke and/or the fire in images from the camera, and having control software, memory for storing events and the images, and an interface to communication means;

a second complex of means installed at a manned central station and comprising a computer including an operating, display and monitoring workplace, control software, memory for the events and the images, means for mixing and outputting the images to at least one monitor, and at least two interfaces to the communication means;

the communication means including;

first bidirectional communication means for image files, data, and voice to interconnect said first and second complexes; and

second bidirectional data and voice communication means to connect said second complex with deployed firefighting crews,

the method comprising;

a) dividing an observation area of the monitoring site into observation sectors each corresponding to a horizontal aperture angle of a lens of the camera,

b) selecting a horizontal angular distance between adjacent observation sectors to create an overlap between them;

c) aiming the camera by positioning means at said observation sectors in automatic succession, or in any order under manual control from the central station;

d) after aiming the camera, providing a plurality of the images timed for adaptation to dynamics of the smoke and the fire;

e) sending the images to a control unit of the monitoring site for storage as an image sequence;

f) processing the images in the control unit of the monitoring site with the image-processing software for detecting the smoke and/or the fire, the image-processing software responding to a presence of the smoke and/or the fire by issuing an event message and data relating to a location and magnitude of the event;

g) if the event message is generated, using the control software of the monitoring site to mark the location of the event in a pertinent one of the images based on the data concerning the location and the magnitude of the event, and to compress the image and to transmit the image to the central station together with an alert message comprising an identity of the monitoring site, an identity of the observation sector a direction of and an estimated distance to the location of the event;

h) visibly or audibly reproducing the alert message received at the central station, decompressing and storing the image, and displaying the image either automatically or in response to a manual request,

i) at the central station, entering a manual request and communicating the request to the monitoring site, causing the control software at the monitoring site to extract image portions corresponding to the marked location of the event from the images of a current image sequence, to compress the image portions, and to transmit the image portions as an image sequence to the central station;

j) when the image portions corresponding to the marked location of the event are received at the central station, the images portions are decompressed, stored, and displayed as a continuous sequence in a fast-motion display mode, and said sequence is inserted into an overall image, or is displayed in a large-scale format, and

in the control unit of the monitoring site, the method further comprising;

k) dividing the image into several horizontal image strips before communicating a video image to the image-processing software;

l) averaging sets of several pixels from the image strips below the horizon, but not including the horizon itself, with a number of pixels so averaged increasing between the image strips in a direction toward a bottom edge of the image;

m) inputting the data-reduced images thus obtained to the image-processing software for detecting the smoke and/or the fire; and

n) de-distorting the data on the location and the magnitude of the event the image-processing software has returned, wherein the de-distorting steps are an inverse of the dividing and averaging steps (k) and (l)

wherein the de-distorting steps are followed by a step of inserting the data into the original image.

18. The method as in claim 17, the method further comprising:

eliminating sources of false alerts including settlements, streets and roads, surfaces of bodies of water, where the smoke or confusing light effects may occur by

gg) manually calling up and displaying at the central station images of the observation sectors, or a panoramic image with the marked observation sectors of the monitoring site,

hh) causing the control software to outline by a polygon of a suitable shape the portions of an individual image, or of the panoramic image, which may lead, or have previously led, to other false alerts;

ii) causing the control software of the central station to determine parameters of the polygon and to communicate the parameters as exclusion areas to the control software of the monitoring site;

jj) determining manually at the central station whether event messages pertaining to exclusion areas are to be reported to the central station, and causing the control software at the central station to communicate results of the determining step to the control software of the monitoring site;

kk) in case the image processing software issues the event message, the control software of the monitoring site checking whether the message pertains to a least one of the exclusion areas; and

ll) in case the event message pertains to the exclusion area, the control software of the monitoring site proceeding, if instructed, reports the event messages to the central station, but without assigning an alert status to the event messages.

19. The method as in claim 17, wherein when the image is transmitted from the monitoring site to the central station, no data reduction takes place in a horizontal direction.

20. A method of monitoring territories and detecting forest and surface fires with a monitoring system including:

a first complex of means stationed at a minimum of one monitoring site, said complex comprising;

a camera mounted at an elevated location with the ability to tilt and swivel, the horizontal swivel range being at least 360°

, control and evaluation means connected to the camera and running image-processing software for detecting smoke and/or the fire in images from the camera, and having control software, memory for storing events and the images, and an interface to communication means;

a second complex of means installed at a manned central station and comprising a computer including an operating, display and monitoring workplace, control software, memory for the events and the images, means for mixing and outputting the images to at least one monitor, and at least two interfaces to the communication means;

the communication means including;

first bidirectional communication means for image files, data, and voice to interconnect said first and second complexes; and

second bidirectional data and voice communication means to connect said second complex with deployed firefighting crews,

the method comprising;

a) dividing an observation area of the monitoring site into observation sectors each corresponding to a horizontal aperture angle of a lens of the camera;

b) selecting a horizontal angular distance between adjacent observation sectors to create an overlap between them;

c) aiming the camera by positioning means at said observation sectors in automatic succession, or in any order under manual control from the central station;

d) after aiming the camera, providing a plurality of the images timed for adaptation to dynamics of the smoke and the fire;

e) sending the images to a control unit of the monitoring site for storage as an image sequence;

f) processing the images in the control unit of the monitoring site with the image-processing software for detecting the smoke and/or the fire, the image-processing software responding to a presence of the smoke and/or the fire by issuing an event message and data relating to a location and magnitude of the event;

g) if the event message is generated, using the control software of the monitoring site to mark the location of the event in a pertinent one of the images based on the data concerning the location and the magnitude of the event, and to compress the image and to transmit the image to the central station together with an alert message comprising an identity of the monitoring site, an identity of the observation sector, a direction of and an estimated distance to the location of the event;

h) visibly or audibly reproducing the alert message received at the central station, decompressing and storing the image, and displaying the image either automatically or in response to a manual request;

i) at the central station, entering a manual request and communicating the request to the monitoring site, causing the control software at the monitoring site to extract image portions corresponding to the marked location of the event from the images of a current image sequence, to compress the image portions, and to transmit the image portions as an image sequence to the central station;

j) when the image portions corresponding to the marked location of the event are received at the central station, the images portions are decompressed, stored, and displayed as a continuous sequence in a fast-motion display mode, and said sequence is inserted into an overall image, or is displayed in a large-scale format,

wherein when the image is transmitted from the monitoring site to the central station, no data reduction takes place in a horizontal direction.