Method and apparatus of detecting fire by flame imaging

• US 20030044042A1
• Filed: 05/10/2002
• Published: 03/06/2003
• Est. Priority Date: 05/11/2001
• Status: Active Grant

• Current Assignee: Detector Electronics Corporation
• Original Assignee: Detector Electronics Corporation
• Inventors: King, John D., Junck, Paul M.
• Granted Patent: US 7,155,029 B2
• Time in Patent Office: Days
• US Class Current: 382/100
• CPC Class Codes: G08B 17/125 by using a video camera to detect fire or smoke

Abstract

An apparatus and method for performing first and second imaging processes contemporaneously, in particular where one of the processes is detecting fires based on images of the flames. The apparatus includes an image sensor for producing a video image, a frame grabber for capturing first frames and second frames, a processor for processing the data within the frames, and an output device. The apparatus may also include an adjustment mechanism for adjusting the image settings of the image sensor between settings suitable for flame imaging and non-flame imaging, and a control mechanism for controlling the image settings of the image sensor. In the method at least two first frames are obtained, and a plurality of second frames are obtained. The first and second frames are used for first and second processes. The first and second processes are contemporaneous, so that they are carried out within the same time period without interfering with one another. When the first process is flame detection, individual pairs of pixels having a property such as intensity that meets a first threshold are identified within the first frames, and are assembled into blobs. Additional properties of the pixel pairs and the blobs overall are evaluated in relation with additional thresholds. Blobs or pixels that do not meet the thresholds are excluded. Any blobs remaining after all evaluations are considered fires.

Claims

1. Method of using a video sensor, comprising the steps of:

generating a video image with said video sensor;

obtaining at least two first frames from said video image;

obtaining a plurality of second frames from said video image; and

contemporaneously performing a first process using said first frames and a second process using said second frames;

wherein said first process comprises flame detection.

2. Method according to claim 1, wherein:

said first frames and said second frames are exclusive, such that said first frames are obtained from a first portion of said video image, and said second frames are obtained from a second portion of said video image,

wherein said first portion is unsuitable for obtaining said second frames therefrom, and said second portion is unsuitable for obtaining said first frames therefrom.

3. Method according to claim 2, wherein:

said image comprises a plurality of consecutive image frames, said first frames comprising at least two of said image frames, and said second frames comprising a remainder of said image frames.

4. Method according to claim 1, wherein:

said first frames are obtained with different image settings than said second frames.

5. Method according to claim 1, wherein:

said first frames and said second frames are non-exclusive, such that said first frames are obtained from a first portion of said video image, and said second frames are obtained from a second portion of said video image,

wherein said first portion is suitable for obtaining said second frames therefrom, and said second portion is suitable for obtaining said first frames therefrom.

6. Method according to claim 5, wherein:

said image comprises a plurality of consecutive image frames, said first frames comprising at least a first portion of at least two of said image frames, and said second frames comprising at least a second portion of said image frames.

7. Method according to claim 6, wherein:

said image frames have an image dynamic range, said first frames have a first dynamic range comprising at least a first portion of said image dynamic range, and said second frames have a second dynamic range comprising at least a second portion of said image dynamic range.

8. Method according to claim 6, wherein:

said first frames comprise an entirety of at least two of said image frames, and said second frames comprise an entirety of said image frames.

9. Method according to claim 1, wherein:

said video image comprises frames, and said at least two first frames comprise consecutive image frames.

10. Method according to claim 1, wherein:

said video image is a color image.

11. Method according to claim 10, wherein:

said first and second frames are color frames.

12. Method according to claim 1, wherein:

said second process comprises displaying a human-viewable output.

13. Method according to claim 1, wherein:

said second process comprises security monitoring.

14. Method according to claim 1, wherein:

said second process comprises traffic observation.

15. Method according to claim 1, wherein:

said second process comprises smoke detection.

16. Method according to claim 1, wherein:

said at least two first frames comprise a base frame and a comparison frame; and

said first process comprises the steps of;

identifying a plurality of base pixels in said base frame, and a plurality of comparison pixels in said comparison frame, wherein for each base frame pixel there is a spatially corresponding comparison frame pixel, each said base frame pixel and said corresponding comparison frame pixel forming a pair, such that said pluralities of base and comparison image pixels comprise a plurality of pairs.

17. Method according to claim 16, wherein said first process further comprises the steps of:

determining a first property of at least some of said pairs;

categorizing said pairs as blob pairs if said first property meets a first threshold; and

assembling said blob pairs into at least one blob.

18. Method according to claim 1, wherein:

said at least two first frames comprise a base frame and a comparison frame; and

said first process comprises the steps of;

identifying a plurality of base pixels in said base frame, and a plurality of comparison pixels in said comparison frame, wherein for each base frame pixel there is a spatially corresponding comparison frame pixel, each said base frame pixel and said corresponding comparison frame pixel forming a pair, such that said pluralities of base and comparison image pixels comprise a plurality of pairs.

19. Method according to claim 18, wherein said first process further comprises the steps of:

determining a first property of at least some of said pairs;

categorizing said pairs as blob pairs if said first property meets a first threshold;

assembling said blob pairs into at least one blob; and

indicating said at least one blob as a fire.

20. Method according to claim 19, wherein:

said first property is intensity, and said first threshold is a minimum intensity threshold.

21. Method according to claim 19, wherein said first process further comprises the steps of:

determining a second property of said blob pairs; and

excluding said blob pairs from said blob if said second property of said blob pairs does not meet a second threshold.

22. Method according to claim 19, wherein said first process further comprises the steps of:

determining a second property of said blob pairs; and

excluding said blob as a non-fire if said second property of said blob pairs does not meet a second threshold.

23. Method according to claim 21, wherein:

said video image is a color image;

said first and second frames are color frames; and

said second property is color, and said second threshold is a color range.

24. Method according to claim 23, wherein:

said color is measured in YC_RC_B coordinates, and said color range is defined in YC_RC_B coordinates.

25. Method according to claim 21, wherein said first process further comprises the steps of:

determining a third property of said blob pairs;

excluding said blob as a non-fire if said third property of said blob pairs does not meet a third threshold.

26. Method according to claim 25, wherein:

determining said third property comprises determining derivatives of differences in intensity and color between said base pixels and said comparison pixels in said blob pairs.

27. Method according to claim 26, wherein:

determining said third property further comprises;

plotting said derivatives as at least one histogram and an incidence in at least two bands in said at least one histogram.

28. Method according to claim 27, wherein:

determining said third property further comprises;

plotting an incidence from at least one of said at least two bands against an incidence of at least another of said at least two bands as at least one x-y plot.

29. Method according to claim 28, wherein:

determining said third property further comprises;

determining a ratio of a number of points on a first side of a cut-off line on said at least one x-y plot to a number of points not on said first side of said cut-off line; and

said third property comprises said ratio from said at least one x-y plot.

30. Method according to claim 25, wherein:

said second property is color;

said second threshold is a color range;

said color is measured in YC_RC_B coordinates;

said color range is defined in YC_RC_B coordinates; and

said derivatives comprise

$\frac{<br><br>Y}{<br><br>t},\frac{<br><br>Y}{<br><br>x},\frac{<br><br>C_R}{<br><br>t},\frac{<br><br>C_R}{<br><br>x},\frac{<br><br>C_B}{<br><br>t},\mathrm{and}<br><br>\text{\hspace{1em}<br><br>}<br><br>\frac{<br><br>C_B}{<br><br>x},\mathrm{wherein}$

$\frac{<br><br>Y}{<br><br>t}$

is a derivative of intensity with respect to time;

$\frac{<br><br>Y}{<br><br>x}$

is a derivative of intensity with respect to position;

$\frac{<br><br>C_R}{<br><br>t}$

is a derivative of red chrominance with respect to time;

$\frac{<br><br>C_R}{<br><br>x}$

is a derivative of red chrominance with respect to position;

$\frac{<br><br>C_B}{<br><br>t}$

is a derivative of blue chrominance with respect to time;

$\frac{<br><br>C_B}{<br><br>x}$

is a derivative of blue chrominance with respect to position.

31. Method according to claim 25, wherein said first process further comprises the steps of:

determining a fourth property of said blob pairs; and

excluding said blob as a non-fire if said fourth property of said blob pairs does not meet a fourth threshold.

32. Method according to claim 31, wherein:

said fourth property comprises a count of a number of instances of meeting said fourth threshold, and said fourth threshold is a minimum count value.

33. Method of using a color video sensor, comprising the steps of:

adjusting said color video sensor to first image settings;

obtaining at least two color first frames from said video sensor at said first image settings;

adjusting said video sensor to second image settings;

obtaining a plurality of color second frames from said video sensor at second image settings; and

contemporaneously performing a first process using said first frames and a second process using said second frames;

wherein

said at least two first images comprise a base frame and a comparison frame;

said first process comprises flame detection, and comprises the steps of;

identifying a plurality of base pixels in said base frame, and a plurality of comparison pixels in said comparison frame, wherein for each base frame pixel there is a spatially corresponding comparison frame pixel, each said base frame pixel and said corresponding comparison frame pixel forming a pair, such that said pluralities of base and comparison image pixels comprise a plurality of pairs;

determining an intensity of at least some of said pairs;

categorizing said pairs as blob pairs if said intensity meets a minimum intensity threshold;

determining a color of said blob pairs in YC_RC_B coordinates;

excluding said blob as a non-fire if said color does not fall within a color range in YC_RC_B coordinates;

determining derivatives of differences between said base pixels and said comparison pixels in said blob pairs, said derivatives comprising

$\frac{<br><br>Y}{<br><br>t},\frac{<br><br>Y}{<br><br>x},\frac{<br><br>C_R}{<br><br>t},\frac{<br><br>C_R}{<br><br>x},\frac{<br><br>C_B}{<br><br>t},\mathrm{and}<br><br>\text{\hspace{1em}<br><br>}<br><br>\frac{<br><br>C_B}{<br><br>x},\mathrm{wherein}$

$\frac{<br><br>Y}{<br><br>t}$

is a derivative of intensity with respect to time;

$\frac{<br><br>Y}{<br><br>x}$

is a derivative of intensity with respect to position;

$\frac{<br><br>C_R}{<br><br>t}$

is a derivative of red chrominance with respect to time;

$\frac{<br><br>C_R}{<br><br>x}$

is a derivative of red chrominance with respect to position;

$\frac{<br><br>C_B}{<br><br>t}$

is a derivative of blue chrominance with respect to time;

$\frac{<br><br>C_B}{<br><br>x}$

is a derivative of blue chrominance with respect to position.

plotting each of said derivatives into a histogram;

dividing each of said histograms into a plurality of bins;

determining a number of derivatives in each of at least some of said bins;

plotting a plurality points on each of a plurality of x-y plots, using an incidence of one of said bins from one of said histograms as an x-value and an incidence of another of said bins from one of said histograms as a y-value to plot each point thereon;

for each of said x-y plots, determining a plot ratio of points plotted on a first side of a cut-off to points not on said first side of said cut-off;

if said ratio does not exceed a plot threshold, identifying said plot as negative, and if said plot ratio does exceed a plot threshold, identifying said plot as positive;

counting a number of positive plots;

excluding said at least one blob as a non-fire if a number of positive plots does not exceed a fire threshold; and

indicating said at least one blob as a fire if a number of positive plots exceeds a fire threshold.

34. Method of adjusting a video sensor, comprising the steps of:

adjusting said video sensor to first image settings;

obtaining at least two first frames from said video sensor at said first image settings;

adjusting said video sensor to second image settings; and

obtaining a plurality of second frames from said video sensor at second image settings.

35. Method according to claim 34, wherein:

said steps are performed such that a first process and a second process may be performed contemporaneously with said first and said second frames.

36. Method according to claim 35, wherein:

adjusting said video sensor to said first image settings, obtaining said at least two first frames, and adjusting said video sensor to said second image settings takes substantially the same time as obtaining two of said second frames.

37. Method according to claim 34, wherein:

said first image settings are suitable for flame imaging, and said second image settings are suitable for a purpose other than flame-imaging.

38. Method of adjusting a video sensor, comprising the steps of:

adjusting said video sensor to first image settings;

obtaining a base frame from said video sensor at said first image settings;

adjusting said video sensor to second image settings;

obtaining at least one second frame from said video sensor at second image settings;

adjusting said video sensor to first image settings;

obtaining a comparison frame from said video sensor at said first image settings;

adjusting said video sensor to second image settings; and

obtaining at least one additional second frame from said video sensor at second image settings.

39. Method according to claim 38, wherein:

said steps are performed such that a first process and a second process may be performed contemporaneously with said first and said second frames.

40. Method according to claim 39, wherein:

adjusting said video sensor to said first image settings, obtaining said base frame, and adjusting said video sensor to said second image settings takes substantially the same time as obtaining one of said second frames.

41. Method according to claim 39, wherein:

adjusting said video sensor to said first image settings, obtaining said comparison frame, and adjusting said video sensor to said second image settings takes substantially the same time as obtaining one of said second frames.

42. Apparatus for performing multiple contemporaneous image processes, comprising:

a video sensor adapted to generate a video image;

a frame grabber in communication with said video sensor, adapted to obtain at least two first frames and a plurality of second frames from said video sensor;

a processor in communication with said frame grabber, adapted to contemporaneously perform a first process using said first frames and a second process using said second frames; and

at least one output mechanism in communication with said processor, adapted to generate a first output from said first process, and a second output from said second process;

wherein said processor is adapted to perform flame detection as said first process.

43. Apparatus according to claim 42, further comprising:

an adjusting mechanism for adjusting image settings of said video sensor; and

a control mechanism in communication with said adjusting mechanism and said processor, adapted to enable said processor to control said adjusting mechanism.

44. Apparatus according to claim 42, wherein:

said frame grabber is adapted to obtain said first frames and said second frames exclusively, such that said first frames are obtained from a first portion of said video image, and said second frames are obtained from a second portion of said video image,

wherein said first portion is unsuitable for obtaining said second frames therefrom, and said second portion is unsuitable for obtaining said first frames therefrom.

45. Apparatus according to claim 42, wherein:

said video sensor is adapted to generate said video image such that said video image comprises a plurality of consecutive image frames; and

said frame grabber is adapted to obtain said first and second frames such that said first frames comprise at least two of said image frames and said second frames comprising a remainder of said image frames.

46. Apparatus according to claim 42, wherein:

said frame grabber is adapted to obtain said first frames and said second frames non-exclusively, such that said first frames are obtained from a first portion of said video image, and said second frames are obtained from a second portion of said video image,

wherein said first portion is suitable for obtaining said second frames therefrom, and said second portion is suitable for obtaining said first frames therefrom.

47. Apparatus according to claim 42, wherein:

said video sensor is adapted to generate said video image such that said video image comprises a plurality of consecutive image frames;

said frame grabber is adapted to obtain said first frames such that said first frames comprise at least a first portion of at least two of said image frames; and

said frame grabber is adapted to obtain said second frames such that said second frames obtained by said frame grabber comprise at least a second portion of said image frames.

48. Apparatus according to claim 47, wherein:

said video sensor is adapted to generate said image frames with an image dynamic range and an image dynamic resolution;

said frame grabber is adapted to obtain said first frames with a first dynamic range comprising at least a first portion of said image dynamic range; and

said frame grabber is adapted to obtain said second frames with a second dynamic range comprising at least a second portion of said image dynamic range.

49. Apparatus according to claim 47, wherein:

said frame grabber is adapted to generate said first frames such that said first frames comprise an entirety of at least two of said image frames; and

said frame grabber is adapted to generate said second frames such that said second frames comprise an entirety of said image frames.

50. Apparatus according to claim 42, wherein:

said video sensor is adapted to generate said video image such that said video image comprises frames, and said frame grabber is adapted to obtain said at least two first frames such that said at least two first frames comprise consecutive image frames.

51. Apparatus according to claim 42, wherein:

said video sensor is adapted to generate said video image such that said video image comprises frames, and said frame grabber is adapted to obtain said at least two first frames such that said at least two first frames comprise non-consecutive image frames.

52. Apparatus according to claim 42, wherein:

said video sensor is a color video sensor adapted to generate a color video image.

53. Apparatus according to claim 52, wherein:

said frame grabber is adapted to obtain said first and second frames as color frames.

54. Apparatus according to claim 42, wherein:

said processor is adapted to display a human-viewable output as said second process.

55. Apparatus according to claim 42, wherein:

said processor is adapted to perform security monitoring as said second process.

56. Apparatus according to claim 42, wherein:

said processor is adapted to perform traffic observation as said second process.

57. Apparatus according to claim 42, wherein:

said processor is adapted to perform smoke detection as said second process.

58. Apparatus according to claim 42, wherein:

said at least two first frames comprise a base frame and a comparison frame; and

at least one of said video sensor, said frame grabber, and said processor is adapted to identify a plurality of base pixels in said base frame and a plurality of comparison pixels in said comparison frame as part of said first process, wherein for each base frame pixel there is a spatially corresponding comparison frame pixel, each said base frame pixel and said corresponding comparison frame pixel forming a pair, such that said pluralities of base and comparison image pixels comprise a plurality of pairs.

59. Apparatus according to claim 42, wherein said processor is adapted to perform the following as part of said first process:

determining a first property of at least some of said pairs;

categorizing said pairs as blob pairs if said first property meets a first threshold; and

assembling said blob pairs into at least one blob.

60. Apparatus according to claim 42, wherein:

said at least two first frames comprise a base frame and a comparison frame; and

at least one of said video sensor, said frame grabber, and said processor is adapted to identify a plurality of base pixels in said base frame and a plurality of comparison pixels in said comparison frame as part of said first process, wherein for each base frame pixel there is a spatially corresponding comparison frame pixel, each said base frame pixel and said corresponding comparison frame pixel forming a pair, such that said pluralities of base and comparison image pixels comprise a plurality of pairs.

61. Apparatus according to claim 60, wherein said processor is adapted to perform the following as part of said first process:

determining a first property of at least some of said pairs;

categorizing said pairs as blob pairs if said first property meets a first threshold;

assembling said blob pairs into at least one blob; and

indicating said at least one blob as a fire.

62. Apparatus according to claim 61, wherein said processor is adapted to perform the following as part of said first process:

determining a second property of said blob pairs; and

excluding said blob pairs if said second property of said blob pairs does not meet a second threshold.

63. Apparatus according to claim 61, wherein said processor is adapted to perform the following as part of said first process:

determining a second property of said blob pairs; and

excluding said blob as a non-fire if said second property of said blob pairs does not meet a second threshold.

64. Apparatus according to claim 62, wherein said processor is adapted to perform the following as part of said first process:

determining a third property of said blob pairs;

excluding said blob as a non-fire if said third property of said blob pairs does not meet a third threshold.

65. Apparatus according to claim 64, wherein:

determining said third property comprises calculating derivatives, and said processor is adapted to calculate said derivatives.

66. Apparatus according to claim 65, wherein:

determining said third property further comprises;

plotting said third property as at least one histogram and determining a number of qualified points thereof for at least two bands in said at least one histogram; and

plotting an incidence of at least one of said at least two bands against an incidence of at least another of said at least two bands as at least one x-y plot.

67. Apparatus according to claim 64, wherein said processor is adapted to perform the following as part of the first process:

determining a fourth property of said blob pairs; and

excluding said blob as a non-fire if said fourth property of said blob pairs does not meet a fourth threshold.