Method for detection and recognition of fog presence within an aircraft compartment using video images

• US 7,505,604 B2
• Filed: 11/05/2003
• Issued: 03/17/2009
• Est. Priority Date: 05/20/2002
• Status: Expired due to Fees

• Current Assignee: Simmonds Precision Products, Inc. (Rtx Corporation)
• Original Assignee: Simmonds Precision Products, Inc. (Rtx Corporation)
• Inventors: Zakrzewski, Radoslaw Romuald, Sadok, Mokhtar, Shirer, Jeffrey James, Zeliff, Robert Lowell
• Primary Examiner(s): Johns; Andrew W
• Application Number:
  US10/702,070
• Publication Number:
  US 20050069207A1
• Time in Patent Office: 1,959 Days
• Field of Search: 382/100, 382/195, 382/206, 382/286, 356/436, 356/437, 250/564, 340/945, 340/601, 340/602
• US Class Current: 382/100
• CPC Class Codes: B64D 45/0053 using visual equipment, e.g. cameras
• CPC Class Codes: G06F 18/24765 Rule-based classification
• CPC Class Codes: G06F 18/253 of extracted features
• CPC Class Codes: G06F 18/256 of results relating to different input data, e.g. multimodal recognition
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06V 10/765 using rules for classification or partitioning the feature space
• CPC Class Codes: G06V 10/806 of extracted features
• CPC Class Codes: G06V 10/811 the classifiers operating on different input data, e.g. multi-modal recognition
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G08B 17/125 by using a video camera to detect fire or smoke

Abstract

Detecting video phenomena, such as fire in an aircraft cargo bay, includes receiving a plurality of video images from a plurality of sources, compensating the images to provide enhanced images, extracting features from the enhanced images, and combining the features from the plurality of sources to detect the video phenomena. Extracting features may include determining an energy indicator for each of a subset of the plurality of frames. Detecting video phenomena may also include comparing energy indicators for each of the subset of the plurality of frames to a reference frame. The reference frame corresponds to a video frame taken when no fire is present, video frame immediately preceding each of the subset of the plurality of frames, or a video frame immediately preceding a frame that is immediately preceding each of the subset of the plurality of frames. Image-based and non-image based techniques are described herein in connection with fire detection and/or verification and other applications.

Claims

1. A method for detecting fog comprising:

using one or more processors to determine at least one visual characteristic of image data for a portion of an image including at least one of;

a change in overall image intensity for said portion, a change in image contrast for said portion, and a change in image sharpness for said portion; and

using one or more processors to determine, using said at least one visual characteristic, whether said change associated with said at least one visual characteristic is approximately uniform with respect to said portion.

2. The method of claim 1, wherein an amount of said change is within a predetermined threshold.

3. The method of claim 1, further comprising:

determining that at least one of the following conditions is present for said portion;

an increase in overall image intensity, a decrease in image sharpness, and a decrease in image contrast; and

determining a positive indicator for fog presence if it is determined that said at least one of the following conditions is present, and it is determined using said at least one visual characteristic that said change associated with said at least one visual characteristic is approximately uniform with respect to the portion.

4. The method of claim 3, further comprising:

detecting that said change occurs rapidly within a predetermined time period.

5. The method of claim 4, wherein said time period is a few seconds.

6. The method of claim 4, further comprising:

detecting that fog droplets form throughout a predetermined area at a same point in time.

7. The method of claim 4, wherein an initial value is associated with said at least one visual characteristic prior to fog formation and a second value is associated with said at least one visual characteristic during fog formation, and the method further comprising:

detecting that a value associated with said at least visual characteristic returns from said second value toward said initial value indicating fog dispersal.

8. The method of claim 7, wherein said value returns to said initial value from said second value in approximately linear time.

9. The method of claim 7, wherein said value returning to said initial value from said second value is approximately uniform across said portion of the image.

10. The method of claim 1, further comprising:

extracting at least one feature of said image data.

11. The method of claim 10, wherein said at least one feature includes an average image intensity of said portion of said image indicating image brightness, the average intensity m of said portion X of an image at a time “

t”

being represented as;

$m<br><br>\left(t\right)=\frac{1}{N}<br><br>\underset{j=1}{\overset{N}{\sum <br><br>}}<br><br>\mathrm{Xj}<br><br>\left(t\right)$

where N represents the total number of pixels in the portion X of the image at a time “

t”

.

12. The method of claim 11, further comprising:

determining an average image intensity for a plurality of portions for each of a plurality of images over a period of time to detect a trend in image brightness; and

determining, using said average image intensity for each of said plurality of portions for each of said plurality of images, if overall brightness of a region defined by said plurality of portions increases over a predetermined time period above a predetermined threshold.

13. The method of claim 11, wherein said at least one feature includes a standard deviation of the image intensity of said portion represented as:

$s<br><br>\left(t\right)=\sqrt{\frac{1}{N}<br><br>\underset{j=1}{\overset{N}{\sum <br><br>}}<br><br>{\left(\mathrm{Xj}<br><br>\left(t\right)-m<br><br>\left(t\right)\right)}^2}.$

14. The method of claim 13, wherein large values of s(t) indicate high variability of intensity and are related to high contrast, low values of s(t) indicate lower contrast, and the method further comprising:

detecting creation of fog when there is a rapid decrease of s(t) within a predetermined threshold; and

detection fog dispersal when there is an increase in s(t).

15. The method of claim 13, wherein said at least one feature includes a correlation based on a likelihood ratio distribution represented as:

$d_1<br><br>\left(t\right)=1-\underset{j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\min }<br><br>\left({\mathrm{LR}}_j,\frac{1}{{\mathrm{LR}}_j}\right)<br><br><br><br>\mathrm{where}$ ${\mathrm{LR}}_j=\frac{{\left[\frac{{\left(s<br><br>\left(t\right)\right)}^2+{\left(s<br><br>\left(t-\mathrm{\delta <br><br>}\right)\right)}^2}{2}+\frac{{\left(X_j<br><br>\left(t\right)-m<br><br>\left(t-\mathrm{\delta <br><br>}\right)\right)}^2}{2}\right]}^2}{{\left(s<br><br>\left(t\right)\right)}^2+{\left(s<br><br>\left(t-\mathrm{\delta <br><br>}\right)\right)}^2}.$

16. The method of claim 11, wherein said at least one feature includes a mean absolute difference from the mean represented as:

$d<br><br>\left(t\right)=\frac{1}{N}<br><br>\underset{j=1}{\overset{N}{\sum <br><br>}}<br><br><br><br>\mathrm{Xj}<br><br>\left(t\right)-m<br><br>\left(t\right)<br><br>.$

17. The method of claim 16, wherein creation of fog is associated with a rapid drop of d(t) above a predetermined threshold and wherein an increase in d(t) is associated with fog dispersal.

18. The method of claim 11, wherein said at least one feature includes a correlation measurement based on the t-Student distribution represented as:

$d_a<br><br>\left(t,\mathrm{\delta <br><br>}\right)=\frac{\underset{j=1}{\overset{N}{\sum <br><br>}}<br><br>\left(\mathrm{Xj}<br><br>\left(t\right)-m<br><br>\left(t-\mathrm{\delta <br><br>}\right)\right)}{\sqrt{\underset{j=1}{\overset{N}{\sum <br><br>}}<br><br>{\left(\mathrm{Xj}<br><br>\left(t\right)-m<br><br>\left(t-\mathrm{\delta <br><br>}\right)\right)}^2}}$

in which t and t−

δ

represent points in time and X(t) and X(t−

δ

) are portions of images taken, respectively, at these two points in time.

19. The method of claim 18, wherein said correlation measurement is used in tracking a statistical evolution of a portion of a video stream as compared to a portion of the reference image at time t−

δ

, and wherein values of said correlation measurement larger than a predetermined threshold indicate fog.

20. The method of claim 10, wherein said at least one feature includes intensity of a change image with respect to said portion X of said image, D(t,δ

)=X(t)−

X(t−

δ

), which is a difference between time instances t and t−

δ

of said portion, represented as;

$m_D<br><br>\left(t,\mathrm{\delta <br><br>}\right)=\frac{1}{N}<br><br>\underset{j=1}{\overset{N}{\sum <br><br>}}<br><br>\mathrm{Dj}<br><br>\left(t,\mathrm{\delta <br><br>}\right)$

where N represents a total number of pixels being analyzed of said portion X.

21. The method of claim 20, further comprising:

determining creation of fog when there are one or more positive values of m_D(t,δ

); and

determining dispersion of fog when there are one or more negative values of m_D(t,δ

).

22. The method of claim 20, wherein the time interval δ

between two compared frames is fixed.

23. The method of claim 20, wherein the time interval δ

is adjusted in accordance with at least one system condition or parameter.

24. The method of claim 20, wherein a reference frame of the portion, X(t−

δ

), represents an initial view of a portion of a cargo bay X(0) such that δ

is a current time t since the start of a flight from a time of the initial view and wherein the difference image with respect to the portion, D(t,δ

), represents a cumulative change of view of the portion since beginning of the flight.

25. The method of claim 20, wherein a portion of reference image X(t−

δ

) is reset periodically to accommodate changes of background.

26. The method of claim 20, wherein a portion of a reference frame X(t−

δ

) is set to a frame immediately preceding a current frame such that δ

=1, and wherein the difference image with respect to said portion D(t,δ

) represents the instantaneous rate of change of a view of said portion.

27. The method of claim 20, wherein said at least one feature includes characterizing image sharpness using an intensity gradient, and wherein said intensity gradient is determined using a portion of a change image.

28. The method of claim 20, wherein said at least one feature includes a dynamic range of intensity change including a standard deviation s_d(t,δ

) of intensity change over some predefined time interval, δ

, defined as;

$s_d<br><br>\left(t,\mathrm{\delta <br><br>}\right)=\sqrt{\frac{1}{N}<br><br>\underset{j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\sum <br><br>}<br><br>{\left(D_j<br><br>\left(t,\mathrm{\delta <br><br>}\right)-m_d<br><br>\left(t,\mathrm{\delta <br><br>}\right)\right)}^2}$

and wherein a value of the standard deviation s_d(t,δ

) is close to zero within some predetermined threshold if there is fog.

29. The method of claim 20, wherein said at least one feature includes a mean absolute deviation of a portion from the mean value of a portion of the change image represented as:

$d_d<br><br>\left(t\right)=\frac{1}{N}<br><br>\underset{j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\sum <br><br>}<br><br><br><br>D_j<br><br>\left(t,\mathrm{\delta <br><br>}\right)-m_d<br><br>\left(t,\mathrm{\delta <br><br>}\right)<br><br>$

and wherein a value of the mean absolute deviation is close to zero within some predetermined threshold if there is fog.

30. The method of claim 20, wherein said at least one feature includes a spatial moment of a portion of the change image, and wherein coordinates of a center of mass a portion of the change image D(t,δ

) are represented as;

$i_c<br><br>\left(t,\mathrm{\delta <br><br>}\right)=\frac{\underset{i,j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\sum <br><br>}<br><br>i<br><br><br><br>D_{i,j}<br><br>\left(t,\mathrm{\delta <br><br>}\right)}{\underset{i,j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\sum <br><br>}<br><br>D_{i,j}<br><br>\left(t,\mathrm{\delta <br><br>}\right)}<br><br>j_c<br><br>\left(t,\mathrm{\delta <br><br>}\right)=\frac{\underset{i,j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\sum <br><br>}<br><br>j<br><br><br><br>D_{i,j}<br><br>\left(t,\mathrm{\delta <br><br>}\right)}{\underset{i,j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\sum <br><br>}<br><br>D_{i,j}<br><br>\left(t,\mathrm{\delta <br><br>}\right)}$

and wherein, if the image change for a portion is uniform across the portion of the image, the coordinates are close to the geometric center of the portion of the image indicating presence of fog.

31. The method of claim 30, wherein said at least one feature includes higher order moments of a portion of a change image represented as:

$M_{2,0}<br><br>\left(t,\mathrm{\delta <br><br>}\right)=\underset{i,j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\sum <br><br>}<br><br>{\left(i-i_c<br><br>\left(t,\mathrm{\delta <br><br>}\right)\right)}^2<br><br>D_{i,j}<br><br>\left(t,\mathrm{\delta <br><br>}\right)$ $M_{0,2}<br><br>\left(t,\mathrm{\delta <br><br>}\right)=\underset{i,j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\sum <br><br>}<br><br>{\left(j-j_c<br><br>\left(t,\mathrm{\delta <br><br>}\right)\right)}^2<br><br>D_{i,j}<br><br>\left(t,\mathrm{\delta <br><br>}\right).$

32. The method of claim 31, wherein said at least one feature includes a moment of inertia of a portion of the change image represented as:

$M_i<br><br>\left(t,\mathrm{\delta <br><br>}\right)=\underset{i,j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\sum <br><br>}<br><br>{\left(i-i_c<br><br>\left(t,\mathrm{\delta <br><br>}\right)\right)}^2+{\left(j-j_c<br><br>\left(t,\mathrm{\delta <br><br>}\right)\right)}^2<br><br>D_{i,j}<br><br>\left(t,\mathrm{\delta <br><br>}\right).$

33. The method of claim 20, wherein said at least one feature includes moments defined using average absolute values of pixels represented as:

$M_{1,0}^a<br><br>\left(t,\mathrm{\delta <br><br>}\right)=\underset{i,j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\sum <br><br>}<br><br><br><br>i-i_c<br><br>\left(t,\mathrm{\delta <br><br>}\right)<br><br><br><br>D_{i,j}<br><br>\left(t,\mathrm{\delta <br><br>}\right)$ $M_{0,1}^a<br><br>\left(t,\mathrm{\delta <br><br>}\right)=\underset{i,j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\sum <br><br>}<br><br><br><br>j-j_c<br><br>\left(t,\mathrm{\delta <br><br>}\right)<br><br><br><br>D_{i,j}<br><br>\left(t,\mathrm{\delta <br><br>}\right)$ $M_i^a<br><br>\left(t,\mathrm{\delta <br><br>}\right)=\underset{i,j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\sum <br><br>}<br><br>(<br><br>i-i_c<br><br>\left(t,\mathrm{\delta <br><br>}\right)<br><br>+<br><br>j-j_c<br><br>\left(t,\mathrm{\delta <br><br>}\right)<br><br><br><br>D_{i,j}<br><br>\left(t,\mathrm{\delta <br><br>}\right)$

and wherein, if a portion of the change image is uniform, values for these moments are larger than a predetermined threshold indicating a presence of fog.

34. The method of claim 10, wherein said at least one feature includes an absolute value of the average intensity change represented as:

$d_a<br><br>\left(t,\mathrm{\delta <br><br>}\right)=<br><br>\frac{1}{N}<br><br>\underset{j=1}{\overset{N}{\sum <br><br>}}<br><br>\left(\mathrm{Xj}<br><br>\left(t\right)-\mathrm{Xj}<br><br>\left(t-\mathrm{\delta <br><br>}\right)\right)<br><br>$

in which t and t−

δ

represent points in time and X(t) and X(t−

δ

) are portions of images taken, respectively, at these two points in time.

35. The method of claim 10, wherein said at least one feature includes an intensity range r(t) at a time t represented as:

r(t)=χ

_max(t)−

χ

_min(t)

where a maximum (Xmax) and a minimum (Xmin) intensity of a portion X of an image at a time t are used to provide an indication of reduced image contrast for the portion X and are represented as;

${\mathrm{\chi <br><br>}}_{\max }<br><br>\left(t\right)=\underset{j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\max }<br><br>X_j<br><br>\left(t\right),{\mathrm{\chi <br><br>}}_{\min }<br><br>\left(t\right)=\underset{j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\min }<br><br>X_j<br><br>\left(t\right).$

36. The method of claim 28, wherein creation of fog is indicated by a rapid drop of r(t), and an increase in r(t) indicates fog dispersal, and wherein, r(t) decreasing below a threshold amount indicates that fog is present.

37. The method of claim 10, wherein said at least one feature includes characterizing image sharpness of a portion X of an image using an intensity gradient.

38. The method of claim 37, wherein x and y gradient components G at pixel i,j of a portion X of an image at a time t are defined as a left difference represented as:

G_i,j^x(t)=X_i,j(t)−

X<sub>i−

1,j</sub>(t) G_i,j^y(t)=X_i,j+1(t)−

X<sub>i−

j,1</sub>(t).

39. The method of claim 37, wherein x and y gradient components G at pixel i,j of a portion X of an image at a time t are defined as a right difference represented as:

G_i,j^x(t)=X_i+1,j(t) G_i,j+1(t)−

X_i,j(t).

40. The method of claim 37, wherein x and y gradient components G at pixel i,j are defined as a double-sided difference represented as:

$G_{i,j}^x<br><br>\left(t\right)=\frac{X_{i+1,j}<br><br>\left(t\right)-X_{i-1,j}<br><br>\left(t\right)}{2}<br><br>G_{i,j}^y<br><br>\left(t\right)=\frac{X_{i,j+1}<br><br>\left(t\right)-X_{i,j-1}<br><br>\left(t\right)}{2}.$

41. The method according to one of claims 38, 39 and 40, wherein said at least one feature includes a mean absolute gradient value represented as:

$g_a^x<br><br>\left(t\right)=\frac{1}{N}<br><br>\underset{i,j=\mathrm{all}<br><br><br><br>\mathrm{pixles}}{\sum <br><br>}<br><br><br><br>G_{i,j}^x<br><br>\left(t\right)<br><br>,g_a^y<br><br>\left(t\right)=\frac{1}{N}<br><br>\underset{i,j=\mathrm{all}<br><br><br><br>\mathrm{pixles}}{\sum <br><br>}<br><br><br><br>G_{i,j}^y<br><br>\left(t\right)<br><br>$

such that creation of fog is signified by a rapid drop in at least one of;

g_α

^x(t) and g_α

^y(t).

42. The method of claim 41, wherein said at least one feature includes an overall average gradient characteristic represented as:

g_α

(t)=g_α

^x(t)+g_α

^y(t).

43. The Method according to one of claims 38, 39, and 40, wherein said at least one feature includes an average gradient norm, wherein a gradient norm G at pixel i,j is represented as:

G_i,jn(t)=√

{square root over (G_i,j^x(t)²+G_i,j^y(t)²)}{square root over (G_i,j^x(t)²+G_i,j^y(t)²)}

for all “

N”

pixels within a portion of an image, and the average gradient norm is represented as;

$g_n<br><br>\left(t\right)=\frac{1}{N}<br><br>\underset{i,j=\mathrm{all}<br><br><br><br>\mathrm{pixles}}{\sum <br><br>}<br><br>G_{i,j}^n<br><br>\left(t\right).$

44. The method of claim 43, wherein creation of fog is related to a rapid drop in g_n(t) below a predetermined threshold value, and an increase in g_n(t) indicates fog dispersal.

45. The method of claim 44, wherein said at least one feature includes maximum and minimum values of x and y components of a gradient norm G represented as:

$g_{\max }<br><br>\left(t\right)<br><br>\underset{i,j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\max }<br><br>G_{i,j}^n<br><br>\left(t\right),g_{\min }<br><br>\left(t\right)=\underset{i,j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\min }<br><br>G_{i,j}^n<br><br>\left(t\right)$ $g_{\max }^x<br><br>\left(t\right)<br><br>\underset{i,j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\max }<br><br><br><br>G_{i,j}^x<br><br>\left(t\right)<br><br>,g_{\min }^x<br><br>\left(t\right)=\underset{i,j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\min }<br><br><br><br>G_{i,j}^n<br><br>\left(t\right)<br><br><br><br><br><br>g_{\max }^y<br><br>\left(t\right)<br><br>\underset{i,j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\max }<br><br><br><br>G_{i,j}^y<br><br>\left(t\right)<br><br>,g_{\min }^y<br><br>\left(t\right)=\underset{i,j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\min }<br><br><br><br>G_{i,j}^y<br><br>\left(t\right)<br><br>.$

46. The method of claim 37, wherein said intensity gradient defines a gradient in terms of differences between pixel locations with time as a constant.

47. The method of claim 37, wherein said intensity gradient defines a gradient in terms of pixel values between portions of images taken at different points in time.

48. The method of claim 37, wherein a large value of said intensity gradient indicates sharp edges within a portion of an image.

49. The method of claim 1, further comprising:

determining that at least one of the following conditions is present for said portion of said image;

an increase in overall image intensity, a decrease in image sharpness, and a decrease in image contrast;

determining an intermediate positive indicator for fog presence if it is determined that said at least one of the following conditions is present, and it is determined using said at least one visual characteristic that said change associated with said at least one visual characteristic is approximately uniform with respect to the portion of the image; and

determining a final positive indicator for fog presence if said intermediate positive indicator indicates that there is fog which is confirmed by at least one other feature.

50. The method of claim 49, wherein said at least one other feature is a non-image feature.

51. The method of claim 50, wherein said at least one other feature includes at least one of temperature, humidity and pressure.

52. The method of claim 49, wherein a plurality of intermediate positive indicators are used in determining said final positive indicator.

53. The method of claim 49, further comprising:

distinguishing fog from one of a plurality of other conditions, wherein said plurality of other conditions includes smoke and an aerosol dispersion.

54. The method of claim 1, wherein said portion is an entire image.

55. The method of claim 1, wherein said portion includes a plurality of regions of said image.

56. The method of claim 55, wherein each of said plurality of regions is a predefined shape in accordance with lighting and camera view.

57. The method of claim 1, wherein said at least one visual characteristic is a frequency-based feature.

58. The method of claim 57, wherein said frequency-based feature estimates motion of an element of said portion.

59. The method of claim 57, wherein said frequency-based feature is used to monitor a an area within a camera view for at least one other condition unrelated to fire.

60. The method of claim 59, wherein a cargo bay area is monitored during cargo loading using said frequency-based feature.

61. The method of claim 1, further comprising:

receiving an image from a CCD camera.

62. The method of claim 61, wherein said CCD camera has an operational wavelength sensitivity between approximately 770 and 1200 nanometers blocking visible light.

63. The method of claim 59, wherein said CCD camera is a conventional CCD camera with an operational wavelength sensitivity between approximately 400 and 1200 nanometers.

64. The method of claim 63, wherein said CCD camera is used when it is determined that a view area is completely obscured except for a predetermined space within which said CCD camera is included.

65. The method of claim 63, wherein said operational wavelength sensitivity of said CCD camera excludes a portion of the range between approximately 400 and 1200 nanometers.

66. The method of claim 65, wherein said at least one excluded portion has a range corresponding to one of:

a light source and a device that emits within said at least one excluded portion to filter out wavelengths within said at least one excluded portion.

67. The method of claim 61, wherein said CCD camera has an operational wavelength sensitivity approximating that of visible light.

68. The method of claim 1, further comprising:

receiving an image from a camera with an associated light source wherein said camera is mounted opposite said associated light source within a viewing area.

69. The method of claim 68, wherein said viewing area is an aircraft cargo bay, and said camera and said associated light source are mounted within a predetermined distance from a ceiling of said aircraft cargo bay.

70. The method of claim 69, wherein said camera and said associated light source are positioned at a same vertical and horizontal location on walls of said cargo bay area.

71. A computer program, stored on a computer-readable storage medium, that detects fog, comprising executable code that:

determines at least one visual characteristic of image data for a portion of an image including at least one of;

a change in overall image intensity for said portion, a change in image contrast for said portion, and a change in image sharpness for said portion; and

determines, using said at least one visual characteristic, whether said change associated with said at least one visual characteristic is approximately uniform with respect to said portion.

72. The computer program of claim 71, wherein an amount of said change is within a predetermined threshold.

73. The computer program of claim 71, further comprising executable code that:

determines at least one of the following conditions is present for said portion;

an increase in overall image intensity, a decrease in image sharpness, and a decrease in image contrast; and

determines a positive indicator for fog presence if it is determined that said at least one of the following conditions is present, and it is determined using said at least one visual characteristic that said change associated with said at least one visual characteristic is approximately uniform with respect to the portion.

74. The computer program of claim 73, further comprising:

executable code that detects that said change occurs rapidly within a predetermined time period.

75. The computer program of claim 74, wherein said time period is a few seconds.

76. The computer program of claim 74, further comprising:

executable code that detects that fog droplets form throughout a predetermined area at a same point in time.

77. The computer program of claim 74, wherein an initial value is associated with said at least one visual characteristic prior to fog formation and a second value is associated with said at least one visual characteristic during fog formation, and the computer program product further comprising:

executable code that detects that a value associated with said at least visual characteristic returns from said second value toward said initial value indicating fog dispersal.

78. The computer program of claim 77, wherein said value returns to said initial value from said second value in approximately linear time.

79. The computer program product of claim 77, wherein said value returning to said initial value from said second value is approximately uniform across said portion of the image.

80. The computer program of claim 71, further comprising:

executable code that extracts at least one feature of said image data.

81. The computer program of claim 80, wherein said at least one feature includes an average image intensity of said portion of said image indicating image brightness, the average intensity m of said portion X of an image at a time “

t”

being represented as;

$m<br><br>\left(t\right)=\frac{1}{N}<br><br>\underset{j=1}{\overset{N}{\sum <br><br>}}<br><br>\mathrm{Xj}<br><br>\left(t\right)$

where N represents the total number of pixels in the portion X of the image at a time “

t”

.

82. The computer program of claim 81, further comprising executable code that:

determines an average image intensity for a plurality of portions for each of a plurality of images over a period of time to detect a trend in image brightness; and

determines, using said average image intensity for each of said plurality of portions for each of said plurality of images, if overall brightness of a region defined by said plurality of portions increases over a predetermined time period above a predetermined threshold.

83. The computer program of claim 81, wherein said at least one feature includes a standard deviation of the image intensity of said portion represented as:

$s<br><br>\left(t\right)=\sqrt{\frac{1}{N}<br><br>\underset{j=1}{\overset{N}{\sum <br><br>}}<br><br>{\left(\mathrm{Xj}<br><br>\left(t\right)-m<br><br>\left(t\right)\right)}^2}.$

84. The computer program of claim 83, wherein large values of s(t) indicate high variability of intensity and are related to high contrast, low values of s(t) indicate lower contrast, and the computer program product further comprising executable code that:

detects creation of fog when there is a rapid decrease of s(t) within a predetermined threshold; and

detects fog dispersal when there is an increase in s(t).

85. The computer program of claim 83, wherein said at least one feature includes a correlation based on a likelihood ratio distribution represented as:

$d_1<br><br>\left(t\right)=1-\underset{j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\min }<br><br>\left({\mathrm{LR}}_j,\frac{1}{{\mathrm{LR}}_j}\right)<br><br><br><br>\mathrm{where}$ ${\mathrm{LR}}_j=\frac{{\left[\frac{{\left(s<br><br>\left(t\right)\right)}^2+{\left(s<br><br>\left(t-\mathrm{\delta <br><br>}\right)\right)}^2}{2}+\frac{{\left(X_j<br><br>\left(t\right)-m<br><br>\left(t-\mathrm{\delta <br><br>}\right)\right)}^2}{2}\right]}^2}{{\left(s<br><br>\left(t\right)\right)}^2+{\left(s<br><br>\left(t-\mathrm{\delta <br><br>}\right)\right)}^2}.$

86. The computer program of claim 81, wherein said at least one feature includes a mean absolute difference from the mean represented as:

$d<br><br>\left(t\right)=\frac{1}{N}<br><br>\underset{j=1}{\overset{N}{\sum <br><br>}}<br><br><br><br>\mathrm{Xj}<br><br>\left(t\right)-m<br><br>\left(t\right)<br><br>.$

87. The computer program of claim 86, wherein creation of fog is associated with a rapid drop of d(t) above a predetermined threshold and wherein an increase in d(t) is associated with fog dispersal.

88. The computer program of claim 81, wherein said at least one feature includes a correlation measurement based on the t-Student distribution represented as:

$d_s<br><br>\left(t,\mathrm{\delta <br><br>}\right)=\frac{\underset{j=1}{\overset{N}{\sum <br><br>}}<br><br>\left(\mathrm{Xj}<br><br>\left(t\right)-m<br><br>\left(t-\mathrm{\delta <br><br>}\right)\right)}{\sqrt{\underset{j=1}{\overset{N}{\sum <br><br>}}<br><br>{\left(\mathrm{Xj}<br><br>\left(t\right)-m<br><br>\left(t-\mathrm{\delta <br><br>}\right)\right)}^2}}$

in which t and t−

δ

represent points in time and X(t) and X(t−

δ

) are portions of images taken, respectively, at these two points in time.

89. The computer program of claim 88, wherein said correlation measurement is used in tracking a statistical evolution of a portion of a video stream as compared to a portion of the reference image at time t−

δ

, and wherein values of said correlation measurement larger than a predetermined threshold indicate fog.

90. The computer program of claim 80, wherein said at least one feature includes intensity of a change image with respect to said portion X of said image, D(t,δ

)=X(t)−

X(t−

δ

), which is a difference between time instances t and t−

δ

of said portion, represented as;

$m_D<br><br>\left(t,\mathrm{\delta <br><br>}\right)=\frac{1}{N}<br><br>\underset{j=1}{\overset{N}{\sum <br><br>}}<br><br>\mathrm{Dj}<br><br>\left(t,\mathrm{\delta <br><br>}\right)$

where N represents a total number of pixels being analyzed of said portion X.

91. The computer program product of claim 90, further comprising executable code that:

determines creation of fog when there are one or more positive values of m_D(t,δ

); and

determines dispersion of fog when there are one or more negative values of m_D(t,δ

).

92. The computer program of claim 90, wherein the time interval δ

between two compared frames is fixed.

93. The computer program of claim 90, wherein the time interval δ

is adjusted in accordance with at least one system condition or parameter.

94. The computer program of claim 90, wherein a reference frame of the portion, X(t−

δ

), represents an initial view of a portion of a cargo bay X(0) such that δ

is a current time t since the start of a flight from a time of the initial view and wherein the difference image with respect to the portion, D(t,δ

), represents a cumulative change of view of the portion since beginning of the flight.

95. The computer program of claim 90, wherein a portion of reference image X(t−

δ

) is reset periodically to accommodate changes of background.

96. The computer program of claim 90, wherein a portion of a reference frame X(t−

δ

) is set to a frame immediately preceding a current frame such that δ

=1, and wherein the difference image with respect to said portion D(t,δ

) represents the instantaneous rate of change of a view of said portion.

97. The computer program of claim 90, wherein said at least one feature includes characterizing image sharpness using an intensity gradient, and wherein said intensity gradient is determined using a portion of a change image.

98. The computer program of claim 90, wherein said at least one feature includes a dynamic range of intensity change including a standard deviation s_d(t,δ

) of intensity change over some predefined time interval, δ

, defined as;

$s_d<br><br>\left(t,\mathrm{\delta <br><br>}\right)=\sqrt{\frac{1}{N}<br><br>\underset{j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\sum <br><br>}<br><br>{\left(D_j<br><br>\left(t,\mathrm{\delta <br><br>}\right)-m_d<br><br>\left(t,\mathrm{\delta <br><br>}\right)\right)}^2}$

and wherein a value of the standard deviation s_d(t,δ

) is close to zero within some predetermined threshold if there is fog.

99. The computer program of claim 90, wherein said at least one feature includes a mean absolute deviation of a portion from the mean value of a portion of the change image represented as:

$d_d<br><br>\left(t\right)=\frac{1}{N}<br><br>\underset{j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\sum <br><br>}<br><br><br><br>D_j<br><br>\left(t,\mathrm{\delta <br><br>}\right)-m_d<br><br>\left(t,\mathrm{\delta <br><br>}\right)<br><br>$

and wherein a value of the mean absolute deviation is close to zero within some predetermined threshold if there is fog.

100. The computer program of claim 90, wherein said at least one feature includes a spatial moment of a portion of the change image, and wherein coordinates of a center of mass a portion of the change image D(t,δ

) are represented as;

$i_c<br><br>\left(t,\mathrm{\delta <br><br>}\right)=\frac{\underset{i,j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\sum <br><br>}<br><br>{\mathrm{iD}}_{i,j}<br><br>\left(t,\mathrm{\delta <br><br>}\right)}{\underset{i,j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\sum <br><br>}<br><br>D_{i,j}<br><br>\left(t,\mathrm{\delta <br><br>}\right)}<br><br><br><br>j_c<br><br>\left(t,\mathrm{\delta <br><br>}\right)=\frac{\underset{i,j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\sum <br><br>}<br><br>{\mathrm{jD}}_{i,j}<br><br>\left(t,\mathrm{\delta <br><br>}\right)}{\underset{i,j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\sum <br><br>}<br><br>D_{i,j}<br><br>\left(t,\mathrm{\delta <br><br>}\right)}$

and wherein, if the image change for a portion is uniform across the portion of the image, the coordinates are close to the geometric center of the portion of the image indicating presence of fog.

101. The computer program of claim 100, wherein said at least one feature includes higher order moments of a portion of a change image represented as:

$M_{2,0}<br><br>\left(t,\mathrm{\delta <br><br>}\right)=\underset{i,j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\sum <br><br>}<br><br>{\left(i-i_c<br><br>\left(t,\mathrm{\delta <br><br>}\right)\right)}^2<br><br>D_{i,j}<br><br>\left(t,\mathrm{\delta <br><br>}\right)$ $M_{0,2}<br><br>\left(t,\mathrm{\delta <br><br>}\right)=\underset{i,j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\sum <br><br>}<br><br>{\left(j-j_c<br><br>\left(t,\mathrm{\delta <br><br>}\right)\right)}^2<br><br>D_{i,j}<br><br>\left(t,\mathrm{\delta <br><br>}\right).$

102. The computer program of claim 101, wherein said at least one feature includes a moment of inertia of a portion of the change image represented as:

$M_i<br><br>\left(t,\mathrm{\delta <br><br>}\right)=\underset{i,j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\sum <br><br>}<br><br>\left({\left(i-i_c<br><br>\left(t,\mathrm{\delta <br><br>}\right)\right)}^2<br><br>{\left(j-j_c<br><br>\left(t,\mathrm{\delta <br><br>}\right)\right)}^2\right)<br><br>D_{i,j}<br><br>\left(t,\mathrm{\delta <br><br>}\right).$

103. The computer program of claim 90, wherein said at least one feature includes moments defined using average absolute values of pixels represented as:

$M_{1,0}^a<br><br>\left(t,\mathrm{\delta <br><br>}\right)=\underset{i,j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\sum <br><br>}<br><br><br><br>i-i_c<br><br>\left(t,\mathrm{\delta <br><br>}\right)<br><br><br><br>D_{i,j}<br><br>\left(t,\mathrm{\delta <br><br>}\right)$ $M_{0,1}^a<br><br>\left(t,\mathrm{\delta <br><br>}\right)=\underset{i,j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\sum <br><br>}<br><br><br><br>j-j_c<br><br>\left(t,\mathrm{\delta <br><br>}\right)<br><br><br><br>D_{i,j}<br><br>\left(t,\mathrm{\delta <br><br>}\right)$ $M_i^a<br><br>\left(t,\mathrm{\delta <br><br>}\right)=\underset{i,j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\sum <br><br>}<br><br>\left(<br><br>i-i_c<br><br>\left(t,\mathrm{\delta <br><br>}\right)<br><br>+<br><br>j-j_c<br><br>\left(t,\mathrm{\delta <br><br>}\right)<br><br>\right)<br><br>D_{i,j}<br><br>\left(t,\mathrm{\delta <br><br>}\right)$

and wherein, if a portion of the change image is uniform, values for these moments are larger than a predetermined threshold indicating a presence of fog.

104. The computer program of claim 80, wherein said at least one feature includes an absolute value of the average intensity change represented as:

$d_a<br><br>\left(t,\mathrm{\delta <br><br>}\right)=<br><br>\frac{1}{N}<br><br>\underset{j=1}{\overset{N}{\sum <br><br>}}<br><br>\left(\mathrm{Xj}<br><br>\left(t\right)-\mathrm{Xj}<br><br>\left(t-\mathrm{\delta <br><br>}\right)\right)<br><br>$

in which t and t−

δ

represent points in time and X(t) and X(t−

δ

) are portions of images taken, respectively, at these two points in time.

105. The computer program of claim 80, wherein said at least one feature includes an intensity range r(t) at a time t represented as:

r(t)=χ

_max(t)−

χ

_min(t)

where a maximum (Xmax) and a minimum (Xmin) intensity of a portion X of an image at a time t are used to provide an indication of reduced image contrast for the portion X and are represented as;

${\mathrm{\chi <br><br>}}_{\max }<br><br>\left(t\right)=\underset{j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\max }<br><br>X_j<br><br>\left(t\right),{\mathrm{\chi <br><br>}}_{\min }<br><br>\left(t\right)=\underset{j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\min }<br><br>X_j<br><br>\left(t\right).$

106. The computer program of claim 105, wherein creation of fog is indicated by a rapid drop of r(t), and an increase in r(t) indicates fog dispersal, and wherein, r(t) decreasing below a threshold amount indicates that fog is present.

107. The computer program of claim 80, wherein said at least one feature includes characterizing image sharpness of a portion X of an image using an intensity gradient.

108. The computer program of claim 107, wherein x and y gradient components G at pixel i,j of a portion X of an image at a time t are defined as a left difference represented as:

G_i,j^x(t)=X_i,j(t)−

X<sub>i−

1,j</sub>(t) G_i,j^y(t)=X_i,j(t)−

X<sub>i−

j,1</sub>(t).

109. The computer program of claim 107, wherein x and y gradient components G at pixel i,j of a portion X of an image at a time t are defined as a right difference represented as:

G_i,j^x(t)=X_i+1,j(t)−

X_i,j(t) G_i,j^y(t)=X_i,j+1(t)−

X_i,j(t).

110. The computer program of claim 107, wherein x and y gradient components G at pixel i,j are defined as a double-sided difference represented as:

$G_{i,j}^x<br><br>\left(t\right)=\frac{X_{i+1,j}<br><br>\left(t\right)-X_{i-1,j}<br><br>\left(t\right)}{2}<br><br>G_{i,j}^y<br><br>\left(t\right)=\frac{X_{i,j+1}<br><br>\left(t\right)-X_{i,j-1}<br><br>\left(t\right)}{2}.$

111. The computer program product according to one of claims 108, 109, and 110, wherein said at least one feature includes a mean absolute gradient value represented as:

$g_a^x<br><br>\left(t\right)=\frac{1}{N}<br><br>\underset{i,j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\sum <br><br>}<br><br><br><br>G_{i,j}^x<br><br>\left(t\right)<br><br>,<br><br>g_a^y<br><br>\left(t\right)=\frac{1}{N}<br><br>\underset{i,j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\sum <br><br>}<br><br><br><br>G_{i,j}^y<br><br>\left(t\right)<br><br>$

such that creation of fog is signified by a rapid drop in at least one of;

g_α

^x(t) and g_α

^y(t).

112. The computer program of claim 111, wherein said at least one feature includes an overall average gradient characteristic represented as:

g_o(t)=g_α

^x(t)+g_α

^y(t).

113. The computer program according to one of claims 108, 109 and 110, wherein said at least one feature includes an average gradient norm, wherein a gradient norm G at pixel i,j is represented as:

G_i,jⁿ(t)=√

{square root over (G_i,j^x(t)²+G_i,j^y(t)²)}{square root over (G_i,j^x(t)²+G_i,j^y(t)²)}

for all “

N”

pixels within a portion of an image, and the average gradient norm is represented as;

$g_n<br><br>\left(t\right)=\frac{1}{N}<br><br>\underset{i,j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\sum <br><br>}<br><br>G_{i,j}^n<br><br>\left(t\right).$

114. The computer program of claim 113, wherein creation of fog is related to a rapid drop in g_n(t) below a predetermined threshold value, and an increase in g_n(t) indicates fog dispersal.

115. The computer program of claim 114, wherein said at least one feature includes maximum and minimum values of x and y components of a gradient norm G represented as:

$g_{\max }<br><br>\left(t\right)=\underset{i,j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\max }<br><br>G_{i,j}^n<br><br>\left(t\right),g_{\min }<br><br>\left(t\right)=\underset{i,j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\max }<br><br>G_{i,j}^n<br><br>\left(t\right)$ $g_{\max }^x<br><br>\left(t\right)=\underset{i,j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\max }<br><br><br><br>G_{i,j}^x<br><br>\left(t\right)<br><br>,g_{\min }^x<br><br>\left(t\right)=\underset{i,j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\max }<br><br><br><br>G_{i,j}^n<br><br>\left(t\right)<br><br>$ $g_{\max }^y<br><br>\left(t\right)=\underset{i,j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\max }<br><br><br><br>G_{i,j}^y<br><br>\left(t\right)<br><br>,g_{\min }^y<br><br>\left(t\right)=\underset{i,j=\mathrm{all}<br><br><br><br>\mathrm{pixels}}{\max }<br><br><br><br>G_{i,j}^y<br><br>\left(t\right)<br><br>.$

116. The computer program of claim 107, wherein said intensity gradient defines a gradient in terms of differences between pixel locations with time as a constant.

117. The computer program of claim 107, wherein said intensity gradient defines a gradient in terms of pixel values between portions of images taken at different points in time.

118. The computer program of claim 107, wherein a large value of said intensity gradient indicates sharp edges within a portion of an image.

119. The computer program of claim 71, further comprising executable code that:

determines that at least one of the following conditions is present for said portion of said image;

an increase in overall image intensity, a decrease in image sharpness, and a decrease in image contrast;

determines an intermediate positive indicator for fog presence if it is determined that said at least one of the following conditions is present, and it is determined using said at least one visual characteristic that said change associated with said at least one visual characteristic is approximately uniform with respect to the portion of the image; and

determines a final positive indicator for fog presence if said intermediate positive indicator indicates that there is fog which is confirmed by at least one other feature.

120. The computer program of claim 119, wherein said at least one other feature is a non-image feature.

121. The computer program of claim 120, wherein said at least one other feature includes at least one of temperature, humidity and pressure.

122. The computer program of claim 119, wherein a plurality of intermediate positive indicators are used in determining said final positive indicator.

123. The computer program of claim 119, further comprising executable code that:

distinguishes fog from one of a plurality of other conditions, wherein said plurality of other conditions includes smoke and an aerosol dispersion.

124. The computer program of claim 71, wherein said portion is an entire image.

125. The computer program of claim 71, wherein said portion includes a plurality of regions of said image.

126. The computer program of claim 125, wherein each of said plurality of regions is a predefined shape in accordance with lighting and camera view.

127. The computer program of claim 71, wherein said at least one visual characteristic is a frequency-based feature.

128. The computer program of claim 127, wherein said frequency-based feature estimates motion of an element of said portion.

129. The computer program of claim 71, further comprising:

executable code that receives an image from a CCD camera.

130. The computer program of claim 129, wherein said CCD camera has an operational wavelength sensitivity between approximately 770 and 1200 nanometers blocking visible light.

131. The computer program of claim 129, wherein said CCD camera is a conventional CCD camera with an operational wavelength sensitivity between approximately 400 and 1200 nanometers.

132. The computer program of claim 131, wherein said CCD camera is used when it is determined that a view area is completely obscured except for a predetermined space within which said CCD camera is included.

133. The computer program of claim 131, wherein said operational wavelength sensitivity of said CCD camera excludes a portion of the range between approximately 400 and 1200 nanometers.

134. The computer program of claim 133, wherein said at least one excluded portion has a range corresponding to one of:

a light source and a device that emits within said at least one excluded portion to filter out wavelengths within said at least one excluded portion.

135. The computer program of claim 129, wherein said CCD camera has an operational wavelength sensitivity approximating that of visible light.

136. The computer program of claim 71, further comprising:

executable code that receives an image from a camera with an associated light source wherein said camera is mounted opposite said associated light source within a viewing area.

137. The computer program of claim 136, wherein said viewing area is an aircraft cargo bay, and said camera and said associated light source are mounted within a predetermined distance from a ceiling of said aircraft cargo bay.

138. The computer program of claim 137, wherein said camera and said associated light source are positioned at a same vertical and horizontal location on walls of said cargo bay area.