Method for distinguishing a smoke condition from a dust condition using video images

US 7,729,510 B2
Filed: 07/31/2008
Issued: 06/01/2010
Est. Priority Date: 05/20/2002
Status: Expired due to Fees

First Claim

Patent Images

1. A method for distinguishing a smoke condition from a dust condition comprising:

obtaining a first frame of video data;

determining at least one feature of a portion of said first frame; and

determining a smoke condition if a threshold value for said at least one feature is crossed, and, in the presence of a dust condition, said threshold value is not crossed.

View all claims

0 Assignments

Timeline View

Assignment View

0 Petitions

Accused Products

Abstract

Detecting video phenomena, such as fire in an aircraft cargo bay, includes receiving a plurality of video images from a plurality of sources, compensating the images to provide enhanced images, extracting features from the enhanced images, and combining the features from the plurality of sources to detect the video phenomena. Extracting features may include determining an energy indicator for each of a subset of the plurality of frames. Detecting video phenomena may also include comparing energy indicators for each of the subset of the plurality of frames to a reference frame. The reference frame corresponds to a video frame taken when no fire is present, video frame immediately preceding each of the subset of the plurality of frames, or a video frame immediately preceding a frame that is immediately preceding each of the subset of the plurality of frames. Image-based and non-image based techniques are described herein in connection with fire detection and/or verification and other applications.

58 Citations

View as Search Results

58 Claims

1. A method for distinguishing a smoke condition from a dust condition comprising:
- obtaining a first frame of video data;
  
  determining at least one feature of a portion of said first frame; and
  
  determining a smoke condition if a threshold value for said at least one feature is crossed, and, in the presence of a dust condition, said threshold value is not crossed.
- View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29)
- - 2. The method of claim 1, further comprising:
    - setting a fire indicator if said threshold value is crossed.
  - 3. The method of claim 1, wherein said at least one feature is an average absolute gray level intensity represented as:
    - $μ (t) = \frac{1}{N_{m} \times M_{m}} \sum_{i = 1}^{N_{m}} \sum_{j = 1}^{M_{m}} \langle P (i, j; t) - P_{Ref} (i, j; t_{0}) \rangle$ where;
      
      P(i, j;
      
      t) is a pixel value at location (i,j) and time t;
      
      P_Ref(i, j;
      
      t₀) is a pixel value at location (i,j) of a reference frame at time 0; and
      
      N_mand M_mare dimensions of said portion.
  - 4. The method of claim 3, wherein said first frame is obtained using one of:
    - an overhead lighting view in which only overhead controlled lights are turned on, a diagonal lighting view in which only a light source opposite a camera obtaining said first frame is on, and a pilot lighting view in which only a light source co-located with a camera obtaining said first frame is on.
  - 5. The method of claim 3, wherein, if said threshold value is crossed, a smoke condition is determined.
  - 6. The method of claim 5, wherein said threshold value is determined in accordance with a vibration level of noise in a system.
  - 7. The method of claim 6, further comprising:
    - selecting said threshold in accordance with criteria that all fire conditions are properly classified and an acceptable rate of dust cases are improperly classified.
  - 8. The method of claim 7, wherein said criteria include detecting a fire condition prior to a selected conventional smoke detector.
  - 9. The method of claim 3, wherein said feature has a larger value in a smoke condition than a dust condition.
  - 10. The method of claim 1, wherein said feature includes a plurality of features each associated with at least one threshold.
  - 11. The method of claim 10, wherein said plurality of features includes at least two of:
    - a mean intesity value of an average pixel intensity for all pixels in said portion;
      
      a maximum pixel intensity value of a maximum pixel intensity for all pixels in said portion;
      
      a number of pixels in a bright region, said number representing a number of pixels in said portion that are one of;
      
      greater than a threshold intensity, and greater than a threshold intensity and connected to a minimum number of neighboring pixels;
      
      a mean norm of intensity gradient vector for all pixels in said portion; and
      
      a norm of the second order moment matrix calculated over all pixels in said portion.
  - 12. The method of claim 11, wherein a fire condition is determined if any one of the plurality of features crosses an associated threshold.
  - 13. The method of claim 11, wherein different features are determined and used in detecting a smoke condition in accordance with the particular lighting view of a frame.
  - 14. The method of claim 13, wherein said mean intensity value is determined for a frame obtained in one of a diagonal and overhead lighting view, said maximum pixel intensity is determined for a frame obtained in one of diagonal view, and said size of a brightness region is determined for a frame obtained in one of a diagonal and overhead lighting views.
  - 15. The method of claim 14, wherein a different threshold is used for each combination of feature and lighting view for at least one feature.
  - 16. The method of claim 15, wherein said plurality of features includes a feature that is a mean intensity represented as:
    - $mean (P) = \frac{1}{MN} \sum_{i = 1}^{M} \sum_{j = 1}^{N} P (i, j)$ where;
      
      M represents a number of pixel rows in said portion;
      
      N represents a number of pixel columns in said portion;
      
      P is said portion of a frame; and
      
      P(i,j) is a pixel intensity or gray level at pixel position i,j in P.
  - 17. The method of claim 16, wherein said plurality of features includes a second feature that is a mean norm of a gradient vector represented as:
    - $mgrad (P) = \frac{1}{MN} \sum_{i = 1}^{M - 1} \sum_{j = 1}^{N - 1} \sqrt{{G_{1} (i, j)}^{2} + {G_{2} (i, j)}^{2}}$ where G₁and G₂are given by;
      
      G₁(i,j)=X(i+1,j)−
      
      X(i,j)
      G₂(i,j)=X(i,j+1)−
      
      X(i,j).
  - 18. The method of claim 17, wherein a value of said second feature decreases as an image decreases in sharpness of detail.
  - 19. The method of claim 17, wherein a third feature of said plurality of features is a norm of a second order moment matrix represented as:
    - $norm 2 (P) = norm ([\begin{matrix} SM 20 (P) & SM 11 (P) \\ SM 11 (P) & SM 02 (P) \end{matrix}])$ where second order moments SM20, SM02 and SM11 are defined via equations;
      
      $X center (P) = \frac{1}{MN} \sum_{i = 1}^{M} \sum_{j = 1}^{N} iP (i, j)$ $Y center (P) = \frac{1}{MN} \sum_{i = 1}^{M} \sum_{j = 1}^{N} jP (i, j)$ $SM 20 (P) = \frac{1}{MN} \sum_{i = 1}^{M} \sum_{j = 1}^{N} {(i - X center)}^{2} P (i, j)$ $SM 02 (P) = \frac{1}{MN} \sum_{i = 1}^{M} \sum_{j = 1}^{N} {(j - Y center)}^{2} P (i, j)$ $and$ $SM 11 (P) = \frac{1}{MN} \sum_{i = 1}^{M} \sum_{j = 1}^{N} (i - X center) (j - Y center) P (i, j) .$
  - 20. The method of claim 19, wherein said plurality of features includes a fourth feature that is the size of a bright region represented as:
    - $br size (P) = \sum_{i = 1}^{M} \sum_{j = 1}^{N} T (i, j)$ where binary thresholded image T is defined through at least an intensity threshold.
  - 21. The method of claim 20, wherein a fire condition is determined if any one value of any one feature crosses a threshold.
  - 22. The method of claim 20, further comprising:
    - determining said first, said second and said fourth features if said first frame is obtained using a diagonal lighting view in which said frame is obtained using only an opposing camera light; and
      
      determining said first, said second and said third features if said first frame is obtained using an overhead lighting view in which said frame is obtained using only overhead lights.
  - 23. The method of claim 22, wherein at least one feature is determined for multiple windows of said portion.
  - 24. The method of claim 23,wherein each of said multiple windows is determined in accordance with a location of at least one light relative to location and field of view of a camera acquiring frames.
  - 25. The method of claim 22, wherein at least one of said plurality of features is determined using a relative change expressed as a ratio of a first value to a second value, said first value representing said feature for said portion and said second value representing a steady state feature value.
  - 26. The method of claim 20, wherein said plurality of features includes a fifth feature that is a maximum pixel intensity value represented as:
    - $\max (P) = \max_{\underset{j = 1, N}{i = 1, M}} P (i, j)$ where;
      
      M represents a number of pixel rows in said portion;
      
      N represents the number of pixel columns in said portion;
      
      P is said portion;
      
      P(i,j) is a pixel intensity or gray level at pixel position i,j in said portion.
  - 27. The method of claim 11, wherein said bright region is determined in accordance with an intensity threshold and a connectivity factor.
  - 28. The method of claim 1, wherein said at least one feature is a rate of change feature based on an average absolute gray level, wherein the rate of change is calculated as:
    - $δ_{μ} = \frac{μ (t) - μ (t - Δ t)}{Δ t}$ where μ
      
      (t) and μ
      
      (t−
      
      Δ
      
      t) are values of an average absolute intensity feature at times t and t−
      
      Δ
      
      t, respectively, and wherein said average absolute gray level is represented as;
      
      $μ (t) = \frac{1}{N_{m} \times M_{m}} \sum_{i = 1}^{N_{m}} \sum_{j = 1}^{M_{m}} \langle P (i, j; t) - P_{Ref} (i, j; t_{0}) \rangle$ where;
      
      P(i,j;
      
      t) is a pixel value at location (i,j) and time t;
      
      P_Ref(i,j;
      
      t₀) is a pixel value at location (i,j) of a reference frame at time 0; and
      
      N_mand M_mare dimensions of said portion.
  - 29. The method of claim 28, wherein a value of said rate of change feature is higher for a smoke condition than for a dust condition.

30. A computer program product embodied on a non-transitory computer-readable medium that distinguishes a smoke condition from a dust condition comprising executable code that:
- obtains a first frame of video data;
  
  determines at least one feature of a portion of said first frame; and
  
  determines a smoke condition if a threshold value for said at least one feature is crossed, and, in the presence of a dust condition, said threshold value is not crossed.
- View Dependent Claims (31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58)
- - 31. The computer program product of claim 30, further comprising:
    - executable code that sets a fire indicator if said threshold value is crossed.
  - 32. The computer program product of claim 31, wherein said at least one feature is an average absolute gray level intensity represented as:
    - $μ (t) = \frac{1}{N_{m} \times M_{m}} \sum_{i = 1}^{N_{m}} \sum_{j = 1}^{M_{m}} \langle P (i, j; t) - P_{Ref} (i, j; t_{0}) \rangle$ where;
      
      P(i,j;
      
      t) is a pixel value at location (i,j) and time t;
      
      P_Ref(i,j;
      
      t₀) is a pixel value at location (i,j) of a reference frame at time 0; and
      
      N_mand M_mare dimensions of said portion.
  - 33. The computer program product of claim 32, wherein said first frame is obtained using one of:
    - an overhead lighting view in which only overhead controlled lights are turned on, a diagonal lighting view in which only a light source opposite a camera obtaining said first frame is on, and a pilot lighting view in which only a light source co-located with a camera obtaining said first frame is on.
  - 34. The computer program product of claim 32, wherein, if said threshold value is crossed, a smoke condition is determined.
  - 35. The computer program product of claim 34, further comprising executable code that determines said threshold value in accordance with a vibration level of noise in a system.
  - 36. The computer program product of claim 35, further comprising:
    - executable code that selects said threshold in accordance with criteria that all fire conditions are properly classified and an acceptable rate of dust cases are improperly classified.
  - 37. The computer program product of claim 36, wherein said criteria include detecting a fire condition prior to a selected conventional smoke detector.
  - 38. The computer program product of claim 32, wherein said feature has a larger value in a smoke condition than a dust condition.
  - 39. The computer program product of claim 30, wherein said feature includes a plurality of features each associated with at least one threshold.
  - 40. The computer program product of claim 39, wherein said plurality of features includes at least two of:
    - a mean intesity value of an average pixel intensity for all pixels in said portion;
      
      a maximum pixel intensity value of a maximum pixel intensity for all pixels in said portion;
      
      a number of pixels in a bright region, said number representing a number of pixels in said portion that are one of;
      
      greater than a threshold intensity, and greater than a threshold intensity and connected to a minimum number of neighboring pixels;
      
      a mean norm of intensity gradient vector for all pixels in said portion; and
      
      a norm of the second order moment matrix calculated over all pixels in said portion.
  - 41. The computer program product of claim 40, wherein a fire condition is determined if any one of the plurality of features crosses an associated threshold.
  - 42. The computer program product of claim 40, further comprising executable code that determines different features used in detecting a smoke condition in accordance with a particular lighting view of a frame.
  - 43. The computer program product of claim 42, wherein said mean intensity value is determined for a frame obtained in one of a diagonal and overhead lighting view, said maximum pixel intensity is determined for a frame obtained in one of diagonal view, and said size of a brightness region is determined for a frame obtained in one of a diagonal and overhead lighting views.
  - 44. The computer program product of claim 43, wherein a different threshold is used for each combination of feature and lighting view for at least one feature.
  - 45. The computer program product of claim 39, wherein said plurality of features includes a feature that is a mean intensity represented as:
    - $mean (P) = \frac{1}{MN} \sum_{i = 1}^{M} \sum_{j = 1}^{N} P (i, j)$ where;
      
      M represents a number of pixel rows in said portion;
      
      N represents a number of pixel columns in said portion;
      
      P is said portion of a frame; and
      
      P(i,j) is a pixel intensity or gray level at pixel position i,j in P.
  - 46. The computer program product of claim 45, wherein said plurality of features includes a second feature that is a mean norm of a gradient vector represented as:
    - $mgrad (P) = \frac{1}{MN} \sum_{i = 1}^{M - 1} \sum_{j = 1}^{N - 1} \sqrt{{G_{1} (i, j)}^{2} + {G_{2} (i, j)}^{2}}$ where G₁and G₂are given by;
      
      G₁(i,j)=X(i+1,j)−
      
      X(i,j)
      G₂(i,j)=X(i,j+1)−
      
      X(i,j).
  - 47. The computer program product of claim 46, wherein a value of said second feature decreases as an image decreases in sharpness of detail.
  - 48. The computer program product of claim 46, wherein a third feature of said plurality of features is a norm of a second order moment matrix represented as:
    - $norm 2 (P) = norm ([\begin{matrix} SM 20 (P) & SM 11 (P) \\ SM 11 (P) & SM 02 (P) \end{matrix}])$ where second order moments SM20, SM02 and SM11 are defined via equations;
      
      $X center (P) = \frac{1}{MN} \sum_{i = 1}^{M} \sum_{j = 1}^{N} iP (i, j)$ $Y center (P) = \frac{1}{MN} \sum_{i = 1}^{M} \sum_{j = 1}^{N} jP (i, j)$ $SM 20 (P) = \frac{1}{MN} \sum_{i = 1}^{M} \sum_{j = 1}^{N} {(i - X center)}^{2} P (i, j)$ $SM 02 (P) = \frac{1}{MN} \sum_{i = 1}^{M} \sum_{j = 1}^{N} {(j - Y center)}^{2} P (i, j)$ $and$ $SM 11 (P) = \frac{1}{MN} \sum_{i = 1}^{M} \sum_{j = 1}^{N} (i - X center) (j - Y center) P (i, j) .$
  - 49. The computer program product of claim 48, wherein said plurality of features includes a fourth feature that is the size of a bright region represented as:
    - $br size (P) = \sum_{i = 1}^{M} \sum_{j = 1}^{N} T (i, j)$ where binary thresholded image T is defined through at least an intensity threshold.
  - 50. The computer program product of claim 49, further comprising executable code that determines said bright region in accordance with an intensity threshold and a connectivity factor.
  - 51. The computer program product of claim 49, wherein a fire condition is determined if any one value of any one feature crosses a threshold.
  - 52. The computer program product of claim 49, further comprising:
    - executable code that determines said first, said second and said fourth features if said first frame is obtained using a diagonal lighting view in which said frame is obtained using only an opposing camera light; and
      
      executable code that determines said first, said second and said third features if said first frame is obtained using an overhead lighting view in which said frame is obtained using only overhead lights.
  - 53. The computer program product of claim 52, wherein at least one feature is determined for multiple windows of said portion.
  - 54. The computer program product of claim 53,wherein each of said multiple windows is determined in accordance with a location of at least one light relative to location and field of view of a camera acquiring frames.
  - 55. The computer program product of claim 52, wherein at least one of said plurality of features is determined using a relative change expressed as a ratio of a first value to a second value, said first value representing said feature for said portion and said second value representing a steady state feature value.
  - 56. The computer program product of claim 49, wherein said plurality of features includes a fifth feature that is a maximum pixel intensity value represented as:
    - $\max (P) = \max_{\underset{j = 1, N}{i = 1, M}} P (i, j)$ where;
      
      M represents a number of pixel rows in said portion;
      
      N represents the number of pixel columns in said portion;
      
      P is said portion;
      
      P(i,j) is a pixel intensity or gray level at pixel position i,j in said portion.
  - 57. The computer program product of claim 30, wherein said at least one feature is a rate of change feature based on an average absolute gray level, wherein the rate of change is calculated as:
    - $δ_{μ} = \frac{μ (t) - μ (t - Δ t)}{Δ t}$ where μ
      
      (t) and μ
      
      (t−
      
      Δ
      
      t) are values of an average absolute intensity feature at times t and t−
      
      Δ
      
      t, respectively, and wherein said average absolute gray level is represented as;
      
      $μ (t) = \frac{1}{N_{m} \times M_{m}} \sum_{i = 1}^{N_{m}} \sum_{j = 1}^{M_{m}} \langle P (i, j; t) - P_{Ref} (i, j; t_{0}) \rangle$ where;
      
      P(i,j;
      
      t) is a pixel value at location (i,j) and time t;
      
      P_Ref(i,j;
      
      t₀) is a pixel value at location (i,j) of a reference frame at time 0; and
      
      N_mand M_mare dimensions of said portion.
  - 58. The computer program product of claim 57, wherein a value of said rate of change feature is higher for a smoke condition than for a dust condition.

Specification

Resources

Litigation Campaign Assessment

Current Assignee
Simmonds Precision Products, Inc. (Raytheon Technologies Corporation d/b/a RTX)
Original Assignee
Simmonds Precision Products, Inc. (Raytheon Technologies Corporation d/b/a RTX)
Inventors
Shirer, Jeffrey James, Zeliff, Robert Lowell, Zakrzewski, Radoslaw Romuald, Sadok, Mokhtar
Primary Examiner(s)
Johns; Andrew W

Application Number

US12/221,244
Publication Number

US 20090016609A1
Time in Patent Office

670 Days
Field of Search

382/100, 382/190, 382/206, 382/270, 382/286, 356/436, 356/438, 250/564, 340/628, 340/630
US Class Current

382/100
CPC Class Codes

B64D 45/0053   using visual equipment, e.g...

G06F 18/24765   Rule-based classification

G06F 18/253   of extracted features

G06F 18/256   of results relating to diff...

G06T 7/254   involving subtraction of im...

G06V 10/765   using rules for classificat...

G06V 10/806   of extracted features

G06V 10/811   the classifiers operating o...

G06V 20/52   Surveillance or monitoring ...

G08B 17/125   by using a video camera to ...

Method for distinguishing a smoke condition from a dust condition using video images

First Claim

0 Assignments

0 Petitions

Accused Products

Abstract

58 Citations

58 Claims

Specification

Use Cases

Quick Links

Others

Method for distinguishing a smoke condition from a dust condition using video images

First Claim

0 Assignments

Subscription Required

Subscription Required

0 Petitions

Subscription Required

Accused Products

Subscription Required

Abstract

58 Citations

58 Claims

Specification

Subscription Required

Use Cases

Quick Links

Others