Method and system for detecting sea-surface oil

• US 9,104,918 B2
• Filed: 08/20/2013
• Issued: 08/11/2015
• Est. Priority Date: 08/20/2012
• Status: Active Grant

• Current Assignee: Intellective Ai, Inc.
• Original Assignee: Behavioral Recognition Systems Incorporated
• Inventors: Cobb, Wesley Kenneth
• Primary Examiner(s): ANSARI, TAHMINA N
• Application Number:
  US13/971,027
• Publication Number:
  US 20140050355A1
• Time in Patent Office: 721 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/30108 Industrial image inspection
• CPC Class Codes: G06T 7/0004 Industrial image inspection
• CPC Class Codes: G06T 7/10 Segmentation; Edge detection motion-based segmentation G06T7/215
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

A behavioral recognition system may include both a computer vision engine and a machine learning engine configured to observe and learn patterns of behavior in video data. Certain embodiments may be configured to detect and evaluate the presence of sea-surface oil on the water surrounding an offshore oil platform. The computer vision engine may be configured to segment image data into detected patches or blobs of surface oil (foreground) present in the field of view of an infrared camera (or cameras). A machine learning engine may evaluate the detected patches of surface oil to learn to distinguish between sea-surface oil incident to the operation of an offshore platform and the appearance of surface oil that should be investigated by platform personnel.

Claims

1. A computer-implemented method for analyzing a scene depicted in an input stream of video frames, the method comprising:

for one or more of the video frames;

identifying one or more foreground blobs in the video frames,

wherein each foreground blob corresponds to one or more contiguous pixels of the video frame determined to depict sea-surface oil; and

evaluating the one or more foreground blobs to derive expected patterns of observations of sea-surface oil.

2. The method of claim 1, wherein the input stream of video frames is generated by combining filtered video frames generated by three or more long-wavelength infrared (LWIR) cameras whose signals are filtered by band-pass filters allowing light in respective wavelength ranges to pass.

3. The method of claim 2, wherein the wavelength ranges are substantially 8-9 micrometers, 8-11.5 micrometers, and 8-13 micrometers.

4. The method of claim 2, wherein the signals of the LWIR cameras are further filtered by one or more polarizing filters.

5. The method of claim 2, wherein the LWIR cameras are mounted to observe an area surrounding an offshore oil platform.

6. The method of claim 2, wherein combining the video frames includes determining differences between pairs of the filtered video frames generated by respective LWIR cameras, and wherein the input stream of video frames is directly proportional to the differences which correspond to contrasts between wavelength ranges in which difference between radiance from seawater is substantially greater than difference between radiance from oil, and inversely proportional to differences which correspond to contrast between wavelength ranges in which difference between radiance from oil is substantially greater than difference between radiance from seawater, or vice versa.

7. The method of claim 1, further comprising:

receiving a set of foreground blobs identified in a subsequent one of the video frames; and

upon determining that at least a first one of the foreground blobs does not correspond to at least one of the expected patterns of occurrences of sea-surface oil, generating an alert message.

8. The method of claim 1, wherein the foreground blobs are evaluated based on one or more features associated with the foreground blobs, wherein the features include at least one of locations of the blobs, sizes of the blobs, rates of change in blob sizes, a measure of intensity of the blobs, motion characteristics of the blobs, whether the blobs have non-sharp edges, whether the blobs have high fractal dimension, and whether the blobs are asymmetrical.

9. The method of claim 1, wherein the expected patterns of observations of sea-surface oil are learned using an adaptive resonance theory (ART) network or a combination of an ART network and a self-organizing map (SOM).

10. A non-transitory computer-readable storage medium storing instructions, which when executed by a computer system, perform operations for analyzing a scene depicted in an input stream of video frames, the operations comprising:

for one or more of the video frames;

identifying one or more foreground blobs in the video frames,

wherein each foreground blob corresponds to one or more contiguous pixels of the video frame determined to depict sea-surface oil; and

evaluating the one or more foreground blobs to derive expected patterns of observations of sea-surface oil.

11. The computer-readable storage medium of claim 10, wherein the input stream of video frames is generated by combining filtered video frames generated by three or more long-wavelength infrared (LWIR) cameras whose signals are filtered by band-pass filters allowing light in respective wavelength ranges to pass.

12. The computer-readable storage medium of claim 11, wherein the wavelength ranges are substantially 8-9 micrometers, 8-11.5 micrometers, and 8-13 micrometers.

13. The computer-readable storage medium of claim 11, wherein the signals of the LWIR cameras are further filtered by one or more polarizing filters.

14. The computer-readable storage medium of claim 11, wherein the LWIR cameras are mounted to observe an area surrounding an offshore oil platform.

15. The computer-readable storage medium of claim 11, wherein combining the video frames includes determining differences between pairs of the filtered video frames generated by respective LWIR cameras, and wherein the input stream of video frames is directly proportional to the differences which correspond to contrasts between wavelength ranges in which difference between radiance from seawater is substantially greater than difference between radiance from oil, and inversely proportional to differences which correspond to contrast between wavelength ranges in which difference between radiance from oil is substantially greater than difference between radiance from seawater, or vice versa.

16. The computer-readable storage medium of claim 10, the operations further comprising:

after deriving the expected patterns of occurrences of sea-surface oil;

receiving a set of foreground blobs identified in a subsequent one of the video frames; and

upon determining that at least a first one of the foreground blobs does not correspond to at least one of the expected patterns of occurrences of sea-surface oil, generating an alert message.

17. The computer-readable storage medium of claim 10, wherein the foreground blobs are evaluated based on one or more features associated with the foreground blobs, wherein the features include at least one of locations of the blobs, sizes of the blobs, rates of change in blob sizes, a measure of intensity of the blobs, motion characteristics of the blobs, whether the blobs have non-sharp edges, whether the blobs have high fractal dimension, and whether the blobs are asymmetrical.

18. The computer-readable storage medium of claim 10, wherein the expected patterns of observations of sea-surface oil are learned using an adaptive resonance theory (ART) network or a combination of an ART network and a self-organizing map (SOM).

19. A system, comprising:

a processor; and

a memory, wherein the memory includes an application program configured to perform operations for analyzing a scene depicted in an input stream of video frames, the operations comprising;

for one or more of the video frames;

identifying one or more foreground blobs in the video frames,

wherein each foreground blob corresponds to one or more contiguous pixels of the video frame determined to depict sea-surface oil, and

evaluating the one or more foreground blobs to derive expected patterns of observations of sea-surface oil.

20. The system of claim 19, wherein the input stream of video frames is generated by combining filtered video frames generated by three or more long-wavelength infrared (LWIR) cameras whose signals are filtered by band-pass filters allowing light in respective wavelength ranges to pass.

21. The system of claim 20, wherein the LWIR cameras are mounted to observe an area surrounding an offshore oil platform.

22. The system of claim 20, wherein combining the video frames includes determining differences between pairs of the filtered video frames generated by respective LWIR cameras, and wherein the input stream of video frames is directly proportional to the differences which correspond to contrasts between wavelength ranges in which difference between radiance from seawater is substantially greater than difference between radiance from oil, and inversely proportional to differences which correspond to contrast between wavelength ranges in which difference between radiance from oil is substantially greater than difference between radiance from seawater, or vice versa.

23. The system of claim 19, the operations further comprising:

receiving a set of foreground blobs identified in a subsequent one of the video frames; and

upon determining that at least a first one of the foreground blobs does not correspond to at least one of the expected patterns of occurrences of sea-surface oil, generating an alert message.

24. The system of claim 19, wherein the foreground blobs are evaluated based on one or more features associated with the foreground blobs, wherein the features include at least one of locations of the blobs, sizes of the blobs, rates of change in blob sizes, a measure of intensity of the blobs, motion characteristics of the blobs, whether the blobs have non-sharp edges, whether the blobs have high fractal dimension, and whether the blobs are asymmetrical.

25. A method of analyzing a scene depicted in an input stream of video frames, comprising:

for one or more of the video frames;

identifying one or more foreground blobs in the video frame, wherein each foreground blob corresponds to contiguous pixels of the video frame determined by a behavior recognition system to depict a patch of sea-surface oil, wherein the behavior recognition system is configured to learn to distinguish between foreground objects depicting patches of sea-surface oil and false positive detections of patches of sea-surface oil resulting from noise occurring in the one or more video frames; and

upon determining one of the foreground blobs depicts a patch of sea-surface oil deviates from expected patterns of sea-surface oil derived by the behavior recognition system, generating an alert message.

26. The method of claim 25, wherein the noise comprises one or more of lighting, absorption, and extinction artifacts in the one or more video frames.

27. The method of claim 25, wherein the expected patterns of observations of sea-surface oil are derived by the behavior recognition system using an adaptive resonance theory (ART) network or a combination of an ART network and a self-organizing map (SOM).