High dynamic range video capture using variable lighting

• US 10,489,896 B2
• Filed: 11/14/2017
• Issued: 11/26/2019
• Est. Priority Date: 11/14/2017
• Status: Active Grant

• Current Assignee: General Electric Company
• Original Assignee: General Electric Company
• Inventors: Sibbach, Arthur William
• Primary Examiner(s): Dunphy, David F
• Application Number:
  US15/812,017
• Publication Number:
  US 20190147571A1
• Time in Patent Office: 742 Days
• CPC Class Codes: F01D 17/20 Devices dealing with sensing elements or final actuators or transmitting means between them, e.g. power-assisted sensing elements alone F01D17/02; final actuators alone F01D17/10
• CPC Class Codes: F01D 21/003 Arrangements for testing or measuring for measuring vibrations G01H
• CPC Class Codes: F05D 2270/8041 Cameras
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10068 Endoscopic image
• CPC Class Codes: G06T 2207/20208 High dynamic range [HDR] image processing
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/30164 Workpiece; Machine component
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/92 based on global image properties
• CPC Class Codes: G06T 7/001 using an image reference approach
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors

Abstract

A system and method is generally provided for producing a high dynamic range video. The system includes a probe and a computer communicatively coupled to the probe. The probe includes a camera and a light source. Further, the probe produces a plurality of frames. The computer includes at least one processor and at least one memory device. The memory device contains instructions configured to combine the plurality of frames into composite frames. The instructions are further configured to filter out portions of frames captured below a first threshold of light and portions of frames captured above a second threshold of light. The light source modulates a lighting characteristic, and the camera captures a first video with the modulated light source to produce a plurality of frames defining a plurality of lighting characteristics. Further, the computer combines the plurality of frames into a second video including a plurality of composite frames.

Claims

1. A method of producing a high dynamic range video using a probe comprising a camera and a light source, wherein the probe produces a plurality of frames, the method comprising:

repeatedly modulating the light source through a sequence of discrete lighting characteristics, wherein the sequence of discrete lighting characteristics repeats at a predetermined frequency;

synchronizing a sensor frame rate with the frequency of the modulation of the discrete lighting characteristics, wherein each frame is captured at a corresponding discrete lighting characteristic of the sequence;

producing a first video at the sensor frame rate, wherein the first video has a first frame rate, wherein the first video comprises a repeated sequence of frames corresponding to the repeated sequence of discrete lighting characteristics;

establishing a lower illumination threshold value;

establishing an upper illumination threshold value;

filtering, by use of the computer installed with video processing instructions, the sequences of frames of the first video so as to remove frame portions having an illumination value less than the lower illumination threshold value;

filtering, by use of the computer installed with video processing instructions, the sequences of frames of the first video so as to remove frame portions having an illumination value greater than the upper illumination threshold value;

generating a composite frame for each sequence of frames corresponding to the repeated sequence of discrete lighting characteristics, wherein the composite frame contains frame portions having only illumination values between the lower and upper illumination threshold values; and

generating from a computer a second video that includes a plurality of the composite frames with a high dynamic range at a second frame rate, the second frame rate lower than the first frame rate.

2. The method of claim 1, wherein the lighting characteristic comprises any combination of frequency, intensity, or amplitude of light.

3. The method of claim 1, wherein each discrete lighting characteristic defines at least one different frequency, intensity value, or an amplitude value.

4. The method of claim 1, further comprising:

transmitting the first video to the computer comprising at least one processor and at least one memory device, the memory device containing the video processing instructions.

5. The method of claim 1, wherein the modulation of the light source through the sequence of discrete lighting characteristics occurs in an ordered sequence from a lowest amplitude of light to a highest amplitude of light, from the highest amplitude of light to the lowest amplitude of light, or both.

6. The method of claim 1, wherein the modulation of the light source through the sequence of discrete lighting characteristics occurs in a random sequence of the light amplitudes.

7. The method of claim 1, wherein the lower illumination threshold value is one candela per square meter (cd/m²) and the upper illumination threshold value is two million candelas per square meter (cd/m²).

8. The method of claim 1, further comprising inserting the probe into an area with ambient light at or below one lux.

9. The method of claim 1, further comprising inserting the probe into a turbomachine.

10. The method of claim 1, further comprising detecting an undesirable characteristic on an exterior surface of an object, on an interior surface of the object, or both using the second video.

11. The method of claim 1, further comprising displaying the second video on a display.

12. The method of claim 1, wherein the first frame rate is at least one of one hundred twenty frames per second or greater or sixty frames per second.

13. The method of claim 1, wherein the second frame rate is thirty frames per second or less.

14. The method of claim 1, wherein the first frame rate is one hundred twenty frames per second or greater, and the second frame rate is twenty-four frames per second or less.

15. A system for producing a high dynamic range video, the system comprising:

a probe comprising a camera and a light source, wherein the probe produces a first video comprising a repeated sequence of frames; and

a computer comprising at least one processor and at least one memory device, the memory device containing instructions configured to combine each sequence of frames into a composite frame and combine the resultant composite frames into a second video with a high dynamic range, the instructions being further configured to filter out portions of frames having an illumination value less than a lower illumination threshold value and portions of frames having an illumination value greater than an upper illumination threshold value,

wherein the light source repeatedly modulates through a sequence of discrete lighting characteristics, wherein the sequence of discrete lighting characteristics repeats at a predetermined frequency, wherein the predetermined frequency corresponds to a frame rate of the first video so that the sequence of discrete lighting characteristics corresponds to the sequence of frames, and wherein each frame is captured at a corresponding discrete lighting characteristic of the sequence.

16. The system of claim 15, wherein the light source comprises a light emitting diode (LED).

17. The system of claim 15, wherein the probe comprises a borescope.

18. The system of claim 15, wherein the lighting characteristic comprises any combination of frequency, intensity, or amplitude of light.