Method of filming high dynamic range videos

• US 9,137,454 B2
• Filed: 11/11/2013
• Issued: 09/15/2015
• Est. Priority Date: 07/29/2013
• Status: Expired due to Fees

• Current Assignee: Quanta Computer
• Original Assignee: Quanta Computer
• Inventors: Yang, Wen-Chu, Lin, Keng-Sheng
• Primary Examiner(s): Velez, Roberto
• Assistant Examiner(s): Kao, Yih-Sien
• Application Number:
  US14/076,440
• Publication Number:
  US 20150029354A1
• Time in Patent Office: 673 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10144 Varying exposure
• CPC Class Codes: G06T 2207/20208 High dynamic range [HDR] image processing
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/92 based on global image properties
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors

Abstract

The present invention discloses a method of filming a high dynamic range video. The method includes: using an image sensor to capture an original frame which is interlaced by a plurality of long exposure areas and a plurality of short exposure areas; forming a long exposure field via a plurality of long exposure areas and forming a short exposure field via a plurality of short exposure areas; forming a reconstructed long exposure field and a reconstructed short exposure field having the same resolution as the original frame via a reconstruction process by a pixel value of each pixel in the long exposure field and the short exposure field; and forming a high dynamic range image based on the pixel value of each pixel of the reconstructed long exposure field and the reconstructed short exposure field via a merging process.

Claims

1. A method of filming a high dynamic range video, applied for an electronic product having an image sensor, and the method comprising the following steps:

using the image sensor to capture an original image which is interlaced by a plurality of long exposure areas and a plurality of short exposure areas, and the original image comprising a plurality of pixels;

forming the plurality of long exposure areas as a long exposure field, and forming the plurality of short exposure areas as a short exposure field;

forming a reconstructed long exposure field and a reconstructed short exposure field having the same resolution as the original image based on a pixel value of each pixel of the long exposure field and the short exposure field via a reconstruction process; and

applying the pixel value of each pixel of the reconstructed long exposure field and the reconstructed short exposure field to form a high dynamic range image via a merging process;

wherein the pixel value is an integer within 0 to 2x−

1 and x is a number of digits of the image sensor, different pixel values correspond to different levels of illuminance, and a ratio of a long exposure duration of the plurality of long exposure area to a short exposure duration of the plurality of short exposure area is a exposure duration ratio (R), and

wherein the reconstruction process and the merging process comprise performing the following steps on each pixel of missing parts of the plurality of the long exposure areas;

when a pixel value of a short exposure area, which is corresponding to a pixel of the missing parts of the plurality of long exposure areas, being less than a third preset value, an average of two pixel values, which is corresponding to the pixel of the missing parts of the plurality of long exposure areas, of previous and next long exposure areas being used as a pixel value of the pixel value of the missing parts of the plurality of long exposure areas; and

when the pixel value of the short exposure area, which is corresponding to the pixel of the missing parts of the plurality of long exposure areas, being between the third preset value and a fourth value, the pixel value of the short exposure area being multiplied by R to replace the pixel value of the pixel of the missing parts of the plurality of long exposure areas.

2. The method of filming a high dynamic range video of claim 1, wherein each long exposure area comprises at least one row of pixels and each short exposure area comprises at least one row of pixels.

3. The method of filming a high dynamic range video of claim 1, wherein exposure time of the first row and the second row of the image sensor is the long exposure duration, and exposure time of the third row and the fourth row of the image sensor is the short exposure duration.

4. The method of filming a high dynamic range video of claim 1, wherein the reconstruction process comprises steps of:

dividing the pixel value of each pixel of the plurality of the long exposure area by the exposure duration ratio (R) to fill up missing parts of the plurality of the short exposure area and combine with the plurality of the short exposure area to form the reconstructed short exposure field; and

multiplying the pixel value of each pixel of the plurality of the short exposure area by the exposure duration ratio (R) to fill up missing parts of the plurality of the long exposure area and combine with the plurality of the long exposure area to form the reconstructed long exposure field.

5. The method of filming a high dynamic range video of claim 4, wherein the merging process comprises a one-by-one comparison of the pixel value of each pixel of the reconstructed long exposure field and the corresponding pixel value of each pixel of the reconstructed short exposure field, and perform the following steps based on the comparison result:

replacing the pixel values of the reconstructed long exposure field by the pixel values of the reconstructed short exposure field multiplied by R to form the pixel values of the high dynamic range image when the pixel values of the reconstructed long exposure field are greater than a first preset value; and

replacing the pixel values of the reconstructed short exposure field by the pixel values of the reconstructed long exposure field divided by R to form the pixel values of the high dynamic range image when the pixel values of the reconstructed short exposure field are less than a second preset value.

6. The method of filming a high dynamic range video of claim 5, wherein the merging process further comprises steps of:

extending the long exposure duration of the plurality of the long exposure area when the pixel values of the reconstructed long exposure field are less than the second preset value;

shortening the short exposure duration of the plurality of the short exposure area when the pixel values of the reconstructed short exposure field are greater than the first preset value; and

taking the pixel values of the reconstructed long exposure field as the pixel values of the high dynamic range image when the pixel values of the reconstructed long exposure field and the reconstructed short exposure field are all between the first preset value and the second preset value.

7. The method of filming a high dynamic range video of claim 6, wherein the first preset value is 250 and the second preset value is 10.

8. The method of filming a high dynamic range video of claim 1, wherein the reconstruction process comprises performing the following steps on each pixel of the missing part of the plurality of the short exposure areas:

when a pixel value of a long exposure area, which is corresponding to a pixel of the missing parts of the plurality of short exposure areas, being greater than a fourth preset value, an average of two pixel values, which is corresponding to the pixel of the missing parts of the plurality of short exposure areas, of previous and next short exposure areas being used as the pixel value of the pixel of the missing parts of the plurality of short exposure areas; and

when the pixel value of the long exposure area, which is corresponding to the pixel of the missing parts of the plurality of short exposure areas, being between the third preset value and a fourth value, the pixel value of the short exposure area being divided by R to replace the pixel value of the pixel the missing parts of the plurality of short exposure areas.

9. The method of filming a high dynamic range video of claim 8, wherein the third preset value is 10 and the fourth preset values is 250.