Low-light video frame enhancement
First Claim
1. A method of combining image data from multiple frames to enhance one or more parameters of digital image quality, comprising using a processor and at least one digital image acquisition system including a lens and an image sensor in:
- acquiring a first image at a first exposure duration;
acquiring a second image at a second exposure duration shorter than the first exposure duration and at a time just before, just after or overlapping in time with acquiring the first image, such that said first and second images comprise approximately a same first scene, and whereby the second image is relatively sharp and under-exposed and the first image is relatively well-exposed and less sharp than the second image;
extracting brightness or color information, or both, from the first image;
applying to the second image the brightness or color information, or both, extracted from the first image to generate an enhanced version of the second image enhanced in terms of brightness or color quality, or both;
displaying, storing, transmitting, or streaming the enhanced version of the second image;
acquiring a third image at or near said first exposure duration when a predetermined number of multiple frames has been recorded since the acquiring of the first image;
acquiring a fourth image at or near said second exposure duration and at a time just before, just after or overlapping in time with the third image, such that said third and fourth images comprise approximately a same second scene different from or same as the first scene;
extracting brightness or color, or both, from the third image;
applying to the fourth image the brightness or color information, or both, extracted from the third image to generate an enhanced version of the fourth image enhanced in terms of brightness or color quality, or both; and
displaying, storing, transmitting, or streaming the enhanced version of the fourth image.
6 Assignments
0 Petitions
Accused Products
Abstract
A method of combining image data from multiple frames to enhance one or more parameters of video image quality includes acquiring a first image at a first exposure duration, as well as acquiring a second image at a second exposure duration shorter than the first exposure duration and at a time just before, just after or overlapping in time with acquiring the first image, such that the first and second images include approximately a same first scene. In this way, the second image is relatively sharp and under-exposed, while the first image is relatively well-exposed and less sharp than the second image. Brightness and/or color information are extracted from the first image and applied to the second image to generate an enhanced version of the second image.
-
Citations
42 Claims
-
1. A method of combining image data from multiple frames to enhance one or more parameters of digital image quality, comprising using a processor and at least one digital image acquisition system including a lens and an image sensor in:
-
acquiring a first image at a first exposure duration; acquiring a second image at a second exposure duration shorter than the first exposure duration and at a time just before, just after or overlapping in time with acquiring the first image, such that said first and second images comprise approximately a same first scene, and whereby the second image is relatively sharp and under-exposed and the first image is relatively well-exposed and less sharp than the second image; extracting brightness or color information, or both, from the first image; applying to the second image the brightness or color information, or both, extracted from the first image to generate an enhanced version of the second image enhanced in terms of brightness or color quality, or both; displaying, storing, transmitting, or streaming the enhanced version of the second image; acquiring a third image at or near said first exposure duration when a predetermined number of multiple frames has been recorded since the acquiring of the first image; acquiring a fourth image at or near said second exposure duration and at a time just before, just after or overlapping in time with the third image, such that said third and fourth images comprise approximately a same second scene different from or same as the first scene; extracting brightness or color, or both, from the third image; applying to the fourth image the brightness or color information, or both, extracted from the third image to generate an enhanced version of the fourth image enhanced in terms of brightness or color quality, or both; and displaying, storing, transmitting, or streaming the enhanced version of the fourth image. - View Dependent Claims (7, 8, 9, 10, 11, 12, 13)
-
-
2. A method of combining image data from multiple frames to enhance one or more parameters of digital image quality, comprising using a processor and at least one digital image acquisition system including a lens and an image sensor in:
-
acquiring a first image at a first exposure duration; acquiring a second image at a second exposure duration shorter than the first exposure duration and at a time just before, just after or overlapping in time with acquiring the first image, such that said first and second images comprise approximately a same first scene, and whereby the second image is relatively sharp and under-exposed and the first image is relatively well-exposed and less sharp than the second image; extracting brightness or color information, or both, from the first image; applying to the second image the brightness or color information, or both, extracted from the first image to generate an enhanced version of the second image enhanced in terms of brightness or color quality, or both; displaying, storing, transmitting, or streaming the enhanced version of the second image; acquiring a third image at or near said first exposure duration when a predetermined amount of camera movement has been detected since the acquiring of the first image; acquiring a fourth image at or near said second exposure duration and at a time just before, just after or overlapping in time with the third image, such that said third and fourth images comprise approximately a same second scene different from the first scene due at least to said camera movement; extracting brightness or color, or both, from the third image; applying to the fourth image the brightness or color information, or both, extracted from the third image to generate an enhanced version of the fourth image enhanced in terms of brightness or color quality, or both; and displaying, storing, transmitting, or streaming the enhanced version of the fourth image.
-
-
3. A method of combining image data from multiple frames to enhance one or more parameters of digital image quality, comprising using a processor and at least one digital image acquisition system including a lens and an image sensor in:
-
acquiring a first image at a first exposure duration; acquiring a second image at a second exposure duration shorter than the first exposure duration and at a time just before, just after or overlapping in time with acquiring the first image, such that said first and second images comprise approximately a same first scene, and whereby the second image is relatively sharp and under-exposed and the first image is relatively well-exposed and less sharp than the second image; extracting brightness or color information, or both, from the first image; applying to the second image the brightness or color information, or both, extracted from the first image to generate an enhanced version of the second image enhanced in terms of brightness or color quality, or both; displaying, storing, transmitting, or streaming the enhanced version of the second image; acquiring a third image at or near said first exposure duration including combining some of the data from the first image with some or all of the data from the second image; acquiring a fourth image at or near said second exposure duration and at a time just before, just after or overlapping in time with the third image, such that said third and fourth images comprise approximately said same first scene; extracting brightness or color information, or both, from the third image; applying to the fourth image the brightness or color information, or both, extracted from the third image to generate an enhanced version of the fourth image enhanced in terms of brightness or color quality, or both; and displaying, storing, transmitting, or streaming the enhanced version of the fourth image. - View Dependent Claims (4, 5)
-
-
6. A method of combining image data from multiple frames to enhance one or more parameters of digital image quality, comprising using a processor and at least one digital image acquisition system including a lens and an image sensor in:
-
acquiring a first image at a first exposure duration; acquiring a second image at a second exposure duration shorter than the first exposure duration and at a time just before, just after or overlapping in time with acquiring the first image, such that said first and second images comprise approximately a same first scene, and whereby the second image is relatively sharp and under-exposed and the first image is relatively well-exposed and less sharp than the second image; extracting brightness or color information, or both, from the first image; applying to the second image the brightness or color information, or both, extracted from the first image to generate an enhanced version of the second image enhanced in terms of brightness or color quality, or both; displaying, storing, transmitting, or streaming the enhanced version of the second image; and wherein the method further comprising iteratively performing the following; acquiring two sharp, under-exposed frames and a well-exposed frame; extracting brightness or color information, or both, from the well-exposed frame; applying to each pair of sharp, under exposed images respectively captured immediately before and immediately after the acquiring of each well-exposed frame the brightness or color information, or both, extracted from the well-exposed frame to generate enhanced versions of each of the pair of sharp, under-exposed images enhanced in terms of brightness or color quality, or both; and displaying, storing, transmitting, or streaming the enhanced versions of the pair of sharp, under-exposed images.
-
-
14. A method of combining image data from multiple frames to enhance one or more parameters of digital image quality, comprising using a processor and at least one digital image acquisition system including a lens and an image sensor in:
-
acquiring a first image at a first exposure duration; acquiring a second image at a second exposure duration shorter than the first exposure duration and at a time just before, just after or overlapping in time with acquiring the first image, such that said first and second images comprise approximately a same first scene, and whereby the second image is relatively sharp and under-exposed and the first image is relatively well-exposed and less sharp than the second image; extracting brightness or color information, or both, from the first image; applying to the second image the brightness or color information, or both, extracted from the first image to generate an enhanced version of the second image enhanced in terms of brightness or color quality, or both; displaying, storing, transmitting, or streaming the enhanced version of the second image; and wherein the method further comprising utilizing separate image acquisition subsystems (IAS'"'"'s) for acquiring the first and second images, and wherein the separate IAS'"'"'s comprise first and second IAS'"'"'s, wherein the first IAS is configured to acquire images at a faster frame rate than the second IAS, such that the first IAS is specially configured to capture relatively sharp, under-exposed images and the second IAS is specially configured to capture well-exposed and less sharp images, and wherein the first IAS has a fixed frame rate.
-
-
15. A digital image acquisition system, comprising a lens, an image sensor for capturing a stream of multiple digital images, a processor and a memory having processor-readable code embedded therein for programming the processor to perform a method of combining image data from multiple frames to enhance one or more parameters of digital image quality, wherein the method comprises:
-
acquiring a first image at a first exposure duration; acquiring a second image at a second exposure duration shorter than the first exposure duration and at a time just before, just after or overlapping in time with acquiring the first image, such that said first and second images comprise approximately a same first scene, and whereby the second image is relatively sharp and under-exposed and the first image is well-exposed and less sharp than the second image; extracting brightness or color information, or both, from the first image; applying to the second image the brightness or color information, or both, extracted from the first image to generate an enhanced version of the second image enhanced in terms of brightness or color quality, or both; acquiring a third image at or near said first exposure duration when a predetermined number of multiple frames has been recorded since the acquiring of the first image; acquiring a fourth image at or near said second exposure duration and at a time just before, just after or overlapping in time with the third image, such that said third and fourth images comprise approximately a same second scene different from or same as the first scene; extracting brightness or color, or both, from the third image; applying to the fourth image the brightness or color information, or both, extracted from the third image to generate an enhanced version of the fourth image enhanced in terms of brightness or color quality, or both; and displaying, storing, transmitting, or streaming the enhanced version of the fourth image. - View Dependent Claims (21, 22, 23, 24, 25, 26, 27, 28)
-
-
16. A digital image acquisition system, comprising a lens, an image sensor for capturing a stream of multiple digital images, a processor and a memory having processor-readable code embedded therein for programming the processor to perform a method of combining image data from multiple frames to enhance one or more parameters of digital image quality, wherein the method comprises:
-
acquiring a first image at a first exposure duration; acquiring a second image at a second exposure duration shorter than the first exposure duration and at a time just before, just after or overlapping in time with acquiring the first image, such that said first and second images comprise approximately a same first scene, and whereby the second image is relatively sharp and under-exposed and the first image is well-exposed and less sharp than the second image; extracting brightness or color information, or both, from the first image; applying to the second image the brightness or color information, or both, extracted from the first image to generate an enhanced version of the second image enhanced in terms of brightness or color quality, or both; acquiring a third image at or near said first exposure duration when a predetermined amount of camera movement has been detected since the acquiring of the first image; acquiring a fourth image at or near said second exposure duration and at a time just before, just after or overlapping in time with the third image, such that said third and fourth images comprise approximately a same second scene different from the first scene due at least to said camera movement; extracting brightness or color, or both, from the third image; applying to the fourth image the brightness or color information, or both, extracted from the third image to generate an enhanced version of the fourth image enhanced in terms of brightness or color quality, or both; and displaying, storing, transmitting, or streaming the enhanced version of the fourth image.
-
-
17. A digital image acquisition system, comprising a lens, an image sensor for capturing a stream of multiple digital images, a processor and a memory having processor-readable code embedded therein for programming the processor to perform a method of combining image data from multiple frames to enhance one or more parameters of digital image quality, wherein the method comprises:
-
acquiring a first image at a first exposure duration; acquiring a second image at a second exposure duration shorter than the first exposure duration and at a time just before, just after or overlapping in time with acquiring the first image, such that said first and second images comprise approximately a same first scene, and whereby the second image is relatively sharp and under-exposed and the first image is well-exposed and less sharp than the second image; extracting brightness or color information, or both, from the first image; applying to the second image the brightness or color information, or both, extracted from the first image to generate an enhanced version of the second image enhanced in terms of brightness or color quality, or both; acquiring a third image at or near said first exposure duration including combining some of the data from the first image with some or all of the data from the second image; acquiring a fourth image at or near said second exposure duration and at a time just before, just after or overlapping in time with the third image, such that said third and fourth images comprise approximately said same first scene; extracting brightness or color information, or both, from the third image; applying to the fourth image the brightness or color information, or both, extracted from the third image to generate an enhanced version of the fourth image enhanced in terms of brightness or color quality, or both; and displaying, storing, transmitting, or streaming the enhanced version of the fourth image. - View Dependent Claims (18, 19)
-
-
20. A digital image acquisition system, comprising a lens, an image sensor for capturing a stream of multiple digital images, a processor and a memory having processor-readable code embedded therein for programming the processor to perform a method of combining image data from multiple frames to enhance one or more parameters of digital image quality, wherein the method comprises:
-
acquiring a first image at a first exposure duration; acquiring a second image at a second exposure duration shorter than the first exposure duration and at a time just before, just after or overlapping in time with acquiring the first image, such that said first and second images comprise approximately a same first scene, and whereby the second image is relatively sharp and under-exposed and the first image is well-exposed and less sharp than the second image; extracting brightness or color information, or both, from the first image; applying to the second image the brightness or color information, or both, extracted from the first image to generate an enhanced version of the second image enhanced in terms of brightness or color quality, or both, and wherein the method further comprises iteratively performing the following; acquiring two sharp, under-exposed frames and a well-exposed frame; extracting brightness or color information, or both, from the well-exposed frame; applying to each pair of sharp, under exposed images respectively captured immediately before and immediately after the acquiring of each well-exposed frame the brightness or color information, or both, extracted from the well-exposed frame to generate enhanced versions of each of the pair of sharp, under-exposed images enhanced in terms of brightness or color quality, or both; and displaying, storing, transmitting, or streaming the enhanced versions of the pair of sharp, under-exposed images.
-
-
29. One or more non-transitory processor-readable media having processor-readable code embedded therein for programming one or more processors to perform a method of combining image data from multiple frames to enhance one or more parameters of digital image quality, comprising:
-
acquiring a first image at a first exposure duration; acquiring a second image at a second exposure duration shorter than the first exposure duration and at a time just before, just after or overlapping in time with acquiring the first image, such that said first and second images comprise approximately a same first scene, and whereby the second image is relatively sharp and under-exposed and the first image is well-exposed and less sharp than the second image; extracting brightness or color information, or both, from the first image; applying to the second image the brightness or color information, or both, extracted from the first image to generate an enhanced version of the second image enhanced in terms of brightness or color quality, or both; acquiring a third image at or near said first exposure duration when a predetermined number of multiple frames has been recorded since the acquiring of the first image; acquiring a fourth image at or near said second exposure duration and at a time just before, just after or overlapping in time with the third image, such that said third and fourth images comprise approximately a same second scene different from or same as the first scene; extracting brightness or color, or both, from the third image; applying to the fourth image the brightness or color information, or both, extracted from the third image to generate an enhanced version of the fourth image enhanced in terms of brightness or color quality, or both; and displaying, storing, transmitting, or streaming the enhanced version of the fourth image. - View Dependent Claims (35, 36, 37, 38, 39, 40, 41, 42)
-
-
30. One or more non-transitory processor-readable media having processor-readable code embedded therein for programming one or more processors to perform a method of combining image data from multiple frames to enhance one or more parameters of digital image quality, comprising:
-
acquiring a first image at a first exposure duration; acquiring a second image at a second exposure duration shorter than the first exposure duration and at a time just before, just after or overlapping in time with acquiring the first image, such that said first and second images comprise approximately a same first scene, and whereby the second image is relatively sharp and under-exposed and the first image is well-exposed and less sharp than the second image; extracting brightness or color information, or both, from the first image; applying to the second image the brightness or color information, or both, extracted from the first image to generate an enhanced version of the second image enhanced in terms of brightness or color quality, or both; acquiring a third image at or near said first exposure duration when a predetermined amount of camera movement has been detected since the acquiring of the first image; acquiring a fourth image at or near said second exposure duration and at a time just before, just after or overlapping in time with the third image, such that said third and fourth images comprise approximately a same second scene different from the first scene due at least to said camera movement; extracting brightness or color, or both, from the third image; applying to the fourth image the brightness or color information, or both, extracted from the third image to generate an enhanced version of the fourth image enhanced in terms of brightness or color quality, or both; and displaying, storing, transmitting, or streaming the enhanced version of the fourth image.
-
-
31. One or more non-transitory processor-readable media having processor-readable code embedded therein for programming one or more processors to perform a method of combining image data from multiple frames to enhance one or more parameters of digital image quality, comprising:
-
acquiring a first image at a first exposure duration; acquiring a second image at a second exposure duration shorter than the first exposure duration and at a time just before, just after or overlapping in time with acquiring the first image, such that said first and second images comprise approximately a same first scene, and whereby the second image is relatively sharp and under-exposed and the first image is well-exposed and less sharp than the second image; extracting brightness or color information, or both, from the first image; applying to the second image the brightness or color information, or both, extracted from the first image to generate an enhanced version of the second image enhanced in terms of brightness or color quality, or both; acquiring a third image at or near said first exposure duration including combining some of the data from the first image with some or all of the data from the second image; acquiring a fourth image at or near said second exposure duration and at a time just before, just after or overlapping in time with the third image, such that said third and fourth images comprise approximately said same first scene; extracting brightness or color information, or both, from the third image; applying to the fourth image the brightness or color information, or both, extracted from the third image to generate an enhanced version of the fourth image enhanced in terms of brightness or color quality, or both; and displaying, storing, transmitting, or streaming the enhanced version of the fourth image. - View Dependent Claims (32, 33)
-
-
34. One or more non-transitory processor-readable media having processor-readable code embedded therein for programming one or more processors to perform a method of combining image data from multiple frames to enhance one or more parameters of digital image quality, comprising:
-
acquiring a first image at a first exposure duration; acquiring a second image at a second exposure duration shorter than the first exposure duration and at a time just before, just after or overlapping in time with acquiring the first image, such that said first and second images comprise approximately a same first scene, and whereby the second image is relatively sharp and under-exposed and the first image is well-exposed and less sharp than the second image; extracting brightness or color information, or both, from the first image; applying to the second image the brightness or color information, or both, extracted from the first image to generate an enhanced version of the second image enhanced in terms of brightness or color quality, or both; and wherein the method further comprises iteratively performing the following; acquiring two sharp, under-exposed frames and a well-exposed frame; extracting brightness or color information, or both, from the well-exposed frame; applying to each pair of sharp, under exposed images respectively captured immediately before and immediately after the acquiring of each well-exposed frame the brightness or color information, or both, extracted from the well-exposed frame to generate enhanced versions of each of the pair of sharp, under-exposed images enhanced in terms of brightness or color quality, or both; and displaying, storing, transmitting, or streaming the enhanced versions of the pair of sharp, under-exposed images.
-
Specification