Scene enhancements in off-center peripheral regions for nonlinear lens geometries

• US 8,860,816 B2
• Filed: 03/31/2011
• Issued: 10/14/2014
• Est. Priority Date: 03/31/2011
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: FotoNation Limited (Adeia Inc.)
• Inventors: Corcoran, Peter, Bigioi, Petronel, Stec, Piotr
• Primary Examiner(s): DAGNEW, MEKONNEN D
• Application Number:
  US13/077,891
• Publication Number:
  US 20120249841A1
• Time in Patent Office: 1,293 Days
• Field of Search: 348/335, 348360-361, 348 36- 39, 348/143, 348169-172, 382/275, 382/276, 382/254
• US Class Current: 348/169
• CPC Class Codes: G06T 3/047 Fisheye or wide-angle transformations
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/81 for suppressing or minimising disturbance in the image signal generation

Abstract

A technique of enhancing a scene containing one or more off-center peripheral regions within an initial distorted image captured with a large field of view includes determining and extracting an off-center region of interest (hereinafter “ROI”) within the image. Geometric correction is applied to reconstruct the off-center ROI into a rectangular or otherwise undistorted or less distorted frame of reference as a reconstructed ROI. A quality of reconstructed pixels is determined within the constructed ROI. Image analysis is selectively applied to the reconstructed ROI based on the quality of the reconstructed pixels.

Claims

1. Within a digital image acquisition apparatus comprising a non-linear, wide-angled lens and an imaging sensor, a method of enhancing a scene containing one or more off-center peripheral regions, the method comprising:

using a processor;

acquiring an initial distorted image with a large field of view, including using a non-linear, wide-angled lens and an imaging sensor;

extracting one or more distorted regions within said initial distorted image in accordance with known distortion characteristics of a configuration of the non-linear, wide-angled lens and the imaging sensor;

applying a geometric correction to reconstruct said one or more distorted regions into an approximately rectangular frame of reference as one or more respectively reconstructed regions within said large field of view and corresponding to one or more off-center reconstructed regions of a scene captured as said initial distorted image, said one or more off-center reconstructed regions having been imaged through a different region of the non-linear, wide angled lens than a center region, and said one or more off-center reconstructed regions having been captured as one or more groups of pixels at one or more different regions of said same imaging sensor that has also captured a non-distorted or less-distorted center region within said large field of view;

determining a quality of reconstructed pixels, relative to pixels of the initial distorted image, within said one or more reconstructed regions; and

selectively applying image analysis to said one or more reconstructed regions based on the quality of the reconstructed pixels.

2. A method as in claim 1, further comprising compensating for global motion of the digital-image acquisition apparatus.

3. A method as in claim 1, further comprising repeating the method for a second distorted image, and generating a second set of one or more reconstructed regions of approximately one or more respectively same regions of said scene as the first set of one or more reconstructed regions.

4. A method as in claim 3, further comprising, responsive to analysis of the first and second sets of one or more reconstructed regions, processing both the first and second sets of one or more reconstructed regions and generating, based on the processing, an enhanced output image of approximately one or more respectively same regions of the scene.

5. A method as in claim 1, further comprising, based on said selectively applying image analysis, adjusting an image acquisition parameter and repeating the method for a second distorted image, and generating, based on the second distorted image, a second set of one or more reconstructed regions of approximately one or more respectively same regions of said scene as the first set of one or more reconstructed regions.

6. A method as in claim 5, further comprising processing the first and second sets of one or more reconstructed regions of approximately one or more respectively same regions of scene, and generating, based on the processing, an enhanced output image of approximately one or more respectively same regions of the scene.

7. A method as in claim 1, responsive to said analysis and said pixel quality selectively applying image enhancement and generating an enhanced output image.

8. A digital image acquisition device, comprising:

a non-linear, wide-angled lens and an imaging sensor configured to capture digital images of scenes containing one or more off-center peripheral regions, including an initial distorted image with a large field of view;

a processor;

a memory having code embedded therein for programming the processor to perform a method of enhancing a first scene containing one or more off-center peripheral regions, wherein the method comprises;

extracting one or more distorted regions within said initial distorted image in accordance with known distortion characteristics of a configuration of the non-linear, wide-angled lens and the imaging sensor;

applying a geometric correction to reconstruct said one or more distorted regions into an approximately rectangular frame of reference as a one or more respectively reconstructed regions within said large field of view and corresponding to one or more off-center reconstructed regions of a scene captured as said initial distorted image, said one or more off-center reconstructed regions having been imaged through a different region of the non-linear, wide angled lens than a center region, and said one or more off-center reconstructed regions having been captured as one or more groups of pixels at one or more different regions of said same imaging sensor that has also captured a non-distorted or less-distorted center region within said large field of view;

determining a quality of reconstructed pixels, relative to pixels of the initial distorted image, within said one or more reconstructed regions; and

selectively applying image analysis to said one or more reconstructed regions based on the quality of the reconstructed pixels.

9. A device as in claim 8, wherein the method comprises compensating for global motion of an image acquisition system.

10. A device as in claim 8, wherein the method comprises repeating the method for a second distorted image, and generating a second set of one or more reconstructed regions of approximately one or more respectively same regions of said scene as the first set of one or more reconstructed regions.

11. A device as in claim 10, wherein the method comprises, responsive to analysis of the first and second sets of one or more reconstructed regions processing both the first and second sets of one or more reconstructed regions and generating, based on the processing, an enhanced output image of approximately one or more respectively same regions of the scene.

12. A device as in claim 8, wherein the method comprises, based on said selectively applying image analysis, adjusting an image acquisition parameter and repeating the method for a second distorted image, and generating, based on the second distorted image, a second set of one or more reconstructed regions of approximately one or more respectively same regions of said scene as the first set of one or more reconstructed regions.

13. A device as in claim 12, wherein the method comprises processing the first and second sets of one or more reconstructed regions of approximately one or more respectively same regions of the scene, and generating, based on the processing, an enhanced output image of approximately one or more respectively same regions of the scene.

14. A device as in claim 8, wherein the method comprises, responsive to said analysis and said pixel quality, selectively applying image enhancement and generating an enhanced output image.

15. One or more non-transitory, processor-readable storage media having code embedded therein for programming a processor to perform a method of enhancing a scene captured with a non-linear, wide-angled lens and containing one or more off-center peripheral regions, wherein the method comprises:

extracting one or more distorted regions within an initial distorted image in accordance with known distortion characteristics of a configuration of the non-linear, wide-angled lens and an imaging sensor;

applying a geometric correction to reconstruct said one or more distorted regions into an approximately rectangular frame of reference as a one or more respectively reconstructed regions within said large field of view and corresponding to one or more off-center reconstructed regions of a scene captured as said initial distorted image, said one or more off-center reconstructed regions having been imaged through a different region of the non-linear, wide angled lens than a center region, and said one or more off-center reconstructed regions having been captured as one or more groups of pixels at one or more different regions of said same imaging sensor that has also captured a non-distorted or less-distorted center region within said large field of view;

determining a quality of reconstructed pixels, relative to pixels of the initial distorted image, within said one or more reconstructed regions; and

selectively applying image analysis to said one or more reconstructed regions based on the quality of the reconstructed pixels.

16. One or more non-transitory, processor-readable storage media as in claim 15, wherein the method comprises compensating for global motion of an image acquisition system.

17. One or more non-transitory, processor-readable storage media as in claim 15, wherein the method comprises repeating the method for a second distorted image, and generating a second set of one or more reconstructed regions of approximately one or more respectively same regions of said scene as the first set of one or more reconstructed regions.

18. One or more non-transitory, processor-readable storage media as in claim 17, wherein the method comprises, responsive to analysis of the first and second sets of one or more reconstructed regions processing both the first and second sets of one or more reconstructed regions and generating, based on the processing, an enhanced output image of approximately one or more respectively same regions of the scene.

19. One or more non-transitory, processor-readable storage media as in claim 15, wherein the method comprises, based on said selectively applying image analysis, adjusting an image acquisition parameter and repeating the method for a second distorted image, and generating, based on the second distorted image, a second set of one or more reconstructed regions of approximately one or more respectively same regions of said scene as the first set of one or more reconstructed regions.

20. One or more non-transitory, processor-readable storage media as in claim 19, wherein the method comprises processing the first and second sets of one or more reconstructed regions of approximately one or more respectively same regions of the scene, and generating, based on the processing, an enhanced output image of approximately one or more respectively same regions of the scene.

21. One or more non-transitory, processor-readable storage media as in claim 15, wherein the method further comprises, responsive to said analysis and said pixel quality, selectively applying image enhancement and generating an enhanced output image.

22. Within a digital image acquisition system comprising a non-linear, wide-angled lens and an imaging sensor, a method of enhancing a scene containing one or more off-center peripheral regions, the method comprising:

using a processor;

acquiring an initial distorted image with a large field of view, including using a non-linear, wide-angled lens and an imaging sensor;

determining and extracting an off-center region of interest (hereinafter “

ROI”

) within said initial distorted image;

applying a geometric correction to reconstruct said off-center ROI into an approximately rectangular frame of reference as a reconstructed ROI;

determining a quality of reconstructed pixels, relative to pixels of the initial distorted image, within said reconstructed ROI;

selectively applying image analysis to said reconstructed ROI based on the quality of the reconstructed pixels; and

repeating the method for a second distorted image, and generating a second reconstructed ROI of approximately a same portion of an image scene as the first reconstructed ROI.

23. A method as in claim 22, further comprising, responsive to analysis of the first and second reconstructed ROIs, processing both the first and second reconstructed ROIs and generating, based on the processing, an enhanced output image of substantially the same portion of the image scene.

24. Within a digital image acquisition system comprising a non-linear, wide-angled lens and an imaging sensor, a method of enhancing a scene containing one or more off-center peripheral regions, the method comprising:

using a processor;

acquiring an initial distorted image with a large field of view, including using a non-linear, wide-angled lens and an imaging sensor;

determining and extracting an off-center region of interest (hereinafter “

ROI”

) within said initial distorted image;

applying a geometric correction to reconstruct said off-center ROI into an approximately rectangular frame of reference as a reconstructed ROI;

determining a quality of reconstructed pixels, relative to pixels of the initial distorted image, within said reconstructed ROI;

selectively applying image analysis to said reconstructed ROI based on the quality of the reconstructed pixels; and

based on said selectively applying image analysis, adjusting an image acquisition parameter and repeating the method for a second distorted image, and generating, based on the second distorted image, a second reconstructed ROI of approximately a same portion of an image scene as the first reconstructed ROI.

25. A method as in claim 24, further comprising processing the first and second reconstructed ROIs of approximately the same portion of the image scene, and generating, based on the processing, an enhanced output image of substantially the same portion of the image scene.

26. A digital image acquisition device, comprising:

a non-linear, wide-angled lens and an imaging sensor configured to capture digital images of scenes containing one or more off-center peripheral regions, including an initial distorted image with a large field of view;

a processor;

a memory having code embedded therein for programming the processor to perform a method of enhancing a scene containing one or more off-center peripheral regions, wherein the method comprises;

determining and extracting an off-center region of interest (hereinafter “

ROI”

) within said initial distorted image;

applying a geometric correction to reconstruct said off-center ROI into an approximately rectangular frame of reference as a reconstructed ROI;

determining a quality of reconstructed pixels, relative to pixels of the initial distorted image, within said reconstructed ROI;

selectively applying image analysis to said reconstructed ROI based on the quality of the reconstructed pixels, and

repeating the method for a second distorted image, and generating a second reconstructed ROI of approximately a same portion of an image scene as the first reconstructed ROI.

27. A device as in claim 26, wherein the method comprises, responsive to analysis of the first and second reconstructed ROI, processing both the first and second reconstructed ROIs and generating, based on the processing, an enhanced output image of substantially the same portion of the image scene.

28. A digital image acquisition device, comprising:

a non-linear, wide-angled lens and an imaging sensor configured to capture digital images of scenes containing one or more off-center peripheral regions, including an initial distorted image with a large field of view;

a processor;

a memory having code embedded therein for programming the processor to perform a method of enhancing a scene containing the one or more off-center peripheral regions, wherein the method comprises;

determining and extracting an off-center region of interest (hereinafter “

ROI”

) within said initial distorted image;

applying a geometric correction to reconstruct said off-center ROI into an approximately rectangular frame of reference as a reconstructed ROI;

determining a quality of reconstructed pixels, relative to pixels of the initial distorted image, within said reconstructed ROI;

selectively applying image analysis to said reconstructed ROI based on the quality of the reconstructed pixels, and

based on said selectively applying image analysis, adjusting an image acquisition parameter and repeating the method for a second distorted image, and generating, based on the second distorted image, a second reconstructed ROI of approximately a same portion of an image scene as the first reconstructed ROI.

29. A device as in claim 28, wherein the method comprises processing the first and second reconstructed ROIs of approximately the same portion of the image scene, and generating, based on the processing, an enhanced output image of substantially the same portion of the image scene.

30. One or more non-transitory, processor-readable storage media having code embedded therein for programming a processor to perform a method of enhancing a scene captured with a non-linear, wide-angled lens and containing one or more off-center peripheral regions, wherein the method comprises:

determining and extracting an off-center region of interest (hereinafter “

ROI”

) within an initial distorted image captured with a large field of view;

applying a geometric correction to reconstruct said off-center ROI into an approximately rectangular frame of reference as a reconstructed ROI;

determining a quality of reconstructed pixels, relative to pixels of the initial distorted image, within said reconstructed ROI;

selectively applying image analysis to said reconstructed ROI based on the quality of the reconstructed pixels; and

repeating the method for a second distorted image, and generating a second reconstructed ROI of approximately a same portion of an image scene as the first reconstructed ROI.

31. One or more non-transitory, processor-readable storage media as in claim 30, wherein the method comprises, responsive to analysis of the first and second reconstructed ROIs, processing both the first and second reconstructed ROIs and generating, based on the processing, an enhanced output image of substantially the same portion of the image scene.

32. One or more non-transitory, processor-readable storage media having code embedded therein for programming a processor to perform a method of enhancing a scene captured with a non-linear, wide-angled lens and containing one or more off-center peripheral regions, wherein the method comprises:

determining and extracting an off-center region of interest (hereinafter “

ROI”

) within an initial distorted image captured with a large field of view;

applying a geometric correction to reconstruct said off-center ROI into an approximately rectangular frame of reference as a reconstructed ROI;

determining a quality of reconstructed pixels, relative to pixels of the initial distorted image, within said reconstructed ROI;

selectively applying image analysis to said reconstructed ROI based on the quality of the reconstructed pixels, and

based on said selectively applying image analysis, adjusting an image acquisition parameter and repeating the method for a second distorted image, and generating, based on the second distorted image, a second reconstructed ROI of approximately a same portion of an image scene as the first reconstructed ROI.

33. One or more non-transitory, processor-readable storage media as in claim 32, wherein the method comprises processing the first and second reconstructed ROIs of approximately the same portion of the image scene, and generating, based on the processing, an enhanced output image of substantially the same portion of the image scene.