Panorama Imaging Using Lo-Res Images

• US 20110141224A1
• Filed: 12/11/2009
• Published: 06/16/2011
• Est. Priority Date: 12/11/2009
• Status: Abandoned Application

• Current Assignee: DigitalOptics Corporation Europe Limited (Adeia, Inc.)
• Original Assignee: FotoNation Ireland Limited (Adeia, Inc.)
• Inventors: Drimbarean, Alexandru, Corcoran, Peter, Barcovschi, Igor, Murray, Larry, Stec, Piotr
• Application Number:
  US12/636,608
• Publication Number:
  US 20110141224A1
• Time in Patent Office: Days
• US Class Current: 348/36
• CPC Class Codes: G06T 1/00 General purpose image data processing
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio

Abstract

A technique of generating a panoramic image involves acquiring a set of at least two main image frames, as well as corresponding generating and/or acquiring corresponding relatively low-resolution image frames, of overlapping portions of a panoramic scene. A map or other information is stored relating to the generation of a low-res panorama. A main panorama image is formed by joining main image frames based on the map or other information gained in the low-res process.

Claims

1. A method of acquiring a panorama image using a portable imaging device, said method comprising:

fixing an exposure level for acquiring the panorama image with the portable imaging device;

panning the imaging device across a panoramic scene;

acquiring a set of at least two relatively low-resolution image frames of overlapping portions of said panoramic scene and processing said image set, said acquiring comprising using an optic and imaging sensor of the portable imaging device, said processing including;

sorting and retaining said set of relatively low resolution image frames,

determining a relative displacement between each image of said set of relatively low resolution image frames,

determining an approximately optimal stitch line between each pair of images of said relatively low resolution image frames;

acquiring and storing a set of main image frames corresponding to the set of relatively low resolution image frames;

registering or aligning, or both, said main image frames based on relative displacements of corresponding images of the set of relatively low resolution images;

mapping the approximately optimal stitch line onto corresponding pairs of registered and/or aligned main image frames;

joining said set of main image frames to form a main panorama image; and

storing, communicating or displaying said main panorama image, or a processed version, or combinations thereof.

2. The method of claim 1 wherein the determining of an approximately optimal stitch line includes further determining an alpha blending map in the vicinity of said approximately optimal stitch line, and wherein the mapping of said approximately optimal stitch line onto said high resolution images includes further mapping said blending map, and wherein the joining of the high resolution images includes blending said images based on said mapping of the alpha blending map.

3. The method of claim 1, wherein the acquiring comprises acquiring a set of at least three low resolution image frames of component portions of said scene.

4. The method of claim 1, further comprising notifying the user or discarding an image, or both, when a horizontal overlap between consecutive images falls outside a predetermined range.

5. The method of claim 1, wherein the joining comprises blending the set of main image frames, including cropping one or more of said main image frames.

6. The method of claim 1, wherein the joining comprises blending the set of main image frames, including mapping the alpha-blending map for a series of the relatively low resolution image frames to a main series of image frames.

7. The method of claim 1, further comprising performing sub-pixel image registration on the relatively low resolution images to prevent pixel offsets in the mapping of the alpha-blending map for the relatively low resolution series to the main series.

8. The method of claim 7, further comprising interleaving the joining of pairs of digital images of the main series with the sub-pixel registration of relatively low resolution images.

9. The method of claim 1, further comprising interleaving the joining of pairs of image frames of the main series with the acquiring or generating, or joining of corresponding pairs of relatively low resolution images, or combinations thereof

10. A portable camera-enabled device capable of in-camera generation of a panorama image, comprising:

a lens;

an image sensor;

a processor; and

a processor readable medium having code embedded therein for programming the processor to perform a panorama image generation method that comprises;

fixing an exposure level for acquiring the panorama image with the portable imaging device;

panning the imaging device across a panoramic scene;

acquiring a set of at least two relatively low-resolution image frames of overlapping portions of said panoramic scene and processing said image set, said acquiring comprising using an optic and imaging sensor of the portable imaging device, said processing including;

sorting and retaining said set of relatively low resolution image frames,

determining a relative displacement between each image of said set of relatively low resolution image frames,

determining an approximately optimal stitch line between each pair of images of said relatively low resolution image frames;

acquiring and storing a set of main image frames corresponding to the set of relatively low resolution image frames;

registering or aligning, or both, said main image frames based on relative displacements of corresponding images of the set of relatively low resolution images;

mapping the approximately optimal stitch line onto corresponding pairs of registered and/or aligned main image frames;

joining said set of main image frames to form a main panorama image; and

storing, communicating or displaying said main panorama image, or a processed version, or combinations thereof.

11. The device of claim 10, wherein the determining of an approximately optimal stitch line includes further determining an alpha blending map in the vicinity of said approximately optimal stitch line, and wherein the mapping of said approximately optimal stitch line onto said high resolution images includes further mapping said blending map, and wherein the joining of the high resolution images includes blending said images based on said mapping of the alpha blending map.

12. The device of claim 10, wherein the acquiring comprises acquiring a set of at least three low resolution image frames of component portions of said scene.

13. The device of claim 10, wherein the method further comprises notifying the user or discarding an image, or both, when a horizontal overlap between consecutive images falls outside a predetermined range.

14. The device of claim 10, wherein the joining comprises blending the set of main image frames, including cropping one or more of said main image frames.

15. The device of claim 10, wherein the joining comprises blending the set of main image frames, including mapping the alpha-blending map for a series of the relatively low resolution image frames to a main series of image frames.

16. The device of claim 10, wherein the method further comprises performing sub-pixel image registration on the relatively low resolution images to prevent pixel offsets in the mapping of the alpha-blending map for the relatively low resolution series to the main series.

17. The device of claim 16, wherein the method further comprises interleaving the joining of pairs of digital images of the main series with the sub-pixel registration of relatively low resolution images.

18. The device of claim 10, wherein the method further comprises interleaving the joining of pairs of image frames of the main series with the acquiring or generating, or joining of corresponding pairs of relatively low resolution images, or combinations thereof.

19. One or more processor-readable storage media having code embedded therein for programming a processor to perform a method of generating a panorama image using a portable imaging device, said method comprising:

fixing an exposure level for acquiring the panorama image with the portable imaging device;

panning the imaging device across a panoramic scene;

acquiring a set of at least two relatively low-resolution image frames of overlapping portions of said panoramic scene and processing said image set, said acquiring comprising using an optic and imaging sensor of the portable imaging device, said processing including;

sorting and retaining said set of relatively low resolution image frames,

determining a relative displacement between each image of said set of relatively low resolution image frames,

determining an approximately optimal stitch line between each pair of images of said relatively low resolution image frames;

acquiring and storing a set of main image frames corresponding to the set of relatively low resolution image frames;

registering or aligning, or both, said main image frames based on relative displacements of corresponding images of the set of relatively low resolution images;

mapping the approximately optimal stitch line onto corresponding pairs of registered and/or aligned main image frames;

joining said set of main image frames to form a main panorama image; and

storing, transmitting or displaying said main panorama image, or combinations thereof.

20. The one or more processor-readable storage media of claim 19, wherein the determining of an approximately optimal stitch line includes further determining an alpha blending map in the vicinity of said approximately optimal stitch line, and wherein the mapping of said approximately optimal stitch line onto said high resolution images includes further mapping said blending map, and wherein the joining of the high resolution images includes blending said images based on said mapping of the alpha blending map.

21. The one or more processor-readable storage media of claim 19, wherein the acquiring comprises acquiring a set of at least three low resolution image frames of component portions of said scene.

22. The one or more processor-readable storage media of claim 19, wherein the method further comprises notifying the user or discarding an image, or both, when a horizontal overlap between consecutive images falls outside a predetermined range.

23. The one or more processor-readable storage media of claim 19, wherein the joining comprises blending the set of main image frames, including cropping one or more of said main image frames.

24. The one or more processor-readable storage media of claim 19, wherein the joining comprises blending the set of main image frames, including mapping the alpha-blending map for a series of the relatively low resolution image frames to a main series of image frames.

25. The one or more processor-readable storage media of claim 19, wherein the method further comprises performing sub-pixel image registration on the relatively low resolution images to prevent pixel offsets in the mapping of the alpha-blending map for the relatively low resolution series to the main series.

26. The one or more processor-readable storage media of claim 25, wherein the method further comprises interleaving the joining of pairs of digital images of the main series with the sub-pixel registration of relatively low resolution images.

27. The one or more processor-readable storage media of claim 19, wherein the method further comprises interleaving the joining of pairs of image frames of the main series with the acquiring or generating, or joining of corresponding pairs of relatively low resolution images, or combinations thereof.