Panorama Imaging Based on Low-Res Images

• US 20110141225A1
• Filed: 12/11/2009
• Published: 06/16/2011
• Est. Priority Date: 12/11/2009
• Status: Abandoned Application

• Current Assignee: DigitalOptics Corporation Europe Limited (Adeia, Inc.)
• Original Assignee: FotoNation Ireland Limited (Adeia, Inc.)
• Inventors: Drimbarean, Alexandru, Corcoran, Peter, Barcovschi, Igor, Murray, Larry, Stec, Piotr
• Application Number:
  US12/636,639
• Publication Number:
  US 20110141225A1
• Time in Patent Office: Days
• US Class Current: 348/36
• CPC Class Codes: G06T 1/00 General purpose image data processing
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio

Abstract

A technique of generating a panoramic image involves acquiring a set of at least two main image frames, as well as corresponding generating and/or acquiring corresponding relatively low-resolution image frames, of overlapping portions of a panoramic scene. A map or other information is stored relating to the generation of a low-res panorama. A main panorama image is formed by joining main image frames based on the map or other information gained in the low-res process.

Claims

1. A method of generating a panorama image, comprising:

panning across a scene a processor-based device configured for acquiring digital images;

during the panning, acquiring multiple main series images with the device, each of the multiple images containing a different angular range of said scene;

also during the panning, acquiring relatively low resolution images corresponding to the multiple main series images, or subsampling the main series images to produce the relatively low resolution images, or a combination thereof,

joining the relatively low resolution images to form a low-res panorama;

displaying the low-res panorama on the device;

compositing the multiple main series images in real-time on the device based on the joining of the relatively low resolution images to form a main series panorama image;

storing the panorama image.

2. The method of claim 1, wherein the joining comprises stitching.

3. The method of claim 1, wherein the joining comprises:

matching exposure, color balance, or luminance, or combinations thereof;

aligning or registering edge regions of images, or both; and

blending matched, aligned images.

4. The method of claim 3, wherein the blending comprises blurring a seam line generated between adjacent component images of the panorama image.

5. The method of claim 1, further comprising fixing an exposure level for acquiring the panorama image with the portable imaging device.

6. The method of claim 5, wherein the joining comprises:

aligning or registering edge regions of images, or both; and

blending matched, aligned images.

7. The method of claim 6, wherein the blending comprises blurring a seam line generated between adjacent component images of the panorama image.

8. The method of claim 1, wherein the compositing includes estimating a global motion of the panning and determining whether said global motion is sufficient.

9. The method of claim 1, wherein the compositing includes estimating a relative scene displacement and determining whether said relative scene displacement is sufficient.

10. The method of claim 9, further comprising notifying a user, discarding one or more of the images or interrupting the method, or combinations thereof, when the relative scene displacement is determined to be insufficient.

11. The method of claim 10, wherein the relative scene displacement is determined to be sufficient when frame to frame overlap comprises a range between 10% and 40%.

12. The method of claim 11, wherein the range is between 20% and 30%.

13. The method of claim 9, wherein the estimating relative displacement comprises multi-dimensional motion estimating for multiple image pairs.

14. The method of claim 1, wherein said method is performed without user intervention.

15. The method of claim 1, wherein said method is performed without device motion measurement.

16. The method of claim 1, wherein the storing comprises generating a compressed version of the panorama image prior to storing.

17. The method of claim 1, wherein the joining of the relatively low resolution images comprises generating an alpha blending map, and wherein the joining of the main series images comprises using said alpha blending map.

19. The device of claim 11, wherein the joining comprises stitching.

18. A portable camera-enabled device capable of in-camera generation of a panorama image, comprising:

a lens;

an image sensor;

a processor; and

a processor readable medium having code embedded therein for programming the processor to perform a panorama image generation method that comprises;

panning across a scene a processor-based device configured for acquiring digital images;

during the panning, acquiring multiple main series images with the device, each of the multiple images containing a different angular range of said scene;

also during the panning, acquiring relatively low resolution images corresponding to the multiple main series images, or subsampling the main series images to produce the relatively low resolution images, or a combination thereof,

joining the relatively low resolution images to form a low-res panorama;

displaying the low-res panorama on the device;

compositing the multiple main series images in real-time on the device based on the joining of the relatively low resolution images to form a main series panorama image;

storing the panorama image.

20. The device of claim 18, wherein the joining comprises:

matching exposure, color balance, or luminance, or combinations thereof;

aligning or registering edge regions of images, or both; and

blending matched, aligned images.

21. The device of claim 20, wherein the blending comprises blurring a seam line generated between adjacent component images of the panorama image.

22. The device of claim 18, wherein the method further comprises fixing an exposure level for acquiring the panorama image with the portable imaging device.

23. The device of claim 22, wherein the joining comprises:

aligning or registering edge regions of images, or both; and

blending matched, aligned images.

24. The device of claim 23, wherein the blending comprises blurring a seam line generated between adjacent component images of the panorama image.

25. The device of claim 18, wherein the compositing includes estimating a global motion of the panning and determining whether said global motion is sufficient.

26. The device of claim 18, wherein the compositing includes estimating a relative scene displacement and determining whether said relative scene displacement is sufficient.

27. The device of claim 26, wherein the method further comprises notifying a user, discarding one or more of the images or interrupting the method, or combinations thereof, when the relative scene displacement is determined to be insufficient.

28. The device of claim 27, wherein the relative scene displacement is determined to be sufficient when frame to frame overlap comprises a range between 10% and 40%.

29. The device of claim 28, wherein the range is between 20% and 30%.

30. The device of claim 26, wherein the estimating relative displacement comprises multi-dimensional motion estimating for multiple image pairs.

31. The device of claim 18, wherein said method is performed without user intervention.

32. The device of claim 18, wherein said method is performed without device motion measurement.

33. The device of claim 18, wherein the storing comprises generating a compressed version of the panorama image prior to storing.

34. The device of claim 18, wherein the joining of the relatively low resolution images comprises generating an alpha blending map, and wherein the joining of the main series images comprises using said alpha blending map.

35. One or more processor-readable storage media having code embedded therein for programming a processor to perform a method of generating a panorama image, the method comprising:

panning across a scene a processor-based device configured for acquiring digital images;

during the panning, acquiring multiple main series images with the device, each of the multiple images containing a different angular range of said scene;

also during the panning, acquiring relatively low resolution images corresponding to the multiple main series images, or subsampling the main series images to produce the relatively low resolution images, or a combination thereof,

joining the relatively low resolution images to form a low-res panorama;

displaying the low-res panorama on the device;

compositing the multiple main series images in real-time on the device based on the joining of the relatively low resolution images to form a main series panorama image;

storing the panorama image.

36. The one or more processor-readable media of claim 35, wherein the joining comprises stitching.

37. The one or more processor-readable media of claim 35, wherein the joining comprises:

matching exposure, color balance, or luminance, or combinations thereof;

aligning or registering edge regions of images, or both; and

blending matched, aligned images.

38. The one or more processor-readable media of claim 37, wherein the blending comprises blurring a seam line generated between adjacent component images of the panorama image.

39. The one or more processor-readable media of claim 35, wherein the method further comprises fixing an exposure level for acquiring the panorama image with the portable imaging device.

40. The one or more processor-readable media of claim 39, wherein the joining comprises:

aligning or registering edge regions of images, or both; and

blending matched, aligned images.

41. The one or more processor-readable media of claim 40, wherein the blending comprises blurring a seam line generated between adjacent component images of the panorama image.

42. The one or more processor-readable media of claim 35, wherein the compositing includes estimating a global motion of the panning and determining whether said global motion is sufficient.

43. The one or more processor-readable media of claim 35, wherein the compositing includes estimating a relative scene displacement and determining whether said relative scene displacement is sufficient.

44. The one or more processor-readable media of claim 43, wherein the method further comprises notifying a user, discarding one or more of the images or interrupting the method, or combinations thereof, when the relative scene displacement is determined to be insufficient.

45. The one or more processor-readable media of claim 44, wherein the relative scene displacement is determined to be sufficient when frame to frame overlap comprises a range between 10% and 40%.

46. The one or more processor-readable media of claim 45, wherein the range is between 20% and 30%.

47. The one or more processor-readable media of claim 43, wherein the estimating relative displacement comprises multi-dimensional motion estimating for multiple image pairs.

48. The one or more processor-readable media of claim 35, wherein said method is performed without user intervention.

49. The one or more processor-readable media of claim 35, wherein said method is performed without device motion measurement.

50. The one or more processor-readable media of claim 35, wherein the storing comprises generating a compressed version of the panorama image prior to storing.

51. The one or more processor-readable media of claim 35, wherein the joining of the relatively low resolution images comprises generating an alpha blending map, and wherein the joining of the main series images comprises using said alpha blending map.