Panorama Processing

• US 20120293607A1
• Filed: 05/17/2011
• Published: 11/22/2012
• Est. Priority Date: 05/17/2011
• Status: Abandoned Application

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Bhogal, Nikhil, Doepke, Frank
• Application Number:
  US13/109,875
• Publication Number:
  US 20120293607A1
• Time in Patent Office: Days
• US Class Current: 348/36
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images

Abstract

This disclosure pertains to devices, methods, and computer readable media for performing panoramic photography processing techniques in handheld personal electronic devices. A few generalized steps may be used to carry out the panoramic photography processing techniques described herein: 1.) acquiring image data from the electronic device'"'"'s image sensor'"'"'s image stream; 2.) displaying a scaled preview version of the image data in real-time on the device'"'"'s display; 3.) performing “motion filtering” on the acquired image data; 4.) generating full-resolution and lower-resolution versions of portions of the images that are not filtered out by the “motion filtering” process; 5.) substantially simultaneously “stitching” both the full-resolution and lower-resolution image portions together to create the panoramic scene; and 6.) substantially simultaneously sending the stitched version of the lower-resolution image portions to a preview region on the device'"'"'s display and storing the stitched version of the full-resolution image portions to a memory.

Claims

1. An image processing method, comprising:

obtaining a first image;

displaying a first scaled version of the first image in a first region of a display at a first time;

storing a full resolution version of a central portion of the first image in a memory;

displaying a second scaled version of the central portion of the first image in a second region of the display at the first time;

obtaining a second image;

replacing the first scaled version of the first image in the first region of the display with a first scaled version of the second image at a second time;

stitching a full resolution version of a central portion of the second image together with the full resolution version of the central portion of the first image to generate a first resultant stitched image, the central portion of the first image and the central portion of the second image sharing an overlapping region;

storing the first resultant stitched image in the memory;

stitching the second scaled version of the central portion of the first image together with a second scaled version of the central portion of the second image to generate a second resultant stitched image, the second scaled version of the central portion of the first image together and the second scaled version of the central portion of the second image sharing the overlapping region; and

displaying the second resultant stitched image in the second region of the display at the second time.

2. The method of claim 1, wherein the act of obtaining a first image comprises:

capturing a full-resolution image of a scene by an image sensor; and

storing the full-resolution image in a memory.

3. The method of claim 1, wherein the act of displaying a first scaled version of the first image in a first region of a display comprises displaying a first scaled version of the first image in a first region of a preview display of an image capture device.

4. The method of claim 1, wherein the act of storing a full resolution version of a central portion of the first image in a memory comprises storing approximately 12.5% of the first image in the memory.

5. The method of claim 1, wherein the first resultant stitched image and the second resultant stitched image are generated substantially simultaneously.

6. A program storage device, readable by a programmable control device, comprising instructions stored thereon for causing the programmable control device to perform the method of claim 1.

7. The program storage device of claim 6, further comprising instructions stored thereon for causing the programmable control device to perform the method of claim 5.

8. An electronic device, comprising:

memory;

an image sensor;

a positional sensor;

a display communicatively coupled to the memory; and

a programmable control device communicatively coupled to the memory, display, positional sensor, and image sensor, wherein the memory includes instructions for causing the programmable control device to perform the method of claim 1.

9. The device of claim 8, wherein the memory further includes instructions for causing the programmable control device to perform the method of claim 5.

10. An image processing method comprising:

receiving a stream of images captured by an image sensor in communication with a device, the stream of images comprising a panoramic scene; and

for each received image;

sending a first portion of data representative of the image down a first graphics pipeline for generating and displaying a real-time preview of the image at the device; and

determining whether to filter the image, and, for each image wherein it is determined that the image will not be filtered;

sending a second portion of data representative of the image down a second graphics pipeline for generating a portion of a panoramic preview of the image,

wherein the generated portion of the panoramic preview of the image is stitched to an existing panoramic preview of the image, creating a resultant panoramic preview of the image, and

wherein the resultant panoramic preview of the image is displayed in real-time at the device.

11. The method of claim 10, wherein the determination to filter a particular image is based at least in part on positional information received at the device.

13. The method of claim 10, wherein the panoramic preview of the image is overlaid on the real-time preview of the image.

14. The method of claim 10, wherein the second portion comprises approximately 12.5% of the image.

15. The method of claim 14, wherein the second portion further comprises approximately the central 12.5% of the image.

16. The method of claim 10, further comprising:

generating with the second graphics pipeline a portion of a panoramic image of the scene;

and appending the generated portion of the panoramic image to the panoramic image of the scene,

wherein the panoramic image of the scene has a higher resolution than the panoramic preview of the image.

17. The method of claim 16, further comprising storing the panoramic:

image of the scene to memory.

18. The method of claim 16, wherein the generated portion of the panoramic image has a resolution substantially equal to the full resolution of the camera.

19. The method of claim 16, wherein the panoramic image of the scene has a resolution substantially equal to the full resolution of the camera.

20. The method of claim 16, wherein the real-time preview comprises a scaled version of the image.

21. The method of claim 20, wherein the scaled version of the real-time has a resolution substantially equal to the resolution of a display of the device.

22. The method of claim 16, wherein the generated portion of the panoramic preview of the image has a resolution substantially less than the image.

23. A program storage device, readable by a programmable control device, comprising instructions stored thereon for causing the programmable control device to perform the method of claim 10.

24. The program storage device of claim 23, further comprising instructions stored thereon for causing the programmable control device to perform the method of claim 16.

25. An electronic device, comprising:

memory;

an image sensor;

a positional sensor;

a display communicatively coupled to the memory; and

a programmable control device communicatively coupled to the memory, display, positional sensor, and image sensor, wherein the memory includes instructions for causing the programmable control device to perform the method of claim 10.

26. The device of claim 25, therein the memory further includes instructions for causing the programmable control device to perform the method of claim 16.

12. The method of Maim 10, wherein the real-time preview of the image and the panoramic preview of the image are displayed simultaneously on a display.